zerowidth positive lookahead

Managing projects in Obsidian

How I manage projects, tasks, and progress notes in Obsidian using Dataview to tie everything together.

Years ago I wrote about tracking tasks in Things.app. A lot has changed, including several iterations of paper and digital systems.

Recently I’ve been using Obsidian as a knowledge base and task and project tracking system. My Obsidian vault is roughly organized using the PARA method, along with a daily note folder.

The things I need from my PKM and task management system include:

  • Daily notes to serve as both running TODO lists and a log of what I’ve done
  • Project/Area documents to capture active work and a log of activity and notes
  • Meeting notes: who, what, when, what did we talk about
  • General reference documents for notes, ideas, quotes, shopping lists, TODOs

A work log helps as a reference at several levels, from “what did I do yesterday?” to “what did I do last week?”, to “what did I accomplish last quarter?”.

Tasks, activity logs, and dated notes

Daily notes

Daily notes are titled YYYY-MM-DD ddd for simple lexicographical sort plus day of week for human reference. On a given day, I’ll add tasks from various open projects or note things that I want to do throughout the day. This is a simple task list:

- [x] Do a thing
- [ ] Review PR xyz#123

If there’s running commentary that I want to log, I’ll include that as text:

Spent some time debugging xyz system.

To get a list of processes listening on port 4567,
use `sudo netstat -lntp | grep 4567`.

Talked with [[Colleague]] about how to update the schema for abc.

That covers smaller or one-off tasks, but most of my work is under the umbrella of projects or areas.

Projects and areas

From PARA, projects are sets of tasks with a defined outcome and areas are ongoing areas of responsibility. I use documents in both places to track and log tasks and running notes.

These documents look like:

## References

- links to relevant GitHub issues

## Log

### [[2022-09-29 Thu]]

- [x] implement thing
- [x] Review PR xyz#120

Some inline notes for the day.

- [x] Open PR xyz#124

### [[2022-09-30 Fri]]

- [x] deploy PR xyz#124 to production
- [x] work with database team to recover the database 
- [x] fix the bug that drops the production database

## Next

- [ ] buy donuts for the database team
- [ ] ship xyz#124

The “Log” heading has subheads for each day, each linking to the daily note, with tasks and notes beneath it.

The “Next” heading contains the next tasks for that project or area. I append new tasks either to the current day’s subhead if I expect to get to it that day or in “Next” if I don’t.

My system requires manual shuffling to move uncompleted tasks from the daily note or daily log subheadings. It hasn’t been burdensome, but this could be something a computer would be good at.

Other daily activity

For miscellaneous reference notes or ideas, I’ll drop either a single dated line:

- [ ] [[2022-09-30 Fri]] would old boring technology work for this instead?

Or add a full dated section:

### [[2022-09-30 Fri]]

- [ ] some tasks I thought of
- [ ] etc.

Some notes or things I was thinking about.

What did I do today?

It’s nice that each log date header links back to the daily note. I can find every reference for a day with backlinks, but there isn’t an easy way to see all the tasks or related notes from that day.

Along with simple markdown and wikilinks for files and cross-referencing, Obsidian has many plugins to enhance its behavior. Of these, the most powerful is Dataview. It indexes page links and tasks for simple queries. It also has a javascript API for advanced querying, filtering, and text processing.

Displaying a list of the tasks I completed on a particular day might look like:

```dataviewjs
// find all tasks with a subpath (section heading)
// that matches the current file (daily note) name
const tasks = dv.pages().file.tasks
  .where(t => t.section.subpath == dv.current().file.name)

if (tasks.length > 0) {
  dv.header(2, "Project tasks")
  dv.taskList(tasks, true)
} else {
  dv.el("p", "_No project tasks found_")
}
```

That’s a good first pass, but this only finds tasks in the ### [[date]] subheads and doesn’t include any text notes.

I would like to include the entire contents of any note created for a given day, e.g. a meeting note “2022-09-30 Meeting with ABC team to discuss new event architecture”.

First, though, reusability: instead of copying a big chunk of dataview javascript into every daily note, I can put the code in a js file in my vault and reference it instead. Every daily note ends with:

```dataviewjs
await dv.view("dataview-daily")
```

And the contents of dataview-daily.js:

// find headers that link to this daily note
const headerPattern =
  new RegExp("^(#{1,6})\\s(.*" + dv.current().file.name + ".*)");
// find any header
const genericHeader = new RegExp("^(#{1,6})\\s.*");
// the YYYY-MM-DD portion of this daily note's filename
const date = dv.current().file.name.split(" ")[0];

// reindent subheaders to the next level down from the given header depth:
function reindent(lines, depth) {
  return lines.map((line) => {
    let match = line.match(genericHeader);
    if (match) {
      headerDepth = match[1].length;
      return "#".repeat(depth - headerDepth + 1)+line;
    } else {
      return line;
    }
  });
}

// get all the unique pages that link to this daily note
// or start with today's date
// and sort by last modified time
let today = dv.pages()
  .where((p) => p.file.name.startsWith(date))
  .filter((p) => p.file.path != dv.current().file.path);
let pages = dv.current()
  .file.inlinks 
  .map(i => dv.page(i.path))
  .concat(today)
  .distinct((p) => p.file.path)
  .sort((p) => p.file.mtime, "asc");

dv.el("h2", "Related");

let rendered = false;

// for each page:
for (let page of pages) {
  // load the text
  let text = await dv.io.load(page.file.path);
  // mentions are inline links, "- [[date]] some note"
  let mentions = [];
  // headers are "### [[date]]" log headers
  let headers = [];
  let lines = dv.array(text.split("\n"));

  // first, if this page is a note that was created on this day
  // e.g. a meeting note titled "<date> Meeting - about things"
  // include the entire contents inline
  if (page.file.name.startsWith(date)) {
    // add a header with the page title and link
    dv.header(3, dv.fileLink(page.file.path));
    // include all the contents, but reindent headers
    // to match the outer indentation
    dv.el("div", reindent(lines, 3).join("\n"));
    continue;
  }

  // otherwise, look for mentions and headers
  lines.where(l => l.includes(dv.current().file.name))
    .forEach((line) => {
      if (line.match(headerPattern)) {
        headers.push(line);
      } else {
        mentions.push(line);
      }
    });

  if (mentions.length > 0 || headers.length > 1) {
    if (mentions.length > 0) {
      // link to the page but not a section in it
      // (there is no subhead to link to)
      dv.header(3, dv.fileLink(page.file.path));
      // and render the mentioned lines directly
      dv.el("div", mentions.join("\n"));
    }
    // error case, I must have done something weird in my note:
    if (headers.length > 1) {
      dv.header(3, dv.fileLink(page.file.path));
      dv.el("p", headers.length + " headers matched!");
    } 
  }
  if (headers.length > 0) {
    // only processing the first one that matches
    header = headers[0];
    let currentHeader = lines.findIndex((line) => line.match(headerPattern));
    // figure out what header level we're at (h2, h3, etc)
    const depth = header.match(headerPattern)[1].length;
    const text = header.match(headerPattern)[2];

    // link to the current header section
    dv.header(3, dv.sectionLink(page.file.path, text));
    if (currentHeader < 0) {
      dv.el("p", "_couldn't find matching header, something went wrong_");
      console.log("couldn't find matching header linking to",
        dv.current().file.name, "in", page.file.path);
      continue;
    }
    // figure out when the next sibling header begins
    let nextHeader = lines.findIndex((line) => {
      const match = line.match(genericHeader);
      return match && match[1].length <= depth;
    }, currentHeader + 1);
    if (nextHeader < 0) {
      nextHeader = lines.length;
    }
    // include every line between the current and next header
    dv.el("div",
      reindent(lines.slice(currentHeader + 1, nextHeader), 3).join("\n"));
  }

  if (mentions.length > 0 || headers.length > 0) {
    rendered = true;
  }
}

if (!rendered) {
  dv.el("p", "_no related notes_");
}

Not pretty but it works. It pulls in everything related to the current daily note and renders it inline:

  • any one-line mention of the current day
  • any log sections from projects for the current day
  • the contents of any notes (meetings) created for this day

Neat.

What do I work on next?

I could page through my various project notes every morning to find out what tasks are in each ## Next section, or… yep, more dataview, this time in a “What’s Next” document.

My ## Next sections might have subheads:

## Next

- [ ] generic next task

### Database changes

- [ ] create new table

### API changes

- [ ] create REST endpoint for the table

A simple dataviewjs query for “all tasks under the ‘Next’ heading” isn’t enough. Let’s list the next few open tasks from every “Next” and subheading from documents in the “Projects” or “Areas” folders:

```dataviewjs
await dv.view("dataview-next", {query: "Projects OR Areas"})
```

And dataview-next.js:

input = input || {};
const taskLimit = input.taskLimit || 3;
const headerPattern = /^#{1,6}\sNext\s*$/;

let pages = dv
  .pages(input.query)
  .sort((p) => p.file.mtime, "desc");
if (input.filter) {
  // optional filter on page attributes
  // e.g. p => status == "in progress"
  pages = pages.where(input.filter);
}

for (let page of pages) {
  let text = await dv.io.load(page.file.path);
  let lines = text.split("\n");
  // find the "Next" header
  let nextLine = lines.findIndex((line) => line.match(headerPattern));
  if (nextLine < 0) {
    nextLine = lines.length; // not found, skip to end
  }
  // find and sort the tasks by subheader,
  // putting the bare "Next" tasks first.
  let next = page.file.tasks
    .where((t) => !t.completed && t.line > nextLine)
    .groupBy((t) => dv.sectionLink(page.file.path, t.header.subpath))
    .map((group) => {
      return { key: group.key, rows: group.rows.limit(taskLimit) };
    }).sort((g) => g.key.subpath == "Next" ? "AAAAAA" : g.key.subpath);
  if (next.length > 0) {
    dv.taskList(next, false);
  } else {
    dv.header(4, dv.fileLink(page.file.path));
    dv.el("p", "_No next tasks_");
  }
}

This renders the next three open tasks from every “Next” heading and subheading in one place. The query and filter options let me scope searches by search query or page metadata. For active projects it would look like {filter: p => p.status == "in progress"} or scoped by company/area {filter: p => p.scope == "work"}.

I’m using this for a “what’s next at work?” document and a separate listing for personal projects.

More or less

That’s two ways I use dataview. It can be as complex as what I wrote above, or as simple as a list of current projects:

```dataview
LIST WHERE status = "in progress"
```

There’s plenty of flexibility for whatever your system might be. And always more to automate with plugins like QuickAdd and Templater.

Update 2022-10-17 Added reindent() helper for daily link, fixed “find all files named starting with today’s date” functionality.