We Chose Redshift!
Previous post: PostgreSQL Optimization (Part 3): Turning tags into an index with Array + GIN Why did we move from PostgreSQL to Redshift? The first three posts focused on tuning tag queries. In production, we also need historical analysis, such as comparing how the same group of customers' average balances change each month and how different products compare.Selection factor RDS for PostgreSQL (the GIN proposal) RedshiftHistorical analysis performance GIN can filter records first; when many records match, a large amount of data still needs to be read and aggregated Columnar storage and compression reduce reads, while parallel processing distributes aggregation workMaintenance for daily writes and updates New records and tag updates add GIN maintenance and cleanup work No GIN index to maintain, though loading, updates, and data organization still have costsCompute resources and cost Costs vary with the instance, IOPS, and storage configuration If analysis does not run around the clock, Serverless on-demand billing can reduce idle compute costs; storage is billed separatelyThis dataset grows by around 20 TB a year. Production needs to analyze at least two years of history—40 TB or more—while handling daily writes and updates. 40 TB itself is within the maximum storage specifications for RDS for PostgreSQL. We chose Redshift for its columnar storage and parallel processing for large historical analyses, without needing to build and maintain a GIN index. When analysis has idle periods, Serverless on-demand billing can also reduce compute costs. SUPER arrays in Redshift The previous post proposed converting tags into PostgreSQL's native text[] arrays and adding a GIN index. In Redshift, we store the tags in SUPER, a data type that can hold arrays and objects. It does not provide GIN indexing. At ingestion, SPLIT_TO_ARRAY turns tag1;tag_VIP into ["tag1","tag_VIP"]. Queries can then compare complete tags without splitting the string again, avoiding the extra matches that LIKE '%tag_VIP%' would produce for values such as tag_VIP_plus. CREATE TEMP TABLE customer_tags_demo AS SELECT 1001 AS customer_id, SPLIT_TO_ARRAY('tag1;tag_VIP', ';') AS tags;SELECT DISTINCT c.customer_id FROM customer_tags_demo AS c, c.tags AS tag WHERE tag = 'tag_VIP';c.tags AS tag takes each tag from the array, and tag = 'tag_VIP' checks for a match. The expected result is customer 1001. DISTINCT keeps each customer from appearing more than once. More records and more tags per record mean more work to expand and compare them. What we learned: revisiting the choice with more information I initially chose PostgreSQL because it was the database I knew best, and the pgvector extension supported vector search. After considering production's historical analysis needs, daily writes and updates, and AWS costs together, the team decided to move to Redshift. Further readingAWS: Differences between Amazon Redshift and PostgreSQL AWS: On-demand billing for Redshift Serverless AWS: Querying arrays and objects in SUPER
Read More
Adopting AI: From Improving Existing Tasks to Rethinking How We Work
I Built an HR Knowledge Assistant and a GenBI Assistant. What Comes Next? I've built quite a few AI agents and workflows, including an HR knowledge assistant and a GenBI assistant. The HR knowledge assistant uses RAG to find information in HR documents and answer questions. The GenBI assistant uses Text2SQL so people who don't write SQL can query data themselves, using everyday language. The goal is straightforward: spend less time looking things up and doing repetitive work. But as I worked on these assistants, I started wondering what happens after someone gets the information. With GenBI, someone might use the results to prepare a report for their manager. They might also keep asking questions and explore the data further. The first use helps them finish an existing task faster. The second could help them pursue questions they previously left unanswered because they couldn't query the data themselves. That reflects two expectations I've seen in companies. Some people want AI to take over work they don't enjoy. Others want to try things that used to be too expensive or impractical. The same tool can serve both purposes. Automation has been around for a long time. Tools such as n8n and RPA can already handle plenty of repetitive tasks. When AI enters the picture, I'm more interested in what else it enables people to do. In my own experience, saving time with AI hasn't led me to do more work. If I measure productivity by the amount of work I finish, it hasn't increased much. The clearest benefit has been the time saved. That also makes me want to look further: do these changes help the team get more done? Employees Deliver Faster. There's Still Only One Manager. Daren Academy's Small Talk, EP715 describes a familiar situation: an employee saves several days of work with AI, submits a proposal, and still has to wait until the following week for their manager to review it. As everyone submits more work, faster, the manager still needs time to read it, find problems, and send it back for revisions. So even if employees spend all the time they save on doing more work, they might simply add more proposals to their manager's pile. Daren Academy's Joe suggests making experienced managers' judgment criteria explicit, so the team can use AI to check work before submitting it and reduce revision cycles. That made me think about turning those criteria into a skill: instructions and reference material an agent can use while doing a task. Employees could document the steps they know well. Managers could document their review criteria: what information must be included, which conditions would cause a rejection, and which situations need a person to decide. This is my own extension of the discussion. Run the checks that can happen earlier, then leave the trade-offs to the manager. The criteria also need to be adjusted based on actual results, including any problems they fail to catch. Tech.wav EP160 also discusses trust in agents. Host Harry recounts Lauren Tan's experience: she could ship thousands of pull requests in a single month, and even product managers and designers on the team could use agents to contribute to production features. Harry explains that Lauren Tan put effort into cleaning up the codebase and removing bad practices, then added automated checks, skills, and verification mechanisms. Together, these created an environment where the team felt comfortable relying on agents. That makes me think we also need to take care of the environment in which agents work when we hand tasks over to them. Looking back at the HR knowledge assistant and GenBI, I want to check a few things too. Are the documents and data definitions clear? How should answers and query results be verified? Getting these things right could reduce the need for someone to watch every step. When the Tools Change, the Way We Work Can Change Too An article on ihower's AIhao AI Engineering Blog discusses Walmart's adoption of computers. It describes how sharing data in real time allowed stores and suppliers to make decisions using inventory information, changing how they replenished stock and worked together. The example made me think that when information becomes easier to access, the division of work can change along with it. GenBI helps people cross one barrier by letting them query data without writing SQL. But once they have the data, what can they handle themselves, and what still needs to wait for a manager? That part deserves attention too. I recently heard about business process reengineering: starting with the outcome we want and redesigning how the work gets done. Applied to my own work, that means laying out the process and asking: which steps can we remove? Which checks can happen earlier? Which decisions can be delegated so the work can keep moving? I can imagine companies with hundreds or thousands of agents taking on many of the tasks we do today. If every task they finish still needs approval from the same manager, today's bottleneck will only get worse. Along with turning tasks into skills, we need to decide how agents hand work over, when they should involve a person, and who handles problems. People need to work those things out first. What Should We Do With the Time We Save? I'd like to hand over the work that can be automated to agents, so people have time to focus on what agents still can't do for us, especially communication. When working across departments, having the information ready doesn't mean the other team will agree to help. Each department has its own goals and priorities. You need to understand what matters to them, explain why the work is worth doing, and persuade them to commit time and resources. You still need to have those conversations yourself. The same goes for getting support from your manager and explaining the direction to your team. Agents can help prepare information and organize what you want to say. But addressing people's concerns and working out an approach everyone can accept still takes people's time. Agents can help analyze options and lay out the pros and cons, but we still need to decide what to choose, what we're willing to give up, and which risks we're prepared to take. We also have to take responsibility for the outcome. Those trade-offs can't all be handed over to an agent. Looking back at the HR knowledge assistant and GenBI, I'd like them to save people time searching documents and querying data, leaving them more room to use that information to discuss problems and move things forward. That's what I want to keep thinking about: once an assistant finishes its part, how do people take the work further?Hand over the work that can be automated to agents, and use the time saved for communication, coordination, persuasion, making decisions, and taking responsibility for the outcomes.Referencesihower / AIhao AI Engineering Blog“Stop Reading AI Predictions: Cedric Chin's Sensemaking Trilogy on Making Sense of AI”Article in Chinese Daren Academy's Small Talk, EP715“AI Saves Employees Three Days of Work, but Their Proposals Sit on a Manager's Desk for a Week. Why Does the Process Still Get Stuck When Everyone Is Faster?”Apple Podcasts, in Chinese · YouTube Wikipedia: Business Process ReengineeringEntry in Chinese Tech.wav EP160“Companies Will Change Completely in the AI Era: How Can One Person Create a Hundred Times the Value? Anthropic and SpaceXAI Offer a Glimpse”Apple Podcasts, in Chinese · YouTube
Read More
One AGENTS.md for Multiple AI Coding Agents (Part 1)
AI coding tools will keep arriving. Once Cursor, Claude Code, Codex, and Antigravity each bring a rule file into a repo, the rules often start saying different things after a few months. I now treat the repo as the stable entry point: shared rules live in AGENTS.md, complete task workflows live in .skills/, and tool-specific files only handle discovery or scoped loading. Tools can change while the material the team maintains stays in the repo.Update (2026-08-24): This article now reflects this repo's native skill-discovery adapters. Continue to Part 2 for the implementation details and the Antigravity and Cursor workflows.1. How Four Tools Connect to the Same Repo Rules The entry point differs by tool, but the principle is the same: keep durable, cross-tool guidance in the repo and let tool-specific configuration handle only how it is loaded. Codex: AGENTS.md Is an Instruction Chain Codex reads AGENTS.md before it works. It walks from the project root to the current working directory and layers instructions along the way. A nearer AGENTS.md or AGENTS.override.md appears later in the chain, so it can refine the more general rules above it. This is a good place for repo-wide information:project structure and common commands build, type-check, and test expectations commit and review requirements the canonical source for shared skillsIf src/content/posts/ needs a local constraint, that directory can carry nearer instructions. The root AGENTS.md still gives every agent the same starting point when it enters the repo. Claude Code: Import the Same Instructions From CLAUDE.md Claude Code uses CLAUDE.md as its default entry point. Since the shared repo rules already live in AGENTS.md, CLAUDE.md only needs the official import syntax: @AGENTS.mdThis file only imports the instructions. Changes to commands, verification, or git workflow happen in AGENTS.md, and Claude Code receives the same version. Antigravity: Put Shared Rules Into an Executable Task Context Antigravity is useful for a full task that crosses the editor, terminal, and browser. In this repo, I start a task by asking the agent to read AGENTS.md, then use the skills index to open the matching .skills/<name>/SKILL.md. For an article update, the task can be as bounded as this: Update the Chinese and English posts. Read AGENTS.md and the matching skill first, then run skills:check, npm run check, and npm run build.The agent can then edit, run commands, and launch a preview. I review the resulting plan, screenshots, or browser recordings before deciding on another round. This is an execution-layer convention; the repo still has one copy of the workflow in .skills/. Cursor: Use AGENTS.md for Shared Rules and .cursor/rules for Path Triggers Cursor can use root AGENTS.md as straightforward project instructions. That covers rules that apply across the repo. Add .cursor/rules/*.mdc only when a rule should load for a specific path. For example, a rule can attach the writing workflow when a post is opened or edited: --- description: Apply the blog writing workflow globs: src/content/posts/**/*.md alwaysApply: false ---Read and follow `.skills/tech-blog-polish/SKILL.md` before editing this post.This adapter answers when to load the workflow. It does not copy the article structure or tone rules. Cursor keeps its file-path behavior while the workflow remains in one place. 2. A Minimal Shared Strategy for the Repo I separate rules into three layers: AGENTS.md holds stable instructions, .skills/ holds complete task workflows, and tool folders hold generated or path-scoped adapters. Each file has one responsibility, and a review can quickly tell where a rule belongs. Luke-Tech-Blog/ ├── AGENTS.md # project instructions ├── CLAUDE.md # imports AGENTS.md ├── .skills/ # complete, canonical workflows ├── .claude/skills/ # generated native-discovery adapters ├── .agents/skills/ # generated native-discovery adapters └── .cursor/rules/ # optional file-path triggersKeep AGENTS.md short and stable: ## Repository Expectations- Keep the complete workflow only in `.skills/<skill-name>/SKILL.md`. - `.claude/skills/` and `.agents/skills/` are generated native-discovery adapters. - Run `npm run skills:sync` after changing skill frontmatter.Put the details that evolve repeatedly in .skills/: the reader profile, structure, and voice for article polishing, or the check sequence and commit rules for git review. When Claude Code and Codex need native discovery, generate and check the adapters: npm run skills:sync npm run skills:checkThe first command updates generated files; the second only verifies their state. Drift becomes a clear failure that can be caught before a commit or in CI.A shared rule set lasts when every tool carries only the entry point it needs.Conclusion: Converge the Rules, Then Use Each Tool Well The four tools have different strengths. Codex and Claude Code connect directly to shared instructions. Antigravity can execute a longer task and leave inspectable artifacts. Cursor can attach rules automatically by file path. Once AGENTS.md and .skills/ are established as the single sources of truth, additional tool integrations stop becoming a document-copying project. The next post returns to this blog repo and shows the native discovery adapters for Claude Code and Codex, plus the Antigravity and Cursor workflows that use the same source.One AGENTS.md for Multiple AI Coding Agents (Part 2)ReferencesOpenAI Codex: Custom instructions with AGENTS.md Claude Code: How Claude remembers your project Cursor: Rules Google Antigravity: agentic development platform
Read More
One AGENTS.md for Multiple AI Coding Agents (Part 2)
In the previous post, I summarized the common direction from the official docs: AGENTS.md can be the shared project instruction entry point for a repo. This post shows how I made Claude Code and Codex discover the same skills natively.One AGENTS.md for Multiple AI Coding Agents (Part 1)Update (2026-08-24): This repo now includes commit-ready native discovery adapters. .skills/ remains the only full workflow source; .claude/skills/ and .agents/skills/ contain generated entry files only.The goal is direct:maintain project instructions in one place maintain task workflows in one place keep tool-specific files as adapters check shared team rules into gitWith this shape, switching tools does not mean rewriting the repo rules. I split the implementation into two parts: native discovery for Claude Code and Codex, then practical ways for Antigravity and Cursor to use the same repo rules. 1. Claude Code and Codex: One Workflow, Native Entry Points AGENTS.md already lets an agent find .skills/. Native discovery makes the handoff smoother: when a tool scans its own skills directory, it can identify the matching workflow without a reminder in every prompt. The finished structure looks like this: Luke-Tech-Blog/ ├── AGENTS.md ├── CLAUDE.md ├── .skills/ │ ├── README.md │ ├── tech-blog-polish/ │ │ └── SKILL.md │ ├── git-change-commit-review/ │ │ └── SKILL.md │ └── linkedin-blog-promo/ │ └── SKILL.md ├── .claude/ │ └── skills/ │ ├── tech-blog-polish/ │ │ └── SKILL.md │ ├── git-change-commit-review/ │ │ └── SKILL.md │ └── linkedin-blog-promo/ │ └── SKILL.md ├── .agents/ │ └── skills/ │ ├── tech-blog-polish/ │ │ └── SKILL.md │ ├── git-change-commit-review/ │ │ └── SKILL.md │ └── linkedin-blog-promo/ │ └── SKILL.md └── scripts/ └── sync-skills.mjsAGENTS.md remains the project-instruction entry point for commands, verification, project structure, and git workflow. CLAUDE.md still imports it with one line: @AGENTS.mdThe full task workflow lives in .skills/<skill-name>/SKILL.md. The matching files in the two tool directories are adapters, so Claude Code and Codex can find that same workflow through their native discovery paths. Write the Full Workflow Once The adapter is deliberately short. For tech-blog-polish, the generated file contains its frontmatter plus a pointer to the canonical source: --- name: tech-blog-polish description: Polish technical blog articles for this project... ---# Canonical SkillThe complete workflow is in `.skills/tech-blog-polish/SKILL.md`. Read that file first.This avoids copying the complete article-polish and git-review rules three times. Every update happens in .skills/<skill-name>/SKILL.md. AGENTS.md makes the division of responsibility explicit: Keep the complete workflow only in `.skills/<skill-name>/SKILL.md`. `.claude/skills/` and `.agents/skills/` are generated native-discovery adapters..skills/ is the only source of full workflows. Native adapters only help tools find it.Generate Adapters and Check for Drift Windows symlinks can still vary by permissions and developer setup. This repo uses a small Node.js script, scripts/sync-skills.mjs, instead of depending on symlinks. The script reads each .skills/ subdirectory, confirms that SKILL.md has name and description frontmatter, then writes a small adapter to both native discovery locations: .skills/<name>/SKILL.md ├─> .claude/skills/<name>/SKILL.md └─> .agents/skills/<name>/SKILL.mdRun this after editing a canonical skill: npm run skills:syncUse this before a commit or in CI: npm run skills:check--check writes nothing. If an adapter is stale, it lists the files that need synchronizing and exits with an error. A forgotten sync step is now a reproducible failure instead of a quiet documentation mismatch. 2. Supplement: How Antigravity and Cursor Fit In These tools do not need another copy of a skill. They fill different layers of the workflow: Antigravity runs a task across the editor, terminal, and browser; Cursor attaches rules at the moment a particular file path matters. Antigravity: Start at AGENTS.md, Then Hand the Whole Task to an Agent When I open this repo in Antigravity, I start by having the agent read the root AGENTS.md. Its skills index points to .skills/, and the agent reads the matching canonical SKILL.md when a task calls for it. Project instructions, task workflow, and tool execution stay in separate, understandable places. For an article update, I can give the agent a bounded instruction such as: Update the Chinese and English Part 2 posts. Read AGENTS.md and the matching skill first, then run skills:check, npm run check, and npm run build.Antigravity's Manager Surface is a natural fit for a longer task that needs edits, terminal commands, and a preview. I review the plan, screenshots, or browser recordings it produces before deciding on another round of changes. The repo does not need a separate copy of .skills/ for that flow. Cursor: Reserve .cursor/rules for File-Path Triggers Cursor can also read root AGENTS.md, so repo-wide rules still have one source. I add a short adapter in .cursor/rules/ only when a rule should load because of a file path. For example, when a blog post is opened or edited, Cursor can attach the writing workflow: --- description: Apply the blog writing workflow globs: src/content/posts/**/*.md alwaysApply: false ---Read and follow `.skills/tech-blog-polish/SKILL.md` before editing this post.English posts can use the same pattern with globs targeting src/content/posts-en/**/*.md. The .mdc file only decides when to load the workflow. Its structure, tone, and acceptance details still live in .skills/tech-blog-polish/SKILL.md. With that split, Cursor's auto-attached rules answer when to load a workflow, Antigravity's agent task handles the end-to-end execution, and Claude Code and Codex use native adapters to discover the skill. All four tools return to the same workflow source. Conclusion: Let Tools Find Rules Without Duplicating Them AI coding tools will keep changing, and so will their discovery directories. Copying full workflows into each tool folder creates maintenance work that compounds over time. This repo keeps the complete content in .skills/, generates native adapters for Claude Code and Codex, and uses skills:check to protect that relationship. Adding another tool now means adding one thin adapter, not moving a whole workflow.Native discovery can vary by tool. A shared workflow should have one source.ReferencesOpenAI Codex: Agent Skills Anthropic: Agent Skills Cursor: Rules Google Antigravity: agentic development platform
Read More
Using AI Coding Agents Well (Part 1): When Branches Are Not Enough for Multi-Agent Work
AI Coding Agents are no longer just tools that help fill in a few lines of code. I personally pay for several AI coding tools, including Cursor, Anthropic Claude Code, and OpenAI Codex. Their interfaces are different, but their capabilities are moving in a similar direction: they can read a project, edit files, run tests, fix errors, and spend time completing a task that has a clear goal. That changed how I use them. I used to keep AI inside the IDE, like an assistant I could call whenever I needed help. Later, I started treating an Agent more like someone who could take a task and work on it for a while. That shift sounds small, but it changes the whole workflow.Once an Agent can work for a long time, we have to manage more than instructions. We also have to manage which copy of the project files it is using.This is Part 1 of my “Using AI Coding Agents Well” series. I want to start with a real problem I ran into: once I began letting an Agent run longer tasks, my old habit of switching Git branches quickly stopped being enough. Starting With Local Edits My first way of working with AI was probably similar to many developers: select a piece of code in the IDE, then ask AI to change that specific part. For example:refactor this function add TypeScript types explain this logic change this API call to another style add testsThis workflow is natural and useful. The unit of work is small, so the risk is easy to control. After AI makes the change, I check the diff, accept it if it looks good, or reject it if it does not. At this stage, the workflow is simple: I know exactly what needs to change, and AI handles that small piece for me. It works like an accelerator for the task already in my hands. When a task only takes a few minutes, this is enough. I still understand the current state of the project, which files changed, and whether I want to keep the result. Long-Running Tasks Changed the Problem Later, while doing Prompt Engineering work, I found a type of task that fits Agents especially well: long-running evaluation. For example:running a batch of prompt evaluation cases comparing outputs from different prompt versions adjusting evaluator or LLM-as-a-judge logic changing prompts based on test results organizing traces or evaluation reportsThese tasks usually do not end after one small edit. The Agent has to read the project, run commands, inspect results, change files, and verify again. A task may grow from a few minutes to half an hour or longer. The problem appears while waiting. If one Agent is running prompt evaluation, I may still want to keep building a new feature, or ask another Agent to fix a bug. My thought at the time was straightforward:Can I let one Agent handle a long-running task in the background while I keep developing on another branch?The obvious answer was to create a new branch. Branches Look Like the Right Tool That is what I tried first. Suppose I have one main project folder: luke-tech-blog/I can create a branch for the long-running task: git checkout -b agent/prompt-evalThen I let the Agent run prompt evaluation on that branch. If you are not familiar with Git, think of a branch as a different path for the same project. One path is for prompt evaluation, another path is for a new feature. That sounds reasonable. Next, I wanted to continue building a new search page: git checkout -b feature/search-pageThat was when the issue became obvious. One project folder can only show the files from one branch at a time. When I switch from agent/prompt-eval to feature/search-page, the files in that folder also switch to another version. That means the Agent running the long task suddenly sees a different set of project files. For a human, this is easy to understand: I just switched branches, so the files changed. For the Agent that is still working, it only sees that the files in its folder suddenly changed. The content it just read, the tests it just ran, and the file it was about to modify may no longer match the new state. Imagine two people sharing the same desk. Agent A is organizing one report. I suddenly replace every document on the desk with documents from another project. Agent A is still sitting there, but the things in front of it have changed.Branches can separate different project paths, but they do not give each Agent its own desk.The Real Problem Is Workspace Separation That experience changed how I understood the problem. The structure I needed was:one project different branches different folders all folders can exist at the same time switching one folder does not affect another AgentIdeally, the project should look like this: luke-tech-blog/ mainluke-tech-blog.prompt-eval/ agent/prompt-evalluke-tech-blog.search-page/ feature/search-pageThen I can assign tasks to different folders: Agent A -> luke-tech-blog.prompt-eval Agent B -> luke-tech-blog.search-pageEach Agent has its own branch and its own folder. The long-running task will not be interrupted when I switch branches elsewhere, and different Agents will not mix their changes in the same place. This is the first practical constraint of multi-Agent collaboration: if tasks can run in parallel, workspaces also need to run in parallel. In the next post, I will introduce the solution I moved to: Git worktree. It lets the same project open different branches in different folders, giving each Agent its own desk. ConclusionOnce an Agent starts handling long-running tasks, each Agent needs a working folder that will not suddenly be replaced by someone else.Using AI Coding Agents Well (Part 2): Give Each Agent Its Own Workspace With Git Worktree
Read More
Want to know more about my technical background?
See About for my background, strengths, and working style.
About me