The admission
In recent weeks, three law firm CMOs have told By Aries the same thing: they feel behind on AI.
These seasoned leaders have carried their firms through rankings seasons, lateral integrations, mergers, and rebrands. Acknowledging they feel behind takes courage.
By Aries takes that admission seriously, because it points to a real problem. The problem is just not the one most CMOs think they have.
Where “behind” comes from
“Behind” is mostly a myth created by public AI claims. Every firm announcement, conference panel, and vendor demo makes it sound like everyone else has figured it out.
Inside the teams By Aries trains, adoption looks nothing like the press release. Most people use a sliver of the tool, mostly to rewrite emails, and believe they have adopted AI. Many have completed the standard Copilot or ChatGPT training and still use only a small fraction of what the tool can do.
The range inside a single team is often wider than leaders expect. By Aries regularly gives assessments to the teams we consult and train on AI, and not surprisingly, the spectrum of competency and understanding of AI tools and resources that most firms display is varied. Even those firms that have invested heavily in internal trainings and workshops often find the gap is wide among colleagues.
In our last hands-on workshop, where we trained experienced marketers familiar with AI in AI workflows, we had questions about next steps on how to take an existing workflow with clearly defined AI steps and migrate that from paper into the AI tools themselves.
This means that the most significant question isn’t about training, but rather about AI competency.
AI training is not the same as AI competency
It all boils down to this: training shows your team how the tool works, while competence is knowing when to trust it. To help identify where your team is, try looking at these four questions and gauging where your team is.
| The question | What trained looks like | What competent looks like |
|---|---|---|
| Can they spot a soft hallucination? | Accepts a confident line like “across the industry, firms are seeing” because it sounds like a partner wrote it | Flags any claim with no source behind it and strikes it before the draft reaches a lawyer |
| When the AI starts drifting, do they know what to fix? | Rewrites the prompt again and again, hoping the next version works | Diagnoses whether the problem is the prompt, the source files, or the setup and fixes that one thing |
| Can they rerun last month’s prompt and get the same result? | Gets a different output every time and starts over each month | Gets consistent output in your firm’s voice, because the standards live where the AI reads them |
| Can they take a workflow they mapped and build it? | Produces a clear diagram that never becomes a working process | Turns each step into a prompt, project, or skill, with human review marked where it belongs |
Many legal marketers are aware of hallucinations, but how many actually understand and can spot consistently and reliably a soft hallucination? This is the most common question legal marketing teams we train and coach miss on our AI fluency assessment. This same question is also the most serious, because they establish AI as the expert, not your lawyers, and this can be hard for legal marketers to identify quickly in review.
Your team will look to you
Your team will get stuck on these questions. When they do, they will look to you.
In a law firm, that moment rarely stays inside the marketing department. An AI-assisted bio with an invented credential or a client alert with an unsourced claim lands in front of a partner. The partner does not ask which tool produced it. They ask the marketing leader if this is the quality of work they can expect from the marketing and business development team? We all know it’s not.
Lawyers have a formal duty to supervise those who use AI within legal work. ABA Formal Opinion 512 established this. Among other duties, it addresses supervision under Model Rules 5.1 and 5.3: managerial lawyers must set clear policies on permissible AI use, and supervising lawyers must make reasonable efforts to ensure the people they oversee use it responsibly.
Obviously, that opinion governs lawyers, not marketers; however, the principle behind it applies to every CMO: you cannot supervise work you do not understand. And you cannot coach someone through a hallucination you have never had to fix yourself.
Coaching when you are still learning
You do not need to be the best builder on your team; however you do need to have worked through each of the four questions yourself, on real work, at least once. Otherwise you cannot tell a good workflow from a lucky one.
This may sound like a lot of effort when your plate is already full, but it doesn’t have to be. It means one real deliverable using AI, not a certification. The goal is to know what getting stuck feels like, so you recognize it when your team hits the same wall.
From there, coaching looks different than most CMOs expect:
- Ask the next question instead of supplying the answer. When someone brings you a drifting draft, ask whether the problem is the prompt, the files, or the setup. The discernment is the skill you are helping cultivate in them.Â
- Know who needs what. Most of your team needs to use AI well. Only a handful need to build workflows for everyone else. Training all of them to build agents wastes the time of the people who need something simpler like a project or a Copilot Notebook.
- Start resistant people with a task they want off their plate. Your most talented skeptic will not be won over by a vision of transformation. But they may be won over by never formatting that monthly report by hand again.Â
- Work beside them, not above them. Sit next to one person, open a real project, and work through the next question together. That hour shows you how your team works, but also where they get stuck and if you can support them or if you yourself also need additional support.Â
None of this requires you to be ahead of your team. It requires you to be in it with them.
What an AI-competent CMO actually looks like
The most forward-looking CMOs we talk with are planning for a shift most firms have not yet begun to consider. As AI makes legal work faster, lawyers will need more clients to sustain the same revenue. More clients means more pitches, more RFPs, more submissions, and more requests landing on a marketing team that is not getting bigger.
So these CMOs are using AI to build the systems that let their teams keep up with the pace of demand for their services.
Here are five examples of what that looks like in practice.
1. Stop rebuilding what the firm already wrote
Every RFP, pitch, award submission, and bio draws on work the firm has already done. Yet most teams start each one from scratch, hunting for last year’s answers in shared drives, databases, and inboxes. An AI-competent CMO works with their team to build one current source of approved answers, matters, and descriptions, and has AI connected to those sources to assemble the first draft from it.
They also create a process to close the loop ensuring all the latest experience and matter information is up-to-date and to combat revision roulette. Debriefs from won and lost pursuits go back into that source, so each draft is sharper than the last.
Speed is only part of the payoff. When drafting takes minutes, the team spends its time where AI cannot, in conversation with lawyers, drawing out the details that make a submission win.
2. Stop copy and pasting information into your reports
Many CMOs spend hours every month compiling what the team did into updates for firm leadership. These can be in narratives, meeting minutes, or presentations, and it’s a huge waste of manual time.
An AI-competent CMO builds a skill that helps create that report consistently into whatever format it needs to be in: a narrative, a dashboard, a presentation. The updates are pulled from the team’s project management tool, analytics, and CRM, in a format leadership already recognizes. The CMO’s time shifts from CTRL+C and CTRL+V to explaining why all of the work their team does matters.
3. Give lawyers the firm’s AI, not their own
Lawyers are already using AI, often on legal-specific AI tools with none of the firm’s marketing and BD context uploaded. Their drafts miss the brand voice, the positioning, and the approved services, and marketing inherits the cleanup.
An AI-competent CMO sets up a shared firm environment that informs the lawyers’ marketing and BD efforts instead. That starts with a context layer: the firm-specific knowledge AI draws on every time, such as brand standards, practice descriptions, approved matters, and content priorities.
Building it well requires a working understanding of the tools inside the AI platform your firm already licenses:
- Projects hold shared files and instructions for a recurring type of work, so everyone starts from the same material. These are specific to Claude, ChatGPT, and Perplexity.Â
- Notebooks keep answers grounded in the sources you select instead of the open internet. These are specific to Copilot.
- Skills save the firm’s standards and processes once, so AI applies them the same way every time. These are specific to Claude Cowork, ChatGPT Work, Copilot Cowork, and Perplexity Computer.
- Plugins and connectors link AI to the systems the firm already runs on, such as the CRM or document management systems.Â
The CMO does not need to build all of it at once. Starting with the context for one workflow, and getting that right, is what makes the rest possible. By Aries covers how to build a context layer in our latest article.
4. Decide what to build and what to buy
Vendors are adding AI to every product, and many of those products will be acquired, merged, or retired in the next few years. An AI-competent CMO weighs each subscription against what their team could build with the tools the firm already licenses. They put budget into skills that outlast any single platform.
To make this judgement call, an AI-competent CMO is required to understand what AI can actually do. It cannot be handed entirely to a vendor or to IT.
5. Fix the data before expecting insight
AI can only connect what the firm has structured. Duplicate project boards, siloed client data, and inconsistent matter records produce answers that sound confident but are not reliable. An AI-competent CMO treats clean data and clear ownership as the first project, and partners with IT to connect the systems marketing and BD depends on.
Once these foundations are in place, the same systems start surfacing opportunities: client news worth a partner’s call, relationships worth expanding, gaps in how the firm shows up.
None of the five mentioned above are writing or content tasks. Each one changes how the team works, and each one needs a leader who has done enough of the work to judge the result.
What to bring to leadership
Most firms report AI progress as training completions, such as how many people attended and how many licenses went out. Those numbers are easy to collect and say almost nothing about whether the work has improved or the team spent their time doing the work that actually matters.
Instead of garnering support for your next AI training by bringing headlines of what other firms are doing, try actually transforming how your team works. Start by picking one recurring workflow, such as RFP intake, bio updates, or award submissions, and track it for 90 days:
- Time returned: hours the team got back, and where they spent them
- Errors caught: hallucinations and inaccuracies flagged before anything reached a partner
- Work shipped: what went out the door that would not have otherwise
- Adoption that stuck: whether the team was still using the workflow at day 90
One workflow, measured honestly, makes a stronger case than a firm-wide rollout with no data behind it. It also gives you a real answer when leadership asks whether AI is working.
The CMO AI Competency Check
This self-assessment takes about five minutes and is for your eyes only. Score each statement honestly: 0 = not yet, 1 = sometimes, 2 = consistently. Then total each section and your overall score.
1. My own hands-on competence
| Statement | Score |
|---|---|
| I have used AI on a real deliverable this month, not just a test prompt. | |
| I have built at least one workflow myself, from map to working prompt or project. | |
| I have rerun a prompt weeks later and checked whether the result held up. | |
| I know what my firm’s approved tools can do beyond chat, including research modes and projects or notebooks. |
2. Judgment and quality control
| Statement | Score |
|---|---|
| I can spot a soft hallucination, a confident claim with no source behind it. | |
| When output goes wrong, I can tell whether to fix the prompt, the source files, or start over. | |
| I know where human review has to happen in my team’s AI workflows. | |
| I can explain to a partner why an AI-assisted draft is trustworthy, or why it is not. |
3. Coaching my team
| Statement | Score |
|---|---|
| I know who on my team is aware of AI, who is using it well, and who is building with it. | |
| I can sit beside someone who is stuck and ask the next question instead of taking over. | |
| I have a plan for the talented person who resists AI, beyond “they have to.” | |
| My team brings me AI problems early, before they reach a partner. |
4. Systems that make it repeatable
| Statement | Score |
|---|---|
| My team works from shared context (projects, notebooks, brand standards), not private chats. | |
| Our standards are written down where the AI reads them, not only in people’s heads. | |
| At least one workflow runs the same way no matter who on the team runs it. | |
| When someone leaves or goes on leave, their AI work does not leave with them. |
5. Leading upward
| Statement | Score |
|---|---|
| I report on competence (time returned, errors caught, work shipped), not training completions. | |
| I have identified one workflow where AI could show measurable results within 90 days. | |
| I know which AI features my firm already licenses and which are turned off. | |
| I can make the case for AI investment in terms my management committee cares about. |
What your score means
| Total (out of 40) | Where you are | Where to focus next |
|---|---|---|
| 0 to 13 | Getting grounded | Start with section 1. You will coach better once you have done the work yourself. |
| 14 to 27 | Building | You have the pieces but not the system. Focus on your lowest section for the next 90 days. |
| 28 to 40 | Leading | Make it repeatable across the team, and make the results visible to leadership. |
Whatever your total, your lowest section tells you where to grow first. Pick one statement in it and aim to improve it this month.
Your first 30 days
You do not need a transformation plan to start. You need one question a week, one real workflow, and one person from your team working alongside you.
- Week 1, soft hallucinations. Take one recent AI-assisted draft. Highlight every claim with no source behind it. Then do the same exercise with a team member.
- Week 2, drift. Pick a prompt that keeps producing uneven results. Decide together whether the fix belongs in the prompt, the source files, or the setup and change only that.
- Week 3, repeatability. Rerun last month’s version of a recurring task. If the output shifted, write down the standard that was missing and put it where the AI reads it.
- Week 4, the build. Choose one mapped workflow and turn a single step into a working prompt or project, with a human review point marked.
By day 30, you will have done each piece of the work yourself, and one person on your team will have done it with you. That is what coaching looks like in this phase.
Leading through the next phase
The CMOs who told By Aries they feel behind are asking the right question. Feeling behind means you can see the gap between aspirational AI and AI that actually helps you do the work.
Your team does not need you to have every answer. They need you to know the work well enough to sit beside them and ask the next question.
Want to see how your score compares with your team’s? The By Aries AI Fluency Assessment measures where each team member stands, and a team program builds competence on one workflow at a time. Reach out to By Aries to start with your team.
Frequently Asked Questions
What is AI competency?
AI competency is knowing when to trust AI, not just how the tool works. Training shows your team how the tool works. A competent team can spot a soft hallucination, diagnose drift, rerun last month’s prompt and get the same result, and turn a mapped workflow into a working prompt, project, or skill.
How do you measure AI competency on a marketing team?
Ask four questions. Can they spot a soft hallucination? When the AI starts drifting, do they know what to fix? Can they rerun last month’s prompt and get the same result? Can they take a workflow they mapped and build it?
What is a soft hallucination?
A soft hallucination is a confident claim with no source behind it, such as a line like “across the industry, firms are seeing” that sounds like a partner wrote it. It is the question legal marketing teams miss most often on the By Aries AI fluency assessment, and the most serious, because it makes AI the expert, not your lawyers.
What does ABA Formal Opinion 512 say about supervising AI use?
ABA Formal Opinion 512 addresses supervision under Model Rules 5.1 and 5.3, stating that managerial lawyers must set clear policies on permissible AI use, and supervising lawyers must make reasonable efforts to ensure the people they oversee use it responsibly. The opinion governs lawyers, not marketers, but the principle applies to every CMO: you cannot supervise work you do not understand.
How can a CMO show leadership that AI is working?
Report competence, not training completions. Pick one recurring workflow, such as RFP intake, bio updates, or award submissions, and track it for 90 days: time returned, errors caught, work shipped, and adoption that stuck. One workflow, measured honestly, makes a stronger case than a firm-wide rollout with no data behind it.
What is the difference between AI fluency and AI competency?
AI fluency describes how comfortably and how often someone works with AI. AI competency is the judgment underneath it: knowing when to trust the output, and what to fix when it drifts. A team can be fluent in the tools and still miss a soft hallucination.
