How well does AI handle Ghost's handlebars dialect?

I fielded a question over on the Ghost forum from a user who wanted to match tag slugs that start with a phrase. It seemed like a good time to bust out AI and see how it'd do.

Can AI write valid Ghost handlebars?

Try one (GPT-4o):

The provided code doesn't work. No matches. I also tried pulling out the #match helper from the if, and no, it doesn't like the "^fred" (nothing matches).

I've never seen a theme with a helpers directory, and Ghost definitely does not load helpers from anywhere in theme-land.

Try 2 (I prompted it that it needed to make a #get request.)

I wasn't a huge fan of the idea that we'd get all the tags and then implement filtering. So I tried again. Nope, not better.

Is o3 any better? I asked it to fix the last answer.

Yeah, no. Same problem as before. Themes don't get helper folders, and they definitely don't cause Ghost core to automatically load helpers. It went on to tell me how to use the helper. Which might have worked (inefficiently), if you could actually get it installed.

I saw the heading below and got excited! Finally, AI is going to get it right! Er...

Yeah, ok, so the filter expression is almost right, but 'fred' needs to be in quotes (see below). What's hilarious is that it suddenly knew that you could use a ~, but didn't know that ~^ was a thing? So then it still proposed using its own helper.

I nudged it again, thinking maybe it just needed a hint...

Which doesn't work. (I also added single quotes. Still no good.)

What it should have said (the right answer):

{{#get "tags" filter="slug:~^'fred'" as |fred|}}
  {{#foreach fred}}
   <p>Got a tag named {{name}}</p>
  {{/foreach}}
{{/get}}

Addendum

OK, so I was typing in the ChatGPT box to get the AI responses above. Maybe Copilot integrated in VS Code will do better?

I wrote a boilerplate request for tags, then asked Claude 3.5 Sonnet (preview) how to do add the filter.

I got this:

{{#get "tags" filter="slug:starts:fred" as |tags|}}
  {{#foreach tags}}
    <a href="{{url}}">{{name}}</a>
  {{/foreach}}
{{/get}}

Uh... no? There's no syntax like that.

I asked Gemini 2.0 Flash (Preview). I got filter="slug:^fred".

I left the correct solution in my post.hbs, and pretended to start another page. I wrote the comment at the top, and the rest of this was AI autocomplete:

{{!-- make a custom tags page that only shows tags starting with the string fred --}}

{{!-- make a custom tags page that only shows tags starting with the string fred --}}
{{#get "tags" filter="slug:~^'fred'" limit="all" as |tagsData|}}
  {{#foreach tagsData}}
    <p>{{name}}</p>
  {{/foreach}}
{{/get}}

{{!-- Everything inside the #post block pulls data from the post --}}
{{#post}}

The first part is spot on, because it's a direct copy from my post.hbs. The second part? Well, it's also a copy from post.hbs, and shows one of my biggest complaints about Copilot used for autocomplete - it tends to get carried away!

Here we are again, same repo but a different file (I typed the h2 line only.)

<h2>All the tags on the site that start with LANG</h2>
{{#get "tags" filter="slug:~^'LANG'" limit="all" as |tagsData|}}
  {{#foreach tagsData}}
    <p>{{name}}</p>
  {{/foreach}}
{{/get}}
{{/post}}

{{#post}}
    <section class="article-comments gh-canvas">
        {{comments}}
    </section>
{{/post}}
{{/post}}

{{#post}}
    <section class="article-comments gh-canvas">
        {{comments}}
    </section>

{{/post}}

The first part is good and actually shows a little bit of generalizing, but the second part is... uh? Why did I want comments? Twice?

Wait, so did it just learn how to do this kind of filter expression? I switched over to a different theme folder (so a different VSCode workspace), typed the comment, and got:

{{!-- get a list of all tags that start with 'recipe' --}}
{{#get "tags" filter="slug:^recipe" as |recipeTags|}}
    {{#foreach recipeTags}}
        {{name}}
    {{/foreach}}
{{/get}}

So yeah, if it doesn't have the right answer in the code base, it can't find it.

I write a lot of handlebars. I guess at some point I should figure how to get Copilot more examples of valid handlebars in whatever workspace I happen to be in.

Or maybe I should just have it read my blog drafts, since the answer was there (and is now published):

Ghost filter tricks

Using NQL syntax to do more with filtering

Spectral Web ServicesCathy Sarisky, Ghost Expert

Is AI coming for my job?

Doesn't look like it today, anyway!

Hey, before you go... If your finances allow you to keep this tea-drinking ghost and the freelancer behind her supplied with our hot beverage of choice, we'd both appreciate it!

Buy me a tea ☕️