[Bug] 2.0.0 Errors when components contain multi-byte characters (e.g. emoji) 💩 (not always, though)

[NullVoxPopuli]

🐞 Describe the Bug

I recieved a syntax error, even though I was not expecting one.

🔬 Minimal Reproduction

the file

import Component from '@glimmer/component';
import { on } from '@ember/modifier';

import { getSnippetElement, toClipboard, withExtraStyles } from './copy-utils';
import Menu from './menu';

/**
 * This component is injected via the markdown rendering
 */
export default class CopyMenu extends Component {
  copyAsText = (event: Event) => {
    let code = getSnippetElement(event);

    navigator.clipboard.writeText(code.innerText);
  };

  copyAsImage = async (event: Event) => {
    let code = getSnippetElement(event);

    await withExtraStyles(code, () => toClipboard(code));
  };

  <template>
    <Menu data-test-copy-menu>
      <:trigger as |t|>
        <t.Default class="absolute top-3 right-4 z-10" data-test-copy-menu>
          📋
        </t.Default>
      </:trigger>

      <:options as |Item|>
        <Item {{on "click" this.copyAsText}}>
          Copy as text
        </Item>
        <Item {{on "click" this.copyAsImage}}>
          Copy as image
        </Item>
      </:options>
    </Menu>
  </template>
}

😕 Actual Behavior

the error:

app/components/limber/copy-menu.gts
[error] app/components/limber/copy-menu.gts: SyntaxError: Unexpected token (41:1)
[error]   39 |     </Menu>
[error]   40 |   </template>
[error] > 41 | }
[error]      | ^
[error]   42 |

🤔 Expected Behavior

no error

🌍 Environment

• prettier-plugin-ember-template-tag version: - 2.0.0-0
• ember-template-imports version (if applicable): - n/a
• content-tag version (if applicable): - n/a
• eslint-plugin-ember version (if applicable): - n/a

➕ Additional Context

prettier 3.1

[gitKrystan]

I have a reproduction of this and I'm digging into the code. It seems like content-tag might not be giving me the range I expect here:

#=> code
'import Component from '@glimmer/component';
import { on } from '@ember/modifier';

import { getSnippetElement, toClipboard, withExtraStyles } from './copy-utils';
import Menu from './menu';

/**
 * This component is injected via the markdown rendering
 */
export default class CopyMenu extends Component {
  copyAsText = (event: Event) => {
    let code = getSnippetElement(event);

    navigator.clipboard.writeText(code.innerText);
  };

  copyAsImage = async (event: Event) => {
    let code = getSnippetElement(event);

    await withExtraStyles(code, () => toClipboard(code));
  };

  <template>
    <Menu data-test-copy-menu>
      <:trigger as |t|>
        <t.Default class='absolute top-3 right-4 z-10' data-test-copy-menu>
          📋
        </t.Default>
      </:trigger>

      <:options as |Item|>
        <Item {{on 'click' this.copyAsText}}>
          Copy as text
        </Item>
        <Item {{on 'click' this.copyAsImage}}>
          Copy as image
        </Item>
      </:options>
    </Menu>
  </template>
}
'
#=> templateNode.contents
'
    <Menu data-test-copy-menu>
      <:trigger as |t|>
        <t.Default class='absolute top-3 right-4 z-10' data-test-copy-menu>
          📋
        </t.Default>
      </:trigger>

      <:options as |Item|>
        <Item {{on 'click' this.copyAsText}}>
          Copy as text
        </Item>
        <Item {{on 'click' this.copyAsImage}}>
          Copy as image
        </Item>
      </:options>
    </Menu>
  '
#=> code.slice(templateNode.range.start, templateNode.range.end)
'<template>
    <Menu data-test-copy-menu>
      <:trigger as |t|>
        <t.Default class='absolute top-3 right-4 z-10' data-test-copy-menu>
          📋
        </t.Default>
      </:trigger>

      <:options as |Item|>
        <Item {{on 'click' this.copyAsText}}>
          Copy as text
        </Item>
        <Item {{on 'click' this.copyAsImage}}>
          Copy as image
        </Item>
      </:options>
    </Menu>
  </template>
}' <-----🐛 SEEMS LIKE A BUG 🐛

[NullVoxPopuli]

With slice, on't you need to -1 with the end index? Or do end-start? Something was length based, yeah?

[gitKrystan]

I don't think that's the case. I have lots of tests for similar component code and they all work. If I try - 1, 💣 💥 😵...

Speaking of emojis, here's the minimal repro 😂 😂 😂

import Component from '@glimmer/component';

class MyComponent extends Component {
  <template>
    💩
  </template>
}

I opened embroider-build/content-tag#45 since I think the fix lives there.

[NullVoxPopuli]

I think the fix could still be here depending on discussions from the meeting this upcoming Tuesday.

We've since learned that Rust's indices are byte-indices , rather than character-indices

[Techn1x]

I hit this bug on prettier-plugin-ember-template-tag v2.0.0, so I suspect this isn't quite fixed

One example;

test('it encodes login for URI', async function (assert) {
  const student = make('student', {
    firstName: 'first_name',
    // login: 'abc다윤6', // THIS FAILS (even if remaining as a comment, any use of <template> after this line and prettier explodes)
    login: 'abc\uB2E4\uC7246', // THIS WORKS
  })

  await render(<template><LoginCard @student={{student}} /></template>)
  ...

Error

app/components/login-card/component-test.gjs
[error] app/components/login-card/component-test.gjs: Error: failed to process all templates, 1 remaining
[error]     at convertAst (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/prettier-plugin-ember-template-tag@2.0.0_prettier@3.1.1/node_modules/prettier-plugin-ember-template-tag/dist/prettier-plugin-ember-template-tag.js:70060:11)
[error]     at Object.parse (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/prettier-plugin-ember-template-tag@2.0.0_prettier@3.1.1/node_modules/prettier-plugin-ember-template-tag/dist/prettier-plugin-ember-template-tag.js:70079:5)
[error]     at async parse (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/prettier@3.1.1/node_modules/prettier/index.mjs:19362:11)
[error]     at async coreFormat (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/prettier@3.1.1/node_modules/prettier/index.mjs:20683:7)
[error]     at async formatWithCursor (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/prettier@3.1.1/node_modules/prettier/index.mjs:20885:14)
[error]     at async formatFiles (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/prettier@3.1.1/node_modules/prettier/internal/cli.mjs:7073:18)
[error]     at async main (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/prettier@3.1.1/node_modules/prettier/internal/cli.mjs:7279:5)
[error]     at async Module.run (file:///Users/boverton/Blake/teacher-ui-client/node_modules/.pnpm/prettier@3.1.1/node_modules/prettier/internal/cli.mjs:7225:5)

I also have an example of the same bug in a non-test GTS file, using the endash character. This also blows up in the same way

  get rows(): LessonGuideRow[] {
    const guides = {
      [Number]: '1–3',
      [Fractions]: '1',
      [Operations]: '1–6',
      [Geometry]: '1–4',
      [Measurement]: '1–5',
    }
  }
  
  <template>...</template>

Would you like a new issue opened?