Skip to content

Commit 0c8424c

Browse files
pks-tgitster
authored andcommitted
t: work around multibyte bug in quoted heredocs with Dash v0.5.13
When executing our test suite with Dash v0.5.13.2 one can observe several test failures that all have the same symptoms: we have a quoted heredoc that contains multibyte characters, but the final data does not match what we actually wanted to write. One such example is in t0300, where we see the diffs like the following: --- expect-stdout 2026-04-01 07:25:45.249919440 +0000 +++ stdout 2026-04-01 07:25:45.254919509 +0000 @@ -1,5 +1,5 @@ protocol=https host=example.com -path=perú.git +path=perú.git username=foo password=bar While seemingly the same, the data that we've written via the heredoc contains some invisible bytes. The expected hex representation of the string is: 7065 72c3 ba2e 6769 74 per...git But what we actually get instead is this string: 7065 7285 02c3 ba02 852e 6769 74 per.......git What's important to note here is that the multibyte character exists in both versions. But in the broken version we see that the bytes are wrapped in a sequence of "85 02" and "02 85". This is the CTLMBCHAR byte sequence of Dash, which it uses internally to quote multibyte sequences. As it turns out, this bug was introduced in c5bf970 (expand: Add multi-byte support to pmatch, 2024-06-02), which adds multibyte support to more contexts of Dash. One of these contexts seems to be in heredocs, and Dash _does_ correctly unquote these multibyte sequences when using an unquoted heredoc. But the bug seems to be that this unquoting does not happen in quoted heredocs, and the bug still exists on the latest "master" branch. For now, work around the bug by using unquoted heredocs instead. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
1 parent 67ad421 commit 0c8424c

6 files changed

Lines changed: 40 additions & 18 deletions

File tree

t/t0300-credentials.sh

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -675,7 +675,9 @@ test_expect_success 'match percent-encoded values' '
675675
test_expect_success 'match percent-encoded UTF-8 values in path' '
676676
test_config credential.https://example.com.useHttpPath true &&
677677
test_config credential.https://example.com/perú.git.helper "$HELPER" &&
678-
check fill <<-\EOF
678+
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
679+
# that contain multibyte chars.
680+
check fill <<-EOF
679681
url=https://example.com/per%C3%BA.git
680682
--
681683
protocol=https

t/t3430-rebase-merges.sh

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -507,9 +507,11 @@ test_expect_success 'octopus merges' '
507507
git rebase -i --force-rebase -r HEAD^^ &&
508508
test "Hank" = "$(git show -s --format=%an HEAD)" &&
509509
test "$before" != $(git rev-parse HEAD) &&
510-
test_cmp_graph HEAD^^.. <<-\EOF
510+
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
511+
# that contain multibyte chars.
512+
test_cmp_graph HEAD^^.. <<-EOF
511513
*-. Tüntenfüsch
512-
|\ \
514+
|\\ \\
513515
| | * three
514516
| * | two
515517
| |/

t/t3902-quoted.sh

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -60,16 +60,18 @@ With SP in it
6060
"\346\277\261\351\207\216\347\264\224"
6161
EOF
6262
63-
cat >expect.raw <<\EOF
63+
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
64+
# that contain multibyte chars.
65+
cat >expect.raw <<EOF
6466
Name
65-
"Name and a\nLF"
66-
"Name and an\tHT"
67-
"Name\""
67+
"Name and a\\nLF"
68+
"Name and an\\tHT"
69+
"Name\\""
6870
With SP in it
69-
"濱野\t純"
70-
"濱野\n純"
71+
"濱野\\t純"
72+
"濱野\\n純"
7173
濱野 純
72-
"濱野\"純"
74+
"濱野\\"純"
7375
濱野/file
7476
濱野純
7577
EOF

t/t4014-format-patch.sh

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1285,7 +1285,9 @@ test_expect_success 'format-patch wraps extremely long from-header (rfc2047)' '
12851285
check_author "Foö Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar"
12861286
'
12871287

1288-
cat >expect <<'EOF'
1288+
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
1289+
# that contain multibyte chars.
1290+
cat >expect <<EOF
12891291
From: Foö Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar
12901292
Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo
12911293
Bar Foo Bar Foo Bar Foo Bar <author@example.com>
@@ -1300,7 +1302,9 @@ test_expect_success 'format-patch wraps extremely long from-header (non-ASCII wi
13001302
test_cmp expect actual
13011303
'
13021304

1303-
cat >expect <<'EOF'
1305+
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
1306+
# that contain multibyte chars.
1307+
cat >expect <<EOF
13041308
Subject: [PATCH] Foö
13051309
EOF
13061310
test_expect_success 'subject lines are unencoded with --no-encode-email-headers' '
@@ -1312,7 +1316,9 @@ test_expect_success 'subject lines are unencoded with --no-encode-email-headers'
13121316
test_cmp expect actual
13131317
'
13141318

1315-
cat >expect <<'EOF'
1319+
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
1320+
# that contain multibyte chars.
1321+
cat >expect <<EOF
13161322
Subject: [PATCH] Foö
13171323
EOF
13181324
test_expect_success 'subject lines are unencoded with format.encodeEmailHeaders=false' '
@@ -1523,7 +1529,9 @@ test_expect_success 'in-body headers trigger content encoding' '
15231529
test_env GIT_AUTHOR_NAME="éxötìc" test_commit exotic &&
15241530
test_when_finished "git reset --hard HEAD^" &&
15251531
git format-patch -1 --stdout --from >patch &&
1526-
cat >expect <<-\EOF &&
1532+
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
1533+
# that contain multibyte chars.
1534+
cat >expect <<-EOF &&
15271535
From: C O Mitter <committer@example.com>
15281536
Content-Type: text/plain; charset=UTF-8
15291537

t/t4201-shortlog.sh

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,9 @@ test_expect_success 'output from user-defined format is re-wrapped' '
105105
'
106106

107107
test_expect_success !MINGW,ICONV 'shortlog wrapping' '
108-
cat >expect <<\EOF &&
108+
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
109+
# that contain multibyte chars.
110+
cat >expect <<EOF &&
109111
A U Thor (5):
110112
Test
111113
This is a very, very long first line for the commit message to see if

t/t9001-send-email.sh

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1649,7 +1649,9 @@ test_expect_success $PREREQ 'To headers from files reset each patch' '
16491649
'
16501650

16511651
test_expect_success $PREREQ 'setup expect' '
1652-
cat >email-using-8bit <<\EOF
1652+
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
1653+
# that contain multibyte chars.
1654+
cat >email-using-8bit <<EOF
16531655
From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
16541656
Message-ID: <bogus-message-id@example.com>
16551657
From: author@example.com
@@ -1735,7 +1737,9 @@ test_expect_success $PREREQ '--8bit-encoding overrides sendemail.8bitEncoding' '
17351737
'
17361738

17371739
test_expect_success $PREREQ 'setup expect' '
1738-
cat >email-using-8bit <<-\EOF
1740+
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
1741+
# that contain multibyte chars.
1742+
cat >email-using-8bit <<-EOF
17391743
From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
17401744
Message-ID: <bogus-message-id@example.com>
17411745
From: author@example.com
@@ -1764,7 +1768,9 @@ test_expect_success $PREREQ '--8bit-encoding also treats subject' '
17641768
'
17651769

17661770
test_expect_success $PREREQ 'setup expect' '
1767-
cat >email-using-8bit <<-\EOF
1771+
# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
1772+
# that contain multibyte chars.
1773+
cat >email-using-8bit <<-EOF
17681774
From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
17691775
Message-ID: <bogus-message-id@example.com>
17701776
From: A U Thor <author@example.com>

0 commit comments

Comments
 (0)