If preparing a book it's natural to append an index of key words and to
list them in alphabetical order. But what if one wanted to index a
technical book in which a reader may want to look up 'symbols' which
begin with or are made up of punctuation characters? What order should
they appear in?

For example, should a symbol such as

%%

come before of after a symbol such as

!=

and where would

$

fit in?

I thought about following ASCII but it has its punctuation characters in
an unintuitive order - for example, its single and double quotes are not
adjacent and it has backslash in between opening and closing square
brackets. Neither of those makes much sense to a human.

Hence the query: does anyone know of a suitable standard or
human-friendly ordering for punctuation symbols?

--
James Harris

James Harris <james.harris.1@gmail.com> wrote or quoted:
>For example, should a symbol such as
>%%
>come before of after a symbol such as
>!=
>and where would
>$
>fit in?

That's a dope question, fo' sho'!

Ain't no "intuitive order" when it comes to them special
characters. ASCII's prob'ly the first thing that comes to
mind for most folks who even think about that kinda stuff.

In the TeXbook, it's "#" first, then "$", and finally "!". "\#"
gets slotted in between "#" and "##". Those special characters
are listed before the regular ol' letters in the index.

You could also put the names of the symbols, instead of the
symbols themselves, into the index.

ram@zedat.fu-berlin.de (Stefan Ram) wrote or quoted:
>In the TeXbook, it's "#" first, then "$", and finally "!". "\#"
>gets slotted in between "#" and "##". Those special characters
>are listed before the regular ol' letters in the index.

Get ready for a hot take - we're about to serve up a spicy quote:

|A special symbol is defined to be any character not appearing
|in the union of digits and the English alphabetic characters.
|Patterns starting with special symbols precede numbers, which
|precede patterns starting with letters.
....
|The default sorting order in "makeindex" is punctuation
|characters (in ASCII order), digits, control characters (1 .
|. . 31), space (32), letters (ignoring case), characters 127
|. . . 255.
| "Tips on Indexing LaTeX Documents" (1998-01-15) - Nelson H. F. Beebe

ram@zedat.fu-berlin.de (Stefan Ram) wrote or quoted:
>In the TeXbook, it's "#" first, then "$", and finally "!". "\#"
>gets slotted in between "#" and "##". Those special characters
>are listed before the regular ol' letters in the index.

Yup, it seems like Donald Knuth is using the ASCII order for the index
of his books, but the combo "\#" is basically "semantically" sorted in
with the "#" character. It's like they're homies or something.

But where does this character order in the ASCII code even come from?

To make it easier to use with programming languages, all the graphic
characters in the ASCII code from '63 had to be in one contiguous
code range, but this range had to be in the middle, since the outer
code blocks already had the two control characters NUL and DEL.

They also made it easier to manufacture keyboards by making sure
characters that usually appear together on a key only differ by one
bit. Plus, it made it simpler to pick useful 4-bit sub-ranges - like
a 4-bit sub-range that has the digits and the basic math operators.

The least significant 4 bits of the digits also correspond to the
numeric value of those digits. The A-to-Z letters fit into a nice
5-bit sub-range. When they were arranging the characters, they also
took into account the usual order of characters when sorting text.

James Harris <james.harris.1@gmail.com> wrote or quoted:
>I thought about following ASCII but it has its punctuation characters in
>an unintuitive order - for example, its single and double quotes are not
>adjacent and it has backslash in between opening and closing square
>brackets. Neither of those makes much sense to a human.

In ASCII, that ain't no single quote, it's a straight-up apostrophe.

And that whole "ABC" alphabetical order? That's just some
nonsense we all had to memorize as kids, otherwise it does
not "make sense" either!

Now, let's continue to talk about how they arranged them ASCII
characters:

With the "<=>", the order actually makes sense. And they
kinda followed that pattern with "[\]" and "{|}". On an old
Teletype Model 33, you'd see the "\" right following the "["
on the keyboard. As for "$%&'", they grouped those together
'cause they're seen as the "business special characters".

James Harris wrote:

> If preparing a book it's natural to append an index of key words and to
> list them in alphabetical order. But what if one wanted to index a
> technical book in which a reader may want to look up 'symbols' which
> begin with or are made up of punctuation characters? What order should
> they appear in?

> For example, should a symbol such as

> %%

> come before of after a symbol such as

> !=

> and where would

> $

> fit in?

> I thought about following ASCII but it has its punctuation characters in
> an unintuitive order - for example, its single and double quotes are not
> adjacent and it has backslash in between opening and closing square
> brackets. Neither of those makes much sense to a human.

> Hence the query: does anyone know of a suitable standard or
> human-friendly ordering for punctuation symbols?

I don't. If you're actually doing this, I'd suggest using whatever
order comes out of your word processor's indexing feature or your
favorite sorting software. If you have so many strings beginning with
symbols that readers can't just glance through the index, you might
want to provide a note at the beginning giving the order you chose.
See for instance

https://www.chicagomanualofstyle.org/qanda/data/faq/topics/Alphabetizing/faq0003.html

Wasn't there a time when publishers usually did the indexing?

--
Jerry Friedman

ram@zedat.fu-berlin.de (Stefan Ram) wrote or quoted:
>That's a dope question, fo' sho'!

"Dope question" meaning "excellent question" here as in
the slang expression, not "dope" as in "a silly person"!

On 2024-04-17 18:45:08 +0000, jerryfriedman said:

> James Harris wrote:
>
>> If preparing a book it's natural to append an index of key words and to
>> list them in alphabetical order. But what if one wanted to index a
>> technical book in which a reader may want to look up 'symbols' which
>> begin with or are made up of punctuation characters? What order should
>> they appear in?
>
>> For example, should a symbol such as
>
>> %%
>
>> come before of after a symbol such as
>
>> !=
>
>> and where would
>
>> $
>
>> fit in?
>
>> I thought about following ASCII but it has its punctuation characters
>> in an unintuitive order - for example, its single and double quotes are
>> not adjacent and it has backslash in between opening and closing square
>> brackets. Neither of those makes much sense to a human.
>
>> Hence the query: does anyone know of a suitable standard or
>> human-friendly ordering for punctuation symbols?
>
> I don't. If you're actually doing this, I'd suggest using whatever
> order comes out of your word processor's indexing feature or your
> favorite sorting software. If you have so many strings beginning with
> symbols that readers can't just glance through the index, you might
> want to provide a note at the beginning giving the order you chose.
> See for instance
>
> https://www.chicagomanualofstyle.org/qanda/data/faq/topics/Alphabetizing/faq0003.html
>
>
> Wasn't there a time when publishers usually did the indexing?

There was, but often they did a very bad job of it. Better do it
yourself if you can.

As for the question, I haven't needed to include symbols in an index.
However, if I did I'd put them at the beginning of the index, in any
order, as unless the list is very long it's easy to skim down to one
you want.

--
Athel -- French and British, living in Marseilles for 37 years; mainly
in England until 1987.

In article <l8akkdFqrc1U1@mid.individual.net>,
Athel Cornish-Bowden <me@yahoo.com> wrote:
>On 2024-04-17 18:45:08 +0000, jerryfriedman said:
>> Wasn't there a time when publishers usually did the indexing?
>
>There was, but often they did a very bad job of it. Better do it
>yourself if you can.

They used to hire professional indexers to do it, but humans are
expensive, and computers are very good at recognizing tokens.
Unfortunately, computers are not very good at evaluating how those
tokens are used, and determining which uses are the relevant ones to
include in the index.

-GAWollman

--
Garrett A. Wollman | "Act to avoid constraining the future; if you can,
wollman@bimajority.org| act to remove constraint from the future. This is
Opinions not shared by| a thing you can do, are able to do, to do together."
my employers. | - Graydon Saunders, _A Succession of Bad Days_ (2015)

Stefan Ram wrote:

> ram@zedat.fu-berlin.de (Stefan Ram) wrote or quoted:
>>That's a dope question, fo' sho'!

> "Dope question" meaning "excellent question" here as in
> the slang expression, not "dope" as in "a silly person"!

Yes, that might inadvertently offended the OP. Speaking of which,
imitation is often used for mockery, and your imitations of non-
standard dialects strike me as much more likely to offend people
than, say, profanity is.

--
Jerry Friedman

On 18/04/2024 4:17 a.m., James Harris wrote:
> If preparing a book it's natural to append an index of key words and to
> list them in alphabetical order. But what if one wanted to index a
> technical book in which a reader may want to look up 'symbols' which
> begin with or are made up of punctuation characters? What order should
> they appear in?
>
> For example, should a symbol such as
>
>   %%
>
> come before of after a symbol such as
>
>   !=
>
> and where would
>
>   $
>
> fit in?
>
> I thought about following ASCII but it has its punctuation characters in
> an unintuitive order - for example, its single and double quotes are not
> adjacent and it has backslash in between opening and closing square
> brackets. Neither of those makes much sense to a human.
>
> Hence the query: does anyone know of a suitable standard or
> human-friendly ordering for punctuation symbols?
>

Don't these symbols have names? E.g. percent, exclamation, dollar...
That would seem to be a simple way of alphabetizing them in an index.
They wouldn't be all together, of course, but why would you need that?

In article <uvpdul$1qbnf$1@dont-email.me>,
Ross Clark <r.clark@auckland.ac.nz> wrote:

>Don't these symbols have names? E.g. percent, exclamation, dollar...
>That would seem to be a simple way of alphabetizing them in an index.
>They wouldn't be all together, of course, but why would you need that?

If the book were *about* the symbols, that might be appropriate, but
presumably the book is about a programming language or environment,
and the intended audience may not all have the same names for the
symbols. Consider that even '(' and ')' and '[' and ']' and '{' and
'}' are not universally known by the same names across the
English-speeking world -- let alone '\' and '/' or '#'. For someone
actually using the index, who probably just wants to find the meaning
of that '`' character, they may not know that backtick is inexplicably
called "GRAVE ACCENT" in the Unicode table.

-GAWollman

--
Garrett A. Wollman | "Act to avoid constraining the future; if you can,
wollman@bimajority.org| act to remove constraint from the future. This is
Opinions not shared by| a thing you can do, are able to do, to do together."
my employers. | - Graydon Saunders, _A Succession of Bad Days_ (2015)

Stefan Ram:
> That's a dope question, fo' sho'!

Plonk.
--
Mark Brader, Toronto | "Effective immediately, all memos are to be written
msb@vex.net | in clear, active-voice English." -- US gov't memo

James Harris:
> If preparing a book it's natural to append an index of key words and to
> list them in alphabetical order. But what if one wanted to index a
> technical book in which a reader may want to look up 'symbols' which
> begin with or are made up of punctuation characters? What order should
> they appear in?

The same issue arises in alphabetical lists of titles if the title
begins with a symbol. I posted this example here in 2011:

In 1989 Donnalyn Frey and Rick Adams wrote a book that O'Reilly
published under the title "!%@:: A Directory of Electronic Mail
Addressing & Networks". Try deciding how you'd find *that* one
in an alphabetical catalog!

For example, I tried the Library of Congress. When I copied the full
title into their basic search form, it rejected it outright -- I had to
delete the first two characters before it would find it. I then tried
searching on just some keywords in the title. From the results I got,
it appears that the library sorts this title after titles beginning
with a digit and before titles beginning with a letter. But you could
equally well imagine it being sorted as if it began with "Exclamation".

(I also tried the Toronto Public Library; if I remember correctly,
*they* used to have it sorted as if it began with the words "At sign".
But I can't check on that, because they don't have a copy of it any more.)

--
Mark Brader, Toronto | "Save our planet: it's the only one with chocolate"
msb@vex.net | --Bumper sticker

My text in this article is in the public domain.

On 17-Apr-24 20:46, jerryfriedman wrote:
> Stefan Ram wrote:
>
>> ram@zedat.fu-berlin.de (Stefan Ram) wrote or quoted:
>>> That's a dope question, fo' sho'!
>
>>   "Dope question" meaning "excellent question" here as in
>>   the slang expression, not "dope" as in "a silly person"!
>
> Yes, that might inadvertently offended the OP.  Speaking of which,
> imitation is often used for mockery, and your imitations of non-
> standard dialects strike me as much more likely to offend people
> than, say, profanity is.
>

It's too late Jerry.
Stefan Ram has been assimilated by a rogue AI machine, and the resulting
construct is spewing out posts at random.

--
Sam Plusnet

On 18/04/24 02:17, James Harris wrote:

> If preparing a book it's natural to append an index of key words and
> to list them in alphabetical order. But what if one wanted to index
> a technical book in which a reader may want to look up 'symbols'
> which begin with or are made up of punctuation characters? What
> order should they appear in?

[...]

> I thought about following ASCII but it has its punctuation
> characters in an unintuitive order - for example, its single and
> double quotes are not adjacent and it has backslash in between
> opening and closing square brackets. Neither of those makes much
> sense to a human.

Perhaps so, but every alternative I can think of is just as bad. On
balance, I would choose ASCII ordering, especially if the target
audience is familiar with ASCII.

--
Peter Moylan http://www.pmoylan.org
Newcastle, NSW

* Athel Cornish-Bowden <l8akkdFqrc1U1@mid.individual.net> :
Wrote on Wed, 17 Apr 2024 21:01:52 +0200:
> On 2024-04-17 18:45:08 +0000, jerryfriedman said:
>> James Harris wrote:
>>> If preparing a book it's natural to append an index of key words
>>> and to list them in alphabetical order. But what if one wanted to
>>> index a technical book in which a reader may want to look up
>>> 'symbols' which begin with or are made up of punctuation
>>> characters? What order should they appear in?
>>> For example, should a symbol such as
>>
>>> %%
>>
>>> come before of after a symbol such as
>>
>>> !=
>>
>>> and where would
>>
>>> $
>>
>>> fit in?
>>
>>> I thought about following ASCII but it has its punctuation
>>> characters in an unintuitive order - for example, its single and
>>> double quotes are not adjacent and it has backslash in between
>>> opening and closing square brackets. Neither of those makes much
>>> sense to a human.
>>
>>> Hence the query: does anyone know of a suitable standard or
>>> human-friendly ordering for punctuation symbols?
>> I don't. If you're actually doing this, I'd suggest using whatever
>> order comes out of your word processor's indexing feature or your
>> favorite sorting software. If you have so many strings beginning
>> with symbols that readers can't just glance through the index, you
>> might want to provide a note at the beginning giving the order you
>> chose. See for instance
>> https://www.chicagomanualofstyle.org/qanda/data/faq/topics/Alphabetizing/faq0003.html
>> Wasn't there a time when publishers usually did the indexing?
>
> There was, but often they did a very bad job of it. Better do it
> yourself if you can.
>
> As for the question, I haven't needed to include symbols in an
> index. However, if I did I'd put them at the beginning of the index,
> in any order, as unless the list is very long it's easy to skim down
> to one you want.

The common Lisp Hyperspec has a symbol index with a page for
non-alphabetic symbols

http://www.lispworks.com/documentation/lw80/CLHS/Front/X_Alph_9.htm

The symbols "&*+-/1<=>" happen to be placed in the order of the ascii
character codes.

(loop for c across "&*+-/1<=>" collect (char-code c))
(38 42 43 45 47 49 60 61 62)

Ross Clark wrote:

> On 18/04/2024 4:17 a.m., James Harris wrote:
>> If preparing a book it's natural to append an index of key words and to
>> list them in alphabetical order. But what if one wanted to index a
>> technical book in which a reader may want to look up 'symbols' which
>> begin with or are made up of punctuation characters? What order should
>> they appear in?
>>
>> For example, should a symbol such as
>>
>>   %%
>>
>> come before of after a symbol such as
>>
>>   !=
>>
>> and where would
>>
>>   $
>>
>> fit in?
>>
>> I thought about following ASCII but it has its punctuation characters in
>> an unintuitive order - for example, its single and double quotes are not
>> adjacent and it has backslash in between opening and closing square
>> brackets. Neither of those makes much sense to a human.
>>
>> Hence the query: does anyone know of a suitable standard or
>> human-friendly ordering for punctuation symbols?
>>

> Don't these symbols have names? E.g. percent, exclamation, dollar...
> That would seem to be a simple way of alphabetizing them in an index.
> They wouldn't be all together, of course, but why would you need that?

I just learned that some authorities--the Chicago Manual of Style,
at least--recommend alphabetizing isolated numbers that way. I don't
remember ever noticing that.

https://press.uchicago.edu/Misc/Chicago/CHIIndexingComplete.pdf

At least in English, unlike French, there's a nearly complete
consensus on the names of the numbers. But does _101 Dalmatians_
come before or after (*wikips*) the Korean TV series _100 Days
My Prince_?

--
Jerry Friedman

Just this Wednesday, jerryfriedman explained that ...
> James Harris wrote:
>
>> If preparing a book it's natural to append an index of key words and to
>> list them in alphabetical order. But what if one wanted to index a
>> technical book in which a reader may want to look up 'symbols' which begin
>> with or are made up of punctuation characters? What order should they
>> appear in?
>
>> For example, should a symbol such as
>
>> %%
>
>> come before of after a symbol such as
>
>> !=
>
>> and where would
>
>> $
>
>> fit in?
>
>> I thought about following ASCII but it has its punctuation characters in an
>> unintuitive order - for example, its single and double quotes are not
>> adjacent and it has backslash in between opening and closing square
>> brackets. Neither of those makes much sense to a human.
>
>> Hence the query: does anyone know of a suitable standard or human-friendly
>> ordering for punctuation symbols?
>
> I don't. If you're actually doing this, I'd suggest using whatever
> order comes out of your word processor's indexing feature or your
> favorite sorting software. If you have so many strings beginning with
> symbols that readers can't just glance through the index, you might
> want to provide a note at the beginning giving the order you chose.
> See for instance
>
> https://www.chicagomanualofstyle.org/qanda/data/faq/topics/Alphabetizing/faq0003.html
>
> Wasn't there a time when publishers usually did the indexing?

Aren't symbol conventions usually given in a page at the beginning or
an appendix at the end (but before the index)?

Although my _gnuplot In Action_ [1] has only a brief mention of
symbols under the "Conventions" subheading of "About This Book", and a
section of the index for symbols and one for numerics. I'm not going
to go into the ordering in detail, but it begins with '^' (caret), ends
with '$' (dollar sign), and has '@' (atsign) in the middle.

[1] Philipp K. Janert, 2nd edition, Manning Publications Co, Shelter
Island NY, 2016

That's the tech book most in reach at the moment, so a very brief
survey.

/dps "unsigned"

--
"This is all very fine, but let us not be carried away be excitement,
but ask calmly, how does this person feel about in in his cooler
moments next day, with six or seven thousand feet of snow and stuff on
top of him?"
_Roughing It_, Mark Twain.

Garrett Wollman hat am 17.04.2024 um 23:43 geschrieben:
> For someone
> actually using the index, who probably just wants to find the meaning
> of that '`' character, they may not know that backtick is inexplicably
> called "GRAVE ACCENT" in the Unicode table.

Inexplicably? How else do you call it?

I do not know about mathematicians, but the linguists who invented
Unicode are trying to list there letters, ideograms and other symbols
used in all known languages. And "`" has been called "accent grave" for
centuries in French.

On 18/04/24 14:39, Silvano wrote:
> Garrett Wollman hat am 17.04.2024 um 23:43 geschrieben:

>> For someone
>> actually using the index, who probably just wants to find the meaning
>> of that '`' character, they may not know that backtick is inexplicably
>> called "GRAVE ACCENT" in the Unicode table.
>
> Inexplicably? How else do you call it?
>
> I do not know about mathematicians, but the linguists who invented
> Unicode are trying to list there letters, ideograms and other symbols
> used in all known languages. And "`" has been called "accent grave" for
> centuries in French.

Backtick is a new one to me. Is that used anywhere?

--
Peter Moylan http://www.pmoylan.org
Newcastle, NSW

On 2024-04-18 01:43:10 +0000, Madhu said:
>
> [ … ]
>
> The common Lisp Hyperspec has a symbol index with a page for
> non-alphabetic symbols
>
> http://www.lispworks.com/documentation/lw80/CLHS/Front/X_Alph_9.htm
>
> The symbols "&*+-/1<=>" happen to be placed in the order of the ascii
> character codes.
>
> (loop for c across "&*+-/1<=>" collect (char-code c))
> (38 42 43 45 47 49 60 61 62)

Even if such an ordering exists, are readers of books supposed to study
it before reading?

--
Athel -- French and British, living in Marseilles for 37 years; mainly
in England until 1987.

jerryfriedman wrote:

> At least in English, unlike French, there's a nearly complete
> consensus on the names of the numbers. But does _101 Dalmatians_
> come before or after (*wikips*) the Korean TV series _100 Days
> My Prince_?

When alphabetizing titles and such I would ignore all non-letters.

When alphabetizing technical characters I would put them all first and
sort them in the easiest way - which means the way a computer would sort
them without special algorithms.

Kernigan&Ritchie have put all the technical characters first in the
index. The principle for their internal sorting is not evident.

--
Bertel
Kolt, Denmark

jerryfriedman <jerry.friedman99@gmail.com> wrote:

> Stefan Ram wrote:
>
> > ram@zedat.fu-berlin.de (Stefan Ram) wrote or quoted:
> >>That's a dope question, fo' sho'!
>
> > "Dope question" meaning "excellent question" here as in
> > the slang expression, not "dope" as in "a silly person"!
>
> Yes, that might inadvertently offended the OP. Speaking of which,
> imitation is often used for mockery, and your imitations of non-
> standard dialects strike me as much more likely to offend people
> than, say, profanity is.

Still, it would be a nice research project:
translating for example Oxford English into American Redneck,
and vica versa.
I guess it would be hard to get it just right,

Jan

On 17/04/2024 17:54, Stefan Ram wrote:
> ram@zedat.fu-berlin.de (Stefan Ram) wrote or quoted:
>> In the TeXbook, it's "#" first, then "$", and finally "!". "\#"
>> gets slotted in between "#" and "##". Those special characters
>> are listed before the regular ol' letters in the index.
>
> Get ready for a hot take - we're about to serve up a spicy quote:
>
> |A special symbol is defined to be any character not appearing
> |in the union of digits and the English alphabetic characters.
> |Patterns starting with special symbols precede numbers, which
> |precede patterns starting with letters.
> ...
> |The default sorting order in "makeindex" is punctuation
> |characters (in ASCII order), digits, control characters (1 .
> |. . 31), space (32), letters (ignoring case), characters 127
> |. . . 255.
> |
> "Tips on Indexing LaTeX Documents" (1998-01-15) - Nelson H. F. Beebe

The ASCII ordering is well-defined but is unfortunately not very 'human'.

I've read all the replies to date and will respond to them but for now
here's a first suggestion for an ordering of common punctuation symbols.
Although they will look different in different typefaces the basic
approach I've taken is to list from lighter to heavier, similar
characters together, and arithmetic operators (as used in computer
programs, e.g. * instead of x) together.

space (would go first, if relevant)
..
, ;
: `
'
“
! |
^ ~
_ =
+ -
* /
\ ?
$ &
% #
@

Then the brackets:

<
>
( )
[ ]
{ }

The main goal, in the absence of a pre-existing standard, can be seen as
essentially to make it feasible for humans to look up 'words' of such
characters in an index.

I already think some changes are needed and there will be other
punctuation characters to add (I've only included the ASCII punctuation
characters so far) but ISTM the principle of going from light to heavy
is likely a good way to go.

Feedback welcome!

--
James Harris