Top to bottom file representation order is not the same as Left to Right

Robin's Avatar

Robin

10 Apr, 2018 01:01 PM

MMD Composer edit display and preview display diverge for MMD tables in which pipes have been inserted in a RTL sequence of strings.

This appears to be because of an understandable (but technically incorrect, and somewhat provincial) assumption that LTR cell order the is 'natural' interpretation of top to bottom string sequence representation in a file.

LTR cell order is never 'natural', but is correct for LTR scripts.

If this is fixed, and RTL cell sequences allowed to be RTL, then the WYSIWYG failure in MMD Composer (and in MMD table rendering generally) will be corrected.

1. Producing `<td>` cells in the order of the source file stream is correct
2. Appending them to the Right, when the text is RTL is *incorrect*, and merely an expression of a provincial blind spot.

Leading to a WYSIWYG collapse in your preview, and the generation of incorrect HTML.

Example for testing in MMD Composer

```
The bidirectional cell here is displayed and rendered correctly.

| Example ||||
|:-------:|:-------:|:-------:|:-------:|
| Genesis בראשית ברא אלהים ||||
| alpha | beta | gamma | delta |

But when pipe characters are inserted between the RTL words, the generated RTL `<td>` cells are append to the Right instead of to the Left.

This is simply a technical error (revealed by the divergence of the editing and Preview displays) rooted in a provincial assumption that 'first then' is 'naturally' the same as left then right :-)

| Example ||||
|:-------:|:------:|:-----:|:-----:|
| Genesis | בראשית | ברא | אלהים |
| alpha | beta | gamma | delta |

```

  1. 1 Posted by robintrew on 10 Apr, 2018 01:34 PM

    robintrew's Avatar

    In short,

    if you want to fix the mismatch between editing display and preview rendering of MMD tables with sequences of RTL cells then:

    when you are appending one RTL `<td>` cell to another, you should append it to the *Left* and not to the *Right*.

    TL;DR 'Next' means 'to the right' in some languages, and 'to the left' in others.

    or alternatively: Addition may be commutative but concatenation is **not**. We have to choose between left append and right append. There is no 'natural' default, it depends on the language:

    With RTL languages, we append each new element to the Left, with LTR, we append to the Right.

    Culturally hegemonic assumptions just lead to WYSIWYG failure :-)

  2. 2 Posted by robintrew on 10 Apr, 2018 02:11 PM

    robintrew's Avatar

    Re: diverging displays (Editing display vs Preview and output):

    See attached screenshot

  3. 3 Posted by robintrew on 10 Apr, 2018 02:14 PM

    robintrew's Avatar

    See difference between edited and previewed versions of second table:

  4. 4 Posted by robintrew on 10 Apr, 2018 02:42 PM

    robintrew's Avatar

    Conceivable also that the current divergence of edit and preview displays of MMD tables with RTL content might prove fixable through use of the `dir` attribute, either now or later:

    https://www.w3.org/International/questions/qa-html-dir

    https://www.w3.org/International/tutorials/bidi-xhtml/#term_bidirectional

    It turns out that you can successfully add attributes like `dir="rtl" lang="he"` to `<table>` or `<tbody>` tags, if you want the sequence of all of the cells in all rows to be RTL, but this won't mend the broken MMD Composer relationship between edit display and preview for bidirectional cases in which a single row contains sequences of both RLT and LRT sequences.

    For those cases, MMD would need to bite the bullet and reluctantly accept that concatenation is not (and has never been, outside the mid West) a choice-free commutative operation – you can't escape deciding, with your next `<td>` element, whether an RTL left-append or a LTR right-append is what's needed to preserve WYSIWYG.

  5. 5 Posted by robintrew on 10 Apr, 2018 06:09 PM

    robintrew's Avatar

    A first rough thumbnail sketch in JS

    ```
    (() => {
      'use strict';

      // MAIN ---------------------------------------------------
      const main = () => {
        // xs :: [String]
        const xs = ['עברית', 'Genesis', 'בראשית', 'ברא', 'אלוהים'];

        // Processing in order, and avoiding the illusion that
        // concatenation is commutative ( somehow Left-Right blind,
        // or naturally right-leaning )

        // Content sequence for <td> cells
        // [String]
        return showJSON(
          appendLeftOrRight(xs)
        );
      };

      // GENERIC FUNCTIONS --------------------------------------

      // foldl :: (a -> b -> a) -> a -> [b] -> a
      const foldl = (f, a, xs) => xs.reduce(f, a);

      // showJSON :: a -> String
      const showJSON = x => JSON.stringify(x, null, 2);

      // NON-COMMUTATIVE APPEND ( SENSITIVE TO LANGUAGE DIRECTION )

      // A very simple sketch of an isRTL function

      // isRTL :: String -> Bool
      const isRTL = s =>
        /^[֐-׿؀-ۿ܀-ݏֿ\s]+$/.test(s) &&
        !(/^\sֿ*$/.test(s));

      // appendLeftOrRight :: [String] -> [String]
      const appendLeftOrRight = xs => {
        const dctRuns = foldl(
          (a, x) =>
          isRTL(x) ? {
            rtl: [x].concat(a.rtl), // SO CONCAT == LEFT APPEND
            ltr: a.ltr
          } : {
            rtl: [],
            ltr: a.ltr.concat(a.rtl).concat(x) // SO CONCAT == RIGHT APPEND
          }, {
            rtl: [],
            ltr: []
          },
          xs
        );

        // MAIN (MIXED) ACCUMULATOR WITH ANY RESIDUE RIGHT-APPENDED
        return dctRuns.ltr.concat(dctRuns.rtl);
      };

      // VALUE ------------------------------------------------------------------
      return main();
    })();

    ```

  6. 6 Posted by robintrew on 10 Apr, 2018 06:12 PM

    robintrew's Avatar

    JS file

  7. 7 Posted by robintrew on 10 Apr, 2018 06:15 PM

    robintrew's Avatar

    First rough JS sketches of isRTL and appendLeftOrRight

  8. 8 Posted by robintrew on 10 Apr, 2018 06:16 PM

    robintrew's Avatar
  9. Support Staff 9 Posted by fletcher on 11 Apr, 2018 12:03 AM

    fletcher's Avatar

    You're over-complicating this.

    TL;DR; MMD is doing the correct thing, just not what you want. Use the dir HTML attribute to toggle between L-R and R-L directionality in your output, and you'll have to be thoughtful and creative when using bidirectional text in a single table row to get the result that you want.

    Longer version to provide a more detailed explanation follows...

    Markdown and derivatives (including MultiMarkdown) parse text as a stream of characters/characters. Whether those characters are displayed on screen in left-to-right or right-to-left is, in a sense, up to the user (via preferences, defaults, CSS, and the like). Therefore it's not a particularly useful way to think of it. It may be more helpful to think of it as a vertical stream of text, top-to-bottom, to avoid preconceived ideas of left/right directionality.

    While a full description of the various Markdown parsing strategies is an entirely separate discussion, a first order approximation is that it happens from top to bottom. The output, with a few small exceptions (such as footnotes), is written in the same order, top to bottom.

    Markdown doesn't "care" about left-to-right or right-to-left scripts. Except for matching items defined by reference (e.g. links, footnotes, etc.) the only thing Markdown cares about are the special characters used in the syntax (e.g. []*_+-#()<> and the like). It is completely agnostic to the "direction" of the script -- it only knows top-to-bottom (using our artificial definition above).

    It is your text editor, or your browser, that understands (or may understand) the concept of directional scripts and arranges the character stream in a two-dimensional space that includes L-R lines or R-L lines.

    MultiMarkdown Composer and most of the others you mentioned (to be honest you had so many tweets that I did not read them all, so if i missed something important I apologize) are based on Apple's Core Text (which automatically interprets script directionality and styles the text as it sees fit). So these applications apply "word processor" styling to what is really more of a programming text document (Markdown is more properly thought of as a programming language than a word processing document in this regard). Sublime Text is an example of a text editor (not word processor) that uses a custom text engine, and does not automatically invert R-L scripts on the screen (at least with default settings).

    This means that the editing pane in Composer, or TextEdit which is the prototypical example of Core Text I always use for error checking, will invert the direction of lines of text using a right-to-left script, even though one could argue that it shouldn't (from a programming standpoint, not a linguistics standpoint.)

    The generated HTML has no directionality applied to it (i.e. the dir attribute discussed in the pages you linked to above), so the default is used which is left-to-right. (One can argue about whether a L-R default is right, fair, overly-English-centric, etc. I'm not arguing any of that. It's just the default.)

    If, however, you want to use R-L for all or part of your document, then you will need to apply the dir attribute accordingly (e.g. at the document level, the table level, the cell level, or even the span level). You will have to do this based on a human's conceptual understanding of the document. Neither Markdown, nor MultiMarkdown, nor any other derivative that I am familiar with, will do this for you automatically. Since you linked to the page describing the dir attribute, I'll assume you are familiar with it and understand how to use it.

    The example you give, where you use single R-L characters in a table row that also has L-R characters, requires additional thought. Because word processors are likely to automatically invert runs of consecutive R-L characters, what you see on screen may not match what is actually in the file. Additionally, since they don't understand that the pipe characters are intended to isolate spans of text, the pipes simply get wrapped up in the inverted directionality. For example, Composer (Apple's Core Text), inverted the last three "letter" characters, and the intervening spaces/pipes. This has the visual effect of reversing the last three cells in the table. Sublime Text does not do this (using default settings anyways), and accurately represents what is actually in the file.

    MultiMarkdown correctly converts this (using the top-bottom idea above) into a table, where the individual table cells match what is in the file. The cells output in the same order they are input -- top to bottom.

    What it sound like you want to happen is that MMD selectively changes the ordering of the cells based on what it thinks you meant to do instead of what is actually there in the text file. That rarely works out well in practice.

    What may be possible is to disable the automatic inversion of R-L characters by macOS in Composer's editing pane so that you see the actual file contents as they are. I'm not sure this is really what you would want to have happen though, as it would also affect words consisting of R-L characters....

    "Wrapping" R-L characters between L-R letters (e.g. x) prevents the automatic inversion from happening (more properly, it probably limits to the single character, which is the same end result). In my experiments, however, using standard punctuation and markup does not do the same (so *א* and א don't break the inversion.) So this technique will allow you to visualize things properly, it would introduce extraneous markup into your HTML output, unless you strip them before processing.

    It seems as though tables would be the primary situation where this would come up. So a better solution is probably to use raw HTML for your complex tables to avoid any confusion.

  10. 10 Posted by robintrew on 11 Apr, 2018 06:15 AM

    robintrew's Avatar

    TL;DR the relationship between your edit display and MMD generation is broken.

    Why ? Because in the MMD generation you are treating concatenation as if it was a commutative operation, invariably equating it to RIGHT-append rather than to LEFT-append.

    This error is not being made in the edit display. Perhaps its display logic is written by someone else.

    Take-away: MMD, and its relationship to the editor view, is broken by a provincial misapprehension, and seems more than unlikely to be mended :-)

  11. 11 Posted by robintrew on 11 Apr, 2018 07:15 AM

    robintrew's Avatar

    You're clearly not going to be able to make progress on the WYSIWYG failure, so let's draw this to an unhappy closing of the curtain, but the locus of your muddle is very eloquently expressed by one of your sentences above:

    > Core Text ... will invert the direction of lines of text using a right-to-left script, even though one could argue that it shouldn't

    No. It does not "Invert the direction". That expresses your confusion between top to bottom file representation order and L or R display sequence. There is NO natural or automatic mapping between top to bottom in a file and Left to Right in a display.

    Core Text does NOT "invert the direction". It **maintains** the top to bottom file representation order absolutely invariantly and consistently.

    What it does do is to make language-sensitive choices about how that vertical file sequence axis is horizontally represented in display. The issues are orthogonal, and you are conflating them – collapsing the real two dimensional model (vertical file sequence, horizontal display sequence) down to a mono-dimensional imaginary picture.

    Your WYSIWYG is broken, and will clearly remain broken, because in South Carolina, any failure to automatically map vertical sequence to God's own RIGHTward horizontal progress is viewed with alarm as 'strange', "Inverting the direction", and probable evidence of malpractice by a client.

    Core text is correct and two-dimensional. It knows that the append operation is not commutative.

    MMD is broken and one-dimensional. It falsely assumes that append is a form of addition.

    That is why Core Text shows the correct picture, and MMD gets it wrong.

  12. Support Staff 12 Posted by fletcher on 11 Apr, 2018 10:45 AM

    fletcher's Avatar
    1. If you knew what you were talking about, you would realize how incorrect you are about the topic we are discussing, as well as your misguided attempt to conflate geography, politics, xenophobism, and religion based on your own stereotypes.

    2. In violation of the sage wisdom often found on xkcd (https://xkcd.com/386/), I will include one more comment for the sake of anyone else reading this thread who wants to understand what is going on and why, and then will stop wasting my time.

    Imagine a document including the the following English table:

    | 1 | 2 | 3 | 
    | ---   | ---   | ----- | 
    | one   | two   | three |
    

    Now, you use find/replace to convert English to Hebrew, one word at a time (using Google Translate, so apologies for any translational errors, but the point still stands).

    This results in the following table:

    | 1 | 2 | 3 | 
    | ---   | ---   | ----- | 
    | אחד    | שתיים    | שְׁלוֹשָׁה  |
    

    The source text (in my browser anyways), has the entire last row inverted, which means that the word for "three" appears to be in the first column, and the word for "one" appears to be in the last column. This is clearly visually incorrect, though it does maintain the sense of the individual words.

    When MMD processes it, however, the resulting HTML table correctly orders the first row and the second row of the table so that "1" and "אחד" are vertically aligned in the same column. If one desired, they could use the dir attribute to visually flip the entire table so that the first column appears on the right, and the logic of the table will be maintained. Which is exactly why the dir attribute exists.

    1 2 3
    אחד שתיים שְׁלוֹשָׁה

    When the source is viewed in Sublime Text, then the visual alignment of the columns is properly maintained, but the words themselves are also backwards, which I would assume would be equivalent to "eno" and "eerht". So, for a reader, it's not correct either.

    So neither Core Text nor Sublime Text actually display the source text properly, but the resulting HTML generated by MMD is correct resulting in a table that allows one to read the words (assuming you read Hebrew) and maintains the vertical connection between columns.

    From this point I will ignore anything that is not constructive discussion, and will delete anything from the site that devolves into ad-hominem attacks assuming that it is just ongoing trolling and no longer an actual discussion about real software concerns.

  13. 13 Posted by robintrew on 11 Apr, 2018 12:11 PM

    robintrew's Avatar

    Absurd example - Core Text still gets this right, and MMD still gets it wrong.

    ```
    Absurd example - WYSIWYG still fails :-)

    Again, Core Text gets it right. MMD gets muddled.

    |:------:|:---:|:-----:|:----:|
    | ערבית | 1 | 2 | 3 |
    | מספרים | אחד | שתיים | שלוש |
    ```

    ( Your argument – how can you seriously think this ? – is that RTL must be wrong because if you place an RTL row beneath an LTR row the cells don't match up vertically :-) Do I really have to explain to you that if you want two rows to match up vertically, you simply give them both the same display direction, whether RTL or LTR, as above ? Your desire to reduce append to a form of addition appears to be blindingly intense :-)

    In short, MMD discards information. When a user specifies an RTL cell display sequence in Unicode-compliant Core Text, MMD makes no attempt to preserve that display sequence, and overwrites it (to the tune, as I learn from your responses, of "doing the right thing", and even questioning the behaviour of the rather more sophisticated and Unicode-compliant Core Text) with an LTR cell display sequence of its own.

    *** *** ***

    On tone, a word to the wise – when a customer brings a bug report (failure of WYSIWYG for example), make an effort not to open the discussion with:

    1. An assertion that an RTL string is 'strange',
    2. a passing of the buck "did you XYZ ?",
    3. and an invocation of two egregiously unicode-non-compliant pieces of software as proofs of a technically specious claim (despite the counter-evidence of your own editor – not to mention Pages, Atom, the Bash shell, MS Word – in fact anything compliant with the Unicode distinction between memory representation sequence and horizontal display sequence).

    See your opening communication below.

    It fails to impress, and sets the tone for what follows.

    (Try instead – "that's interesting, let me try to understand what's happening" :-)

  14. 14 Posted by robintrew on 14 Apr, 2018 05:47 PM

    robintrew's Avatar

    > Because word processors are likely to automatically invert runs of consecutive R-L characters, what you see on screen may not match what is actually in the file

    No. [Unicode compliant](http://unicode.org/reports/tr9/) software does not **invert** runs.

    On the contrary, it correctly represents the unicode distinction between (vertical) memory storage sequence and (horizontal) display sequence. It inverts nothing.

    It **chooses** the correct display sequence – RTL or LTR – required to correctly represent each unicode data sequence.

    > why the dir attribute exists

    [Structural markup for text direction in HTML](https://www.w3.org/International/questions/qa-html-dir) uses the `dir` attribute to select the predominant direction for whole tables or whole rows (and other whole units above that level), but does **not** use the dir attribute for cell sequences which are themselves bidirectional.

    At that level, the correct HTML encoding is to map the Unicode bidirectional cell information directly to the sequence of `<td>` and `<th>` cells.

    The Unicode bidirectional algorithm identifies some character types (essentially those representing numbers) as neutral in their display direction, inheriting it from their immediate predecessors (as in the example in the previous post).

    A JS sketch of this would be:

    ```
    // bidir :: [Node] -> [Node]
    const bidir = xs => {

        // isRTL :: String -> Bool
        const isRTL = s =>
            /^[֐-׿؀-ۿ܀-ݏֿ\s]+$/.test(s) &&
            !(/^\sֿ*$/.test(s));

        // isNum :: String -> Bool
        const isNum = s =>
            !isNaN(s) && /\d/.test(s);

        const dctRuns = foldr(
            (x, a) => {
                const
                    s = x.root.text,
                    blnInRTL = a.ltr.length > 0;
                return isRTL(s) ? { // APPENDED TO LEFT
                    rtl: a.rtl.concat(x),
                    ltr: a.ltr
                } : isNum(s) ? { // AT LEFT IN RTL RUN, AND VICE VERSA
                    rtl: blnInRTL ? a.rtl.concat(x) : [],
                    ltr: blnInRTL ? a.ltr : [x].concat(a.ltr)
                } : {
                    rtl: [], // APPENDED TO RIGHT
                    ltr: [x].concat(a.rtl).concat(a.ltr)
                };
            }, {
                rtl: [],
                ltr: []
            },
            xs
        );

        // MAIN (MIXED) ACCUMULATOR WITH ANY RESIDUE RIGHT-APPENDED
        return dctRuns.ltr.concat(dctRuns.rtl);
    };
    ```

  15. 15 Posted by robintrew on 14 Apr, 2018 07:43 PM

    robintrew's Avatar

    Visual summary of how MultiMarkdown discards the user's Unicode display sequence information, and displays data in the wrong columns

    (PNG file below)

  16. 16 Posted by robintrew on 14 Apr, 2018 08:08 PM

    robintrew's Avatar

    unicode.txt source file for examples above attached below.

    Note that MMD is not *entirely* alone in its lack of compliance with Unicode display sequence.

    A small number of other applications will also fail to display a bidirectional text file in the way in which it appears in the Bash shell, or in macOS Quicklook.

    Most notorious amongst these lagging applications are the current build of Sublime Text (at the time of writing still needs a plugin for full compliance with Unicode bidirectional text display), and TextEdit (old and under-maintained).

    The file below will, however, be displayed correctly by:

    - The Bash shell (cat etc)
    - macOS Quicklook
    - Atom
    - Pages
    - Omni Applications
    - TaskPaper
    - MS Office
    - In fact all fully Unicode-compliant applications.

  17. 17 Posted by robintrew on 14 Apr, 2018 08:09 PM

    robintrew's Avatar

    unicode.txt

  18. 18 Posted by robintrew on 14 Apr, 2018 08:12 PM

    robintrew's Avatar

    To raise current builds of Sublime 3 to full Unicode display sequence compliance:

    https://packagecontrol.io/packages/Bidirectional%20text%20support

    ( Atom is already fully compliant )

    https://atom.io/

  19. Support Staff 19 Posted by fletcher on 14 Apr, 2018 11:58 PM

    fletcher's Avatar

    You are conflating writing text in a word processor with writing in a programming language (and MD/MMD is a programming language). And in particular, your fundamental example has to do with a string of characters meant to represent a 2-dimensional table, which is not done this way in word processors.

    The attached image is a screenshot of LibreOffice using the table described in my example, which is only "absurd" because it demonstrates the incorrectness of your position. It shows the relationship between Arabic numerals, English words, and Hebrew words (again assuming the correctness of Google Translate).

    Copying that text from LibreOffice and pasting it into MultiMarkdown Composer as a table results in:

    | 1 | 2 | 3 |  
    | ---   | ----- | ----------    |  
    | one   | two   | three |  
    | אחד    | שתיים    | שְׁלוֹשָׁה  |
    

    Because of the RTL characters inside the table, the entire last table row is displayed in RTL script, rather than displaying each word RTL in an overall line of text which is LTR. I don't care that CoreText does this "incorrectly", because it is an edge case specific to MultiMarkdown and I would not expect the behavior to change for this example. In other contexts, the resulting behavior would probably be correct.

    However, it is important when dealing with MultiMarkdown to understand exactly what is going on, and to understand that LTR characters (e.g. |) are being displayed in a RTL fashion, because it is not immediately visually obvious.

    When processing the example with MultiMarkdown, the result is this:

    1 2 3
    one two three
    אחד שתיים שְׁלוֹשָׁה

    Once again, this shows the proper result. It demonstrates that a table copied from a spreadsheet, converted into MultiMarkdown text, and then converted into HTML maintains the proper relationship between table columns.

  20. 20 Posted by robintrew on 15 Apr, 2018 12:07 AM

    robintrew's Avatar

    No. Your example is absurd because your complaint is that RTL and LTR rows and cell sequences don't match each other vertically.

    You keep referring to 'word processors' instead of Unicode.

    This has nothing to do with 'word processors'. Simply Unicode.

    Look at Bash and Quicklook.

    There are some programming languages (not entirely sure that I would glorify a filter like MMD with a title like 'programming language' – you think it's Turing-complete ?) which trail behind the mainstream of standards-compliance, and if you are happy to:

    - Lag behind Bash and Quicklook etc on Unicode compliance
    - discard user information, and
    - scramble user data,

    then that is, of course, your privilege :-)

    Have fun !

    (But make the most of it while you can – standards-compliant programming editors like Atom, and OS shells like Bash, don't discard or misrepresent Unicode display direction information, and out-of-the-box Sublime won't continue to do so for long – the water is already running out of that bath, I'm afraid ...)

  21. 21 Posted by robintrew on 15 Apr, 2018 12:38 AM

    robintrew's Avatar

    And, incidentally, I've now written a pre-processor to protect the Unicode display sequence information in tables from destruction by MMD (deliberate and by design, I understand from your explanations ...).

    So, while MMD remains a slightly broken instrument for translating plain text documents into HTML documents, I do now have the band-aid for the MMD filter, and the prophylaxis for my data, that I need.

    Thank you for your time and effort. Have a good weekend.

  22. 22 Posted by robintrew on 15 Apr, 2018 08:11 AM

    robintrew's Avatar

    and good luck, incidentally, with finding (or writing ?) a replacement for Core Text to use in your MMD Composer editor.

    You expressed misgivings about its Unicode-compliance, and if you feel that failing to discard Unicode display-sequence information is below the dignity of a 'programming language' like your MMD filter, then you probably need to look for an editor which agrees with you.

    At the moment your customers are ill-served by the fractured relationship between the Unicode-compliant editing view, and the Unicode-defiant (information discarding, and display scrambling) output:

  23. 23 Posted by robintrew on 15 Apr, 2018 08:12 AM

    robintrew's Avatar

    Incompatible left and right hand views in MultiMarkdown Composer

  24. Support Staff 24 Posted by fletcher on 15 Apr, 2018 04:46 PM

    fletcher's Avatar

    I would need the actual text for your last example, because the visual image produced by Core Text is insufficient to determine the actual text string.

  25. 25 Posted by robintrew on 15 Apr, 2018 04:48 PM

    robintrew's Avatar

    > I would need the actual text for your last example

    You already have it.

    ( Post 17 )

  26. Support Staff 26 Posted by fletcher on 15 Apr, 2018 05:41 PM

    fletcher's Avatar

    That is an equivalent example to what you have sent before, except this time it includes digits which are also treated by Core Text as part of the RTL text, rather than properly treating them as LTR, since | characters serve as delimiters in MMD and should "reset" the flow.

    Core Text's behavior isn't a bug per se, as it was not intended to understand MD/MMD. But it's handling of LTR vs RTL text is poorly suited to handling MMD tables, since it doesn't understand the semantic meaning of the | characters in this context, and is not capable of properly isolating RTL runs of text from the surrounding LTR characters (pipe delimiters, etc.).

    I have given several examples demonstrating the internal consistency of MMD, as well as its ability to round trip data from a table (LibreOffice) to text (MMD) and back to a table (HTML, LaTeX, ODT, etc.).

    I get that you don't like this explanation, but you have yet to demonstrate an actual problem with my examples. I'm not sure that I have any additional ways to continue to try to explain this to you, so I refer you back to the LibreOffice example and would recommend that you experiment with interconversion between tables in spreadsheets, MMD text, and HTML until you satisfy yourself that the data copied between them is maintained between formats.

    At this point, this discussion seems to be going nowhere, so I will sign off.

  27. 27 Posted by robintrew on 15 Apr, 2018 05:58 PM

    robintrew's Avatar

    > digits which are also treated by Core Text as part of the RTL text, rather than properly treating them as LTR

    No. They are **properly** treated as part of the RTL text, in competent and straightforward accordance with the Bidirectional algorithm which is part of the Unicode standard.

    http://unicode.org/reports/tr9/

    Core Text, unlike your MMD filter, treats these digits absolutely correctly in terms of the Unicode standard, and displays them absolutely correctly.

    > [Core Text]'s handling of LTR vs RTL text is poorly suited to handling MMD tables

    On the contrary - MMD is poorly suited to handling the Unicode-compliant plain text which your customers work with.

    Your product **is** the relationship between a standards-compliant plain text display and a display in HTML and other formats.

    That relationship is broken.

    To blame the Unicode standards for failing to understand the 'internal consistency' of MMD, and to turn a blind eye to the reality that you are discarding information which users have encoded in their plain text files, is to grandly proclaim that you alone are in step, and the rest of the world is marching out of step.

    The Unicode standards do not fail to understand MMD.

    MMD fails to understand the Unicode standards.

    It performs a lossy translation, high-handedly and self-righteously discarding information encoded in the user's plain text.

    Easily fixed, but you clearly don't want to fix it. Happy marching.

  28. 28 Posted by robintrew on 15 Apr, 2018 06:12 PM

    robintrew's Avatar

    WYSIWYG failure in @MultiMarkown is blamed on:

    > "digits which are also treated by Core Text as part of the RTL text, rather than properly treating them as LTR"

    Wrong. Core Text is correctly doing **exactly** what Unicode specifies for digits:

    http://unicode.org/reports/tr9/ https://twitter.com/ComplexPoint/status/985581116542345222/photo/1

  29. 29 Posted by robintrew on 15 Apr, 2018 06:34 PM

    robintrew's Avatar

    Incidentally you may need to extend your suspicions and unease about Unicode-compliant plain text beyond the behaviour of just Core Text:

    > Not a bug per se ... but ...
    > Arguably shouldn't ...

    to Bash cat, vi and emacs etc. View the unicode.txt file in any of these and you will see the same.

    If there is indeed a conspiracy against your right-append vision of what it is proper and consistent for MMD to do, then it goes well beyond the Core Text editor in MultiMarkdown Composer, I'm afraid ...

  30. 30 Posted by robintrew on 15 Apr, 2018 07:13 PM

    robintrew's Avatar

    PS spreadsheets are irrelevant to the relationship between bidirectional text and MMD tables.

    Spreadsheet grids are not bidirectional.

    Text is.

    If you feel that MMD is a tool for translating spreadsheets rather than documents, then perhaps you should sell MultiMarkdown Composer with a spreadsheet control to the left, rather than with a text editing control.

    The text editor's high level of compliance with the Unicode standard seems to be causing you some discomfort.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac

Recent Discussions

22 Apr, 2018 11:04 PM
13 Apr, 2018 10:58 PM
09 Apr, 2018 09:47 PM
09 Apr, 2018 04:28 PM
19 Feb, 2018 10:29 PM

Recent Articles