[LONG][RFC] WML 2.0.0, quotes and tags

[Denis Barbier <nospam@thanx>]

You notice that many bug reports are identical in substance, i.e.
problems with tags in attributes.

1. Which problems do mp4h solve?
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

As i already wrote, i was unhappy with Meta-HTML.  Especially i was
unable to
* pass attributes with same names, as in
    <foo name="<get-var name>">
* understand when attributes were expanded or not
* predict whether syntax like
    <ifeq 0 0 <: print "Good" :>>
  works or not.
* add new tags in mhc.

Those who wrote WML tutorial with WML know it was a pain, because of the
syntax of define-tag.

So i decided to implement my own parser to fit our goals (better
interaction with other WML passes).

2. Tag expansion with mp4h
=-=-=-=-=-=-=-=-=-=-=-=-=-

In this section, double quotes are used to group letters visually, do
not focus on them now.

Suppose that <baz> has been defined as
<define-tag baz>quux</define-tag>

What happens when mp4h reads
       <foo bar="<baz>">
?

* If the <foo> tag has been defined
    Input          Action             Buffer
    
    <foo        attributes are read
      bar="                           bar="
        <baz    attributes are read   bar="
        >       macro expansion
                  baz()  --> quux     bar="quux
      "                               bar="quux"
    >           stop reading attr. 
                Attr #1: bar="quux"
                macro expansion
                  foo(bar="quux")

* If the <foo> tag is undefined
    Input          Action             Buffer
    
    <foo        tag unknown, written
                like normal text      <foo
      bar="                           <foo bar="
        <baz    attributes are read   <foo bar="
        >       macro expansion
                  baz()  --> quux     <foo bar="quux
      "                               <foo bar="quux"
    >                                 <foo bar="quux">

We see that attributes are expanded first. This is very important, since
it ensures that we can type
    <foo name="<get-var name>">

And now it is easy to understand why this fails:
    <ifeq 0 0 <img src="foo.png" alt="">>

    Input                 Action                         Output
    
    <ifeq              read attributes
       0               Attr #1: 0
       0               Attr #2: 0
      <img             Attr #3: <img
      src="foo.png"    Attr #4: src="foo.png"
      alt=""           Attr #5: alt=""
    >                  stop reading attr.
                       macro expansion
           ifeq(0, 0, <img, src="foo.png", alt="")        <img
    >                  nothing special                    >


This was about attributes.  You may now want to know how complex tags
are handled too. 
  <define-tag foo endtag=required>x:%body:x</define-tag>
  <define-tag bar endtag=required>y:%body:y</define-tag>
  <foo><bar>baz</bar></foo>

    Input                 Action                         Output
    
    <foo>              complex tag, read body
       <bar>           complex tag, read body
          baz          Body: baz
       </bar>          tag is complete
    </foo>             Body: <bar>baz</bar>
                         foo(<bar>baz</bar>)
                       replaced by
                         x:<bar>baz</bar>:x
    x:                 normal text                       x:
    <bar>              complex tag, read body
       baz             Body: baz
    </bar>               bar(baz)
                       replaced by
                         y:baz:y
    y:baz:y:x          normal text                       y:baz:y:x

Body is first collected without expansion, after that function is
evaluated.  As body tags is not the key point, i won't go into deeper
details in this post.

3. Quoting
=-=-=-=-=-

With Meta-HTML we write
   <ifeq 0 0 "<href url=\"foo.png\">">

Now remember that with mp4h attributes are evaluated *before* macro
expansion.  This is why escaping quotes does not make sense, and
   <ifeq 0 0 "<href url="foo.png">">
is perfectly valid. Here is how it works:

    Input                 Action
    
    <ifeq              read attributes
       0               Attr #1: 0
       0               Attr #2: 0
      "                start of a group
      <href            known tag, read attributes
         url="foo.png" Attr #1: url="foo.png"
      >                stop reading attr.
                         href(url="foo.png")
                       replaced by
                         <a href="foo.png"><tt>foo.png</tt></a>
      "                end of a group
                       This group contains
                       <a href="foo.png"><tt>foo.png</tt></a>
    >                  stop reading attr.
                       macro expansion
           ifeq(0, 0, <a href="foo.png"><tt>foo.png</tt></a>)
                       replaced by
                         <a href="foo.png"><tt>foo.png</tt></a>

On the other hand, we must write
    <ifeq 0 0 "<img src=\"foo.png\" alt=\"\">">
and not
    <ifeq 0 0 "<img src="foo.png" alt="">">
because in this case text within quotes is ordinary text, this is not a
tag with attributes.

4. What must be changed?
=-=-=-=-=-=-=-=-=-=-=-=-

I agree with your bug reports, it is a very bad idea to have different
behaviours depending whether tags are defined or not.

Of course it is easy to consider that unknown tags are simple tags, this
will fix
    <ifeq 0 0 <img src="foo.png" alt="">>

But it breaks
    <: print "<img src=\"" . $file . "\" alt=\"" . $alt ."\">"; :>

And even worse, consider
    <: print <<EOT;
    Hey, how to finish this pseudo EOT tag?
    EOT
    :>

In fact i focused my view on these last topics, to ease mixing ePerl and
mp4h, but i did not realize that this syntax was confusing.
Now that the problem is clearly (?) expressed, it should be solved
quickly ;-)

-- 
Denis Barbier
WML Maintainer

______________________________________________________________________
Website META Language (WML)                www.engelschall.com/sw/wml/
Official Support Mailing List                   sw-wml@engelschall.com
Automated List Manager                       majordomo@engelschall.com