Line # Revision Author
1 98 abw #========================================================================
2 #
3 # TODO
4 #
5 # DESCRIPTION
6 460 abw # TODO list for the Template Toolkit version 2.07c, containing
7 98 abw # known bugs, limitations, planned enhancements, long term visions
8 # and a few whacky ideas.
9 #
10 # AUTHOR
11 # Andy Wardley <abw@kfs.org>
12 #
13 #------------------------------------------------------------------------
14 # $Id$
15 #========================================================================
16 26 abw
17 66 abw #------------------------------------------------------------------------
18 98 abw # Miscellaneous
19 43 abw #------------------------------------------------------------------------
20
21 287 abw * mylist.sort.uniq
22
23 131 abw * Errors thrown via the Error module are not correctly caught. I looked
24 at this briefly but the problem wasn't immediately obvious and needs some
25 182 abw more considered investigation. There are also problems with CGI::Carp
26 throwing errors that don't get properly caught.
27 131 abw
28 98 abw * The 'eval' filter leaks memory, as reported by Colin Johnson. The
29 filter subroutine created contains a reference to the context and then
30 gets cached in the FILTER_CACHE item of the context. Hey presto -
31 circular references. The reset() method should probably clear the
32 FILTER_CACHE. Also need to check the plugins cache for similar
33 problems.
34 51 abw
35 124 abw * The handling of the DELIMITER parameter could be improved. At the
36 moments it's hardcoded and hacked to Do The Right Thing for Win32
37 but I'd prefer it to Do The Good Thing.
38 27 abw
39 43 abw * If you use 'ttree' with a COMPILE_EXT or COMPILE_DIR option then
40 templates in the 'lib' directories will be compiled, but those in
41 the src directories will not. This is because ttree does a chdir()
42 to the src directory and processes files as './myfile'. TT doesn't
43 compile RELATIVE files by default.
44 27 abw
45 66 abw * No recursion checking is performed for BLOCKs, only
46 27 abw Template::Document instances. This is probably the way it will stay
47 (unless anyone shouts loudly enough) but it should be documented
48 66 abw anyway. STOP PRESS: I had an idea that bare BLOCK subs should be
49 blessed into Template::Document class to allow $template->process()
50 to be called regardless. Template::Document methods would need to
51 test $self for CODE/HASH and Do The Right Thing. This would then
52 allow recursion testing for BLOCKs as well as Template::Document
53 objects.
54 27 abw
55 131 abw * It would be nice if there was an option so that the files generated
56 under the COMPILE_DIR are relative to the INCLUDE_PATH and not absolute.
57 This could cause potential conflicts (e.g. if INCLUDE_PATH changes
58 between sessions and the same files in different INCLUDE_PATH dirs
59 maps to the samed compiled version) but is convenient for those times
60 when you know that's not going to be a problem.
61 121 abw
62 138 abw * Richard Tietjen's patch for stash replace. Allows back references
63 (e.g. $1) but it would be nice to find a rock-solid way to implement
64 182 abw it without relying on unusual ^A delimiter character.
65 138 abw
66 * Further to the above, Craig Barratt has this solution which will be
67 182 abw going into the next verion (2.05b) unless anyone has any further
68 138 abw suggestions to make before then.
69
70 It would be great if replace handled backreferences. I don't like
71 the ^A solution since the string could contain ^A, plus it is a
72 security hole. The attempt I posted only works for up to 9
73 backreferences and doesn't handle an escaped '\$' and uses nested
74 evals:
75
76 $str =~ s{$search}{
77 my $r = $replace;
78 my @d = (0, $1, $2, $3, $4, $5, $6, $7, $8, $9);
79 $r =~ s/\$(\d+)/$d[$1]/eg;
80 $r;
81 }eg;
82
83 I wish there was a perl predefined variable array containing all
84 the backreferences (is there one?). You can avoid the hard-coded
85 limit of 9 with extra evals, and a bit of work on the re could
86 handle the escaped '\$' case, so maybe that would be good enough.
87
88 * Craig also notes, in fixing the problem with NEXT not working
89 inside SWITCH (see Changes v2.04):
90
91 By the way, I came across another arcane bug:
92
93 NEXT FOREACH k = [1];
94
95 is legal syntax but is an infinite loop, since $_[0]->{ INFOR } in
96 Parser.yp is not set when the NEXT is parsed, so it generates a
97 plain "next;" rather than calling $factor->next(). I don't see an
98 easy, clean fix.
99
100
101 98 abw #------------------------------------------------------------------------
102 # Documentation
103 #------------------------------------------------------------------------
104
105 * Extend the FAQ.
106
107 * Document the Splash! library properly, once the interface is a little
108 more settled.
109
110 * Examples for libraries (HTML, Splash & PostScript) should be integrated
111 into the documentation.
112
113
114 #------------------------------------------------------------------------
115 # Directives
116 #------------------------------------------------------------------------
117
118 * A 'FOR', like 'FOREACH' but without using an iterator. You wouldn't get
119 the 'loop' reference to test 'first', 'last', etc., against, but it would
120 be faster for those cases when you didn't need it. This will likely
121 be implemented as a facility feature (see later).
122
123 * PRINT should be defined as a new directive, doing what the print()
124 method of Template::View currently does (the Right Thing).
125
126 [% PRINT node %] === [% tt.view.print(node) %]
127
128 NOTE TO SELF: this is a Very Good Idea [tm]. PRINT becomes the way to
129 display a data structure (e.g. hash, list, XML element, MyThingy, database
130 record, etc.) in an "intelligent" fashion. Implemented underneath via
131 the current default VIEW.
132
133 * ARGS. There may be a requirement for reusable template components
134 to define what variables they plan to use. This would allow some
135 optimisation and also possibly help to avoid global variable clashes.
136 Would also be a useful "comment" directive for human readers and maybe
137 also help in debugging (WARNING: expected 'title' argument).
138
139 [% ARGS title # no default
140 bgcol='#ffffff' # default value
141 %]
142
143
144 #------------------------------------------------------------------------
145 # Stash
146 #------------------------------------------------------------------------
147
148 287 abw * The XS Stash does not work with tied hashes (e.g. DBI.tie). Also note
149 174 abw that enabling the XS Stash cause all the Template::* modules to be
150 installed in an architecture-dependant directory. For info, see
151 http://www.tt2.org/pipermail/templates/2001-September/001568.html
152
153 182 abw * Jonas Liljegren reports a segfault when using the XS Stash under
154 Apache/mod_perl with certain undefined variables. At the time of
155 writing detail is scarce. Check the mailing list for further
156 details.
157
158 98 abw * Stas Bekman raised the issue of the Stash not being able to correctly
159 differentiate between scalar/list context and in particular, the
160 157 abw cgi.param examples not working as expected. This is fixed in v3 and
161 in Craig's Stash/Context.pm which does the right lookahead to allow
162 'scalar' and 'list' postfix operators. e.g. cgi.param.list
163 98 abw
164 * Have stash, etc., add current template name/line when reporting errors.
165 (may be tricky under the current implementation)
166
167
168 #------------------------------------------------------------------------
169 # Parser
170 #------------------------------------------------------------------------
171
172 * Lists don't accept arbitrary expressions as elements. In other words
173 138 abw you can't do this: [% foo(bar + 1) %]. This has been fixed in the v3
174 parser. See http://www.tt2.org/v3/
175 98 abw
176 73 abw * The parser isn't as intelligent as it could be about blocks of template
177 code commented out en masse. The pre-scanner find the first terminating
178 98 abw END_TAG after an opening tag, regardless of it being on a
179 73 abw commented line or not.
180 e.g.
181 [%#
182 #
183 # [% INCLUDE blah %] <- directive ends here
184 # foo <- this gets printed
185 %]
186 28 abw
187 98 abw * Allow { and } as block delimiters, replacing for the ugly ';' and
188 big, chunky [% END %] approach.
189 78 abw
190 98 abw e.g. something like:
191 78 abw
192 98 abw [% FOREACH a = [ 1 2 3 ] %]
193 [% IF b == a %]
194 [% INCLUDE foo %]
195 [% ELSE %]
196 [% INCLUDE bar %]
197 [% END %]
198 [% END %]
199 78 abw
200 98 abw could be written as:
201
202 [% FOREACH a = [ 1 2 3 ] {
203 IF b == a {
204 INCLUDE foo
205 }
206 ELSE {
207 INCLUDE bar
208 }
209 }
210 %]
211
212 78 abw * Ability to set different parser options for BLOCK definitions, etc.
213
214 [% BLOCK header
215 eval_perl = 0
216 pre_chomp = 1
217 %]
218 ...
219 [% END %]
220
221 Anonymous BLOCK can then be used to set a parser scope
222
223 [% BLOCK trim=1 %]
224 98 abw ...
225 78 abw [% END %]
226
227 [% BLOCK trim=0 %]
228 98 abw ...
229 78 abw [% END %]
230
231 And/or set different tag styles, etc.
232
233 [% BLOCK tags='star' %]
234 [* INCLUDE this_is_a_directive *]
235 [% INCLUDE this_is_not %]
236 [* END *]
237
238 [% INCLUDE back_to_normal %]
239
240 131 abw This is likely to be a TT3 feature and I've already got the basic
241 parser for this up and running. It might get back-pactched into
242 version 2, otherwise you might have to wait for the first alpha
243 release of verion 3.
244 78 abw
245 138 abw * Craig Barratt reports the following:
246 98 abw
247 138 abw I looked at Parse.yp to see how hard it would be to push FILTER
248 evaluation down into the expr rule, so that you could put filters
249 inside expressions (eg: using repeat() just like &quot;x&quot; in
250 perl). More about that later.
251
252 In browsing through Parser.yp I noticed several issues:
253
254 - The operator precedence is very different to perl, C etc.
255 For example, these expressions evaluate differently in
256 TT2 versus perl, C etc:
257
258 + "1 || 0 && 0" evaluates to 0 in TT2 and 1 in perl or C.
259 TT2 parses it as (1||0) && 0; in perl and C && is higher
260 precedence than ||.
261
262 + "1 + !0 + 1" evaluates to 1 in TT2 and 3 in perl or C.
263 TT2 parses it as 1 + !(0 + 1); in perl and C ! is higher
264 precedence than +.
265
266 + Many other expressions parse incorrectly, but the effect
267 is benign since most rules return flat text that perl
268 correctly re-parses. Eg, 2 * 3 + 4 is incorrectly parsed
269 as (2 * (3 + 4)), but happily just the string "2 * 3 + 4"
270 is compiled by perl, which correctly evaluates it as
271 (2 * 3) + 4.
272
273 - There is no unary minus and the NUMBER token is signed. So you can
274 write "x = -2;" but not "x = -y;". Moreover, "x = 1 -1;" is a syntax
275 error (since "1 -1" returns just two tokens NUMBER, NUMBER). (As a
276 workaround you can rewrite these as "x = 0-y;" and "x = 1 - 1".)
277
278 - You cannot have expressions in lists ([..]) and function arguments.
279
280 I have modified the Parser.pm (to make NUMBER unsigned) and modified
281 Grammar.pm.skel and Parser.yp to fix most of these issues (improved
282 operator precedence, unary minus and plus), and also to allow
283 expressions in a few more places (eg: range). But the last item
284 has me stuck.
285
286 The parse rules for lists and function arguments make COMMA optional,
287 so you can equivalently write [1 2 3 4] or [1,,,,,2 3 4] or [1,2,3,4].
288 This makes it very difficult to make each term an expression, because
289 the resulting grammar has many ambiguities. For example, is [1 -1]
290 two elements [1, -1] or a single element [0]? One partial solution is
291 to move the bracketed expression rule '(' expr ')' to the term rule,
292 allowing expressions to be included via parens. But there are also
293 ambiguities, eg: does [foo (1+1)] have 2 elements or is it a function
294 call to foo?
295
296 Without allowing expressions in lists or function arguments, the unary
297 minus change I've made means that the NUMBER token is unsigned, so with
298 my changes you cannot write [-1, 2, 3]. Not a good thing.
299
300 One solution is to change the grammar so that COMMAs are required in
301 lists and arguments, but that would break several test cases and
302 probably break lots of old templates. But this might be the only
303 way to produce a grammar that is a lot more similar to perl.
304
305 Another solution is to ignore these issues altogether and use temporary
306 variables to precompute expressions that you need in lists or function
307 arguments, or use explicit lvalue assignments, eg:
308
309 foo(x + 2); becomes temp = x + 2;
310 foo(temp);
311
312 or
313
314 List = [x+1,x+2,x+4]; becomes List = [];
315 List.0 = x+1;
316 List.1 = x+2;
317 List.2 = x+4;
318
319 Both of these look ugly to me.
320
321 Back to the FILTER issues. Ultimately I'd like to be able to embed filters
322 as low precedence operators in expressions, and write:
323
324 List = [
325 "foo" | repeat(10),
326 "bar" | repeat(10)
327 ];
328
329 but I doubt there is a non-ambiguous upward compatible grammar that
330 supports this.
331
332 Comments?
333
334
335 78 abw #------------------------------------------------------------------------
336 98 abw # Plugins
337 27 abw #------------------------------------------------------------------------
338
339 98 abw * We need a way to easily enable/disable certain plugins. This should
340 131 abw be addressed by facility provision. Probably something for v3.
341 98 abw
342 * The Text::Autoformat module has some problems with versions of Perl
343 prior to 5.6.0 when using a locale which has a decimal separator
344 other than '.' (e.g. Swedish, which uses ','). Damian has been made
345 aware of the problem (and I note he now has a new version out which
346 I need to check). For now, the Makefile.PL issues a warning but
347 continues regardless.
348
349 * The Template::Plugin DBI iterator first/last() methods don't behave
350 the same as list first/last(). Randal also reports that get_all()
351 doesn't work as it should - may be a conflict in code/docs?
352
353 66 abw * PLUGINS could accept a reference to an object which is used as a
354 98 abw singleton factory for a plugin. (NOTE: 2.01 includes PLUGIN_FACTORY
355 to implement this, but currently undocumented because it's likely to
356 change).
357 66 abw
358 98 abw * Add Leo & Leon's Page plugin, or try to find some way of implementing
359 it in terms of the Table plugin. I think the jury is still out on the
360 matter of whether it counts as duplicated functionality.
361 66 abw
362 131 abw * A more general solution for XML (e.g. DOM, XPath, etc) would be for
363 TT to support a PerlSAX handler which generates the appropriate
364 callbacks to the view. This should make it possible to easily
365 display XML content from XML::DOM, XML::XPath, or any other SAX
366 compliant source.
367 66 abw
368 98 abw Something like this:
369 70 abw
370 98 abw # define a view
371 [% VIEW my_view
372 prefix="my/xml/dom/path/" ;
373 END
374 %]
375 70 abw
376 98 abw # get some XML
377 [% USE dom = XML.DOM %]
378 [% doc = dom.parser(my.files.xmldata) %]
379
380 # ask the view to print the data
381 [% my_view.print(doc) %]
382 27 abw
383 98 abw The view print() method will call the relevant 2SAX method on the
384 XML node, passing a SAX2TTView handler to make the relevant calls
385 back to the view to display parts of the XML data model as SAX events
386 are received.
387 27 abw
388
389 98 abw #------------------------------------------------------------------------
390 # Views
391 #------------------------------------------------------------------------
392 66 abw
393 98 abw The current implementation is there to get me (and anybody else who's
394 interested) using it and trying to identify the problems, requirements
395 and general issues involved. I've got a better idea now about what a
396 VIEW should be in notional terms, but I'm still not quite sure about
397 the syntax and API.
398 11 abw
399 98 abw General thoughts:
400
401 * A view defines a set of templates. Things like prefix, suffix,
402 default, etc., can be specified to customise template selection.
403 In this sense, it is like a custom provider of those templates.
404 It implements the template() method to fetch a template according
405 to those rules.
406
407 * It is also a custom processor of those templates. It implements the
408 process() method. In this sense, it is like a custom context.
409
410 * It also implements dispatch logic to apply the right template to the
411 right kind of data. It does this via the print() method. It may
412 have all kinds of custom dispatch logic.
413
414 * A view takes responsiblity for things template related as opposed
415 to anything data related (stash) or application logic related
416 (plugins, runtime code, etc). It is the user interface facility
417 within the engine.
418
419
420 78 abw #------------------------------------------------------------------------
421 98 abw # Splash!
422 78 abw #------------------------------------------------------------------------
423 10 abw
424 98 abw The current implementation is a fairly ugly hack to get something up
425 and running that's good enough to use. It's mainly template driven
426 and doesn't scale very well, particularly with global variables
427 clashing all over the place. My plan is that this will become a
428 view-based system and will no doubt be a test-bed for the
429 implementation of the view facility.
430 78 abw
431 98 abw To include:
432 78 abw
433 98 abw * Variable management for storing metadata relating to an
434 interface/view, protected from the rest of template space.
435
436 * Also, style management for defining different styles (e.g. plain,
437 fancy, icecold, whitehot) for different interface elements (e.g.
438 bars, borders, menu text) in different modes (e.g. selected, unselected),
439 or under different, possibly custom conditions (e.g. user preferences,
440 guest or logged in, etc.).
441
442 * Clearly define API for different elements, allowing people to write
443 apps to the API which run across different conformant widget sets.
444
445 124 abw * support themes which define a set of styles
446 98 abw
447 124 abw * May be wise to move Splash out to a separate distribution.
448 98 abw
449 131 abw Randal Schwartz highlighted some problems with non-compliant HTML
450 being generated. These include:
451 98 abw
452 131 abw * no DOCTYPE declaration (added to html/header)
453
454 * ALT attribute missing from many <img> tags
455
456 * <font ...><table>...</table></font> is illegal (not sure where this
457 gets done)
458
459 * <tr> shouldn't have HEIGHT attribute
460
461 * <H3> block element inside inline element
462
463 * <A> not allowed here (not sure)
464
465
466 98 abw #------------------------------------------------------------------------
467 # Test Suite
468 #------------------------------------------------------------------------
469
470 124 abw * t/file.t and t/directry.t are currently disable on Win32 until I get
471 a chance to fix a couple of minor bugs relating to '/' vs '\' file
472 separators.
473 98 abw
474
475 #------------------------------------------------------------------------
476 # Facilities
477 #------------------------------------------------------------------------
478
479 Core facilities currently implemented by Template::Context should be
480 moved out into separate facilities. These currently fall into the
481 categories of things like stash, view, plugins, filters, parser and
482 maybe some others. (NOTE: this might tie in very closely with Camelot
483 and the resource/presenter/actor breakdown, aka model/view/controller).
484
485 * 'view' would handle template fetching and processing. It is the view
486 that talks to a provider, possibly adding special prefixes, suffixes,
487 doing default templates, special dispatch logic, etc.
488
489 * 'stash' is responsible for storing variables, as it currently is.
490
491 * 'plugins' is responsible for fetching plugins.
492
493 * 'filters' is responsible for fetching filters.
494
495 * maybe both the above would get bundled into something like 'logic'?
496
497 * 'parser' would make parts of the parser accessible
498
499 * 'output' could be used to generate output
500
501 There would be some facility to install, customise and remove facilities
502 to modify TT behaviour as required. This would allow us to disable
503 certain plugins, or remove the plugin facility altogether, for example.
504 Or we could install a new stash facility which generated a different kind
505 of code (e.g. less magic, more speed). Or install a new custom facility
506 to do some application or domain specific task.
507
508 Facilities should be accessible via the context:
509
510 my $stash = $context->stash(); # currently works
511 my $view = $context->view(); # not yet
512
513 General form:
514
515 my $xyz = $context->facility('xyz');
516
517 Multiple form:
518
519 my ($a, $b, $c) = $context->facility(qw[ a b c ]);
520
521 This can then be written into generated code pretty much as the stash
522 currently is. The facilities would define the code generators that
523 currently clutter up Template::Directives. They would ensure that the
524 facility is scheduled to be requested from the context at the start of
525 the sub:
526
527 sub {
528 my $context = shift;
529 my ($stash, $view) = $context->faciity(qw( stash view ));
530
531 and then transform the various directives into appropriate callbacks
532 into the facility:
533
534 $output .= $stash->get('x');
535