Release Meeting: 19 Mar 2018

(08:00:37 AM) gac410: Hi everyone   Good morning / day ...   Ready for a release meeting?
(08:30:15 AM) MichaelDaum: Hi George. Bit early, isnt it. We still did not switch over to daylight saving times over here ;)
(08:30:43 AM) gac410: 1300Z    That never changes.   Everyone else switches around it.
(08:31:08 AM) gac410: So it's an hour earlier for me too  :(  (Which I really hate)
(08:31:28 AM) MichaelDaum: atm it is 12:31 utc
(08:33:26 AM) gac410: crap.   I calculated it backwards.  And I should know better.   I'm a pilot and have to do all flight planning etc. on UTC    
(08:35:58 AM) MichaelDaum: there goes your license :D
(08:36:10 AM) gac410: And to think I could have gotten another hour of sleep.   ;)
(09:02:05 AM) gac410: Hi everyone ... let's try this again now that I'm in the right timezone.    Good Morning / good day.    Time for a release meeting?
(09:04:32 AM) ***MichaelDaum on a phone call  ... give me a sec
(09:11:53 AM) MichaelDaum: Hi gac410
(09:11:58 AM) MichaelDaum: sorry for the delay
(09:12:05 AM) gac410: Hi MichaelDaum   No problem.  
(09:12:24 AM) MichaelDaum: had a discussion about TinyMCE/WysiwygPlugin and ExplicitNumberingPlugin
(09:12:41 AM) vrurg: Hi all
(09:12:43 AM) gac410: I saw some of that in a task IIRC
(09:12:48 AM) gac410: Hi vrurg
(09:13:05 AM) MichaelDaum: Ive added my analysis to this bug report: https://foswiki.org/Tasks/Item11751
(09:13:17 AM) MichaelDaum: which is ... yikes ... 6 years old
(09:13:24 AM) gac410: y,   time flies 
(09:14:24 AM) MichaelDaum: I'll investigate a new plugin to cover this feature, called ImplicitNumberingPlugin.
(09:14:48 AM) gac410: hm   Another new one vs, just fixing the old??
(09:14:52 AM) MichaelDaum: it leaves heading notation untouched and just changes rendering of headings as they occur.
(09:14:59 AM) gac410: Ah...
(09:15:34 AM) MichaelDaum: ExplicitNumberingPlugin is just fine in its own... however bugs are in the core as well as in WysiwygPlugin
(09:15:45 AM) MichaelDaum: the latter is broken by design and wont cut it as it is now
(09:16:03 AM) MichaelDaum: for more details see Item11751
(09:16:16 AM) gac410: y, and when bugs cross over between core, default extensions and others,  it make it challenging.
(09:17:15 AM) MichaelDaum: there are two approaches we could take wrt core and core plugins: (1) remove the broken and incomplete support for explicit numberiung (2) embrace ExplicitNumberingPlugin and extend TML
(09:17:34 AM) MichaelDaum: thats why I bring it up here
(09:18:02 AM) gac410: Numbering headers is a very nice feature.  So trying to clean up and extend what we've got makes good sese.
(09:18:04 AM) gac410: sense
(09:19:16 AM) MichaelDaum: we can compare two sets of efforts to implement and fix numbering headers
(09:19:56 AM) MichaelDaum: (1) implement an ImplicitNumberingPlugin that has got a makro to influence how headings are rendered 
(09:20:21 AM) MichaelDaum: (2) fix core and WysiwygPlugin thus making ExplicitNumberingPlugin's heading notation a first class TML citizen
(09:21:12 AM) gac410: I somewhat lean towards #2,   as it (hopefully) preserves what people have been doing with the old plugin.
(09:21:36 AM) MichaelDaum: my guts say #1 is less effort
(09:23:00 AM) gac410: Would #1 allow existing deployments of the ENP  survive, or would people have to go back and edit existing topics?
(09:23:43 AM) MichaelDaum: ENP can still be used as buggy as WysiwygPlugin is atm
(09:24:09 AM) MichaelDaum: means HTML2TML & TML2HTML destroy heading notations on the roundtrip
(09:24:38 AM) MichaelDaum: if you dont wysiwyg ENP is a fine solution today
(09:24:39 AM) gac410: hm  I wonder if there is some easy way to have Wysiwyg detect ENP syntax, and just refused to Wisywigy
(09:25:16 AM) gac410: IIRC there is a setting to cause certain html tags / attributes to block wysiwyg.  
(09:25:42 AM) gac410: I wonder if a beforeEditHandler could do something.
(09:26:39 AM) MichaelDaum: yea, well
(09:26:40 AM) gac410: ENP has some nice features.   Separate numbered sequences,   divided sequences, etc.   
(09:27:02 AM) MichaelDaum: I know. I regularly install it for all of my clients.
(09:27:13 AM) MichaelDaum: but they want wysiwyg with it
(09:28:13 AM) gac410: would your Option (1) would lose some of the ENP features?  
(09:28:30 AM) MichaelDaum: not sure. I didnt investigate yet.
(09:29:41 AM) MichaelDaum: the idea is to have some macro, say %HEADINGFORMAT{start="..." type="alpha/numeric/..." digits="1/2/3/..."}% and have it alter headings as a sideeffect ... if possible#
(09:29:54 AM) gac410: With wide deployments of ENP,  I think that making it's syntax fully supported part of TML makes a lot of sense.
(09:31:06 AM) MichaelDaum: %HEADINGFORMAT{pattern="##5..."}% for compatibility
(09:32:02 AM) MichaelDaum: just brainstorming ... not sure whether it is feasible at all to change heading format easily
(09:32:35 AM) gac410: Render.pm doesn't have a callback to allow custom formatting.    Maybe another handler is in order.   
(09:33:17 AM) gac410: Another area that a would improve a handler is a true link handler instead of just the title handler that we currently have.
(09:33:21 AM) MichaelDaum: my point of even think about implenenting an implicit numbering is that extending TML, WysiwygPlugin as well as TinyMCEPlugin ( we need a ui to alter formats ) is A LOT of work
(09:34:15 AM) gac410: Y,  if you want true wysiwgy, where the heading shows the numbers and not the markup, that would be huge.
(09:35:22 AM) MichaelDaum: preRenderingHandler seems to be the obvious place were to hook in
(09:36:25 AM) MichaelDaum: it would prefix numbers to the original heading text and then let it go down the pipe
(09:37:53 AM) MichaelDaum: something like %HEADINGFORMAT{start="1"}% ---+ foo -> ---+ 1. foo -> <h1> 1. foo </h1>
(09:39:18 AM) MichaelDaum: does that make sense?
(09:41:42 AM) gac410: I think al %MACROS% would be expanded by the time the PreRenddering hander gets control.    So %HEADINGFORMAT probably has to expand to some marker so that preRendering handler can process it in a line loop
(09:42:26 AM) MichaelDaum: ah true
(09:43:33 AM) gac410: Render.pm  processes the ---+  (and <hn...> headers   calling a subroutine that creates the anchor. 
(09:44:15 AM) gac410: Render::_makeAnchorHeading
(09:45:31 AM) ***MichaelDaum has to leave in 15 minutes
(09:45:33 AM) gac410: maybe _makeAnchorHeading  could have a  callout to a  renderHeadingHandler 
(09:45:42 AM) MichaelDaum: cool idea
(09:46:36 AM) MichaelDaum: still difficult to properly create sideeffects from macros into the rendering process ...
(09:46:53 AM) gac410: Okay.  Other release meeting topics. I started some work on https://foswiki.org/Development/AddDefaultWebName
(09:47:33 AM) MichaelDaum: how did things work out?
(09:47:36 AM) gac410: re side effects.   Yes,  I ran into this difficulty trying to figure out how to implement a macro based control of verbatim, hidden, redacted or whatever.
(09:48:10 AM) gac410: Didn't get very far,.    I created an item branch (not pushed yet),  and just the one {ConfigWebname} spec entry.
(09:48:31 AM) gac410: went down a rathole fixing unit tests I broke with an accidental spec file change a while back.
(09:48:56 AM) MichaelDaum: yikes
(09:49:33 AM) MichaelDaum: it seemed to be such as straight forward enhancment ...
(09:50:27 AM) gac410: I was thinking...   (dangerous, I know)    Maybe the System/Config web should not be in our distribution.  but ship it as a template web (admin only).  Then bin/configure, in a "onSave" handler for the {ConfigWebName} would create/populate,  or rename the config web whever the key changes.
(09:51:25 AM) MichaelDaum: booya
(09:51:35 AM) gac410: I'm going to start with just a staic populated ConfigWeb,  and work in the %CONFIGWEB% macro.    Then we can look at how to make this easier.
(09:52:39 AM) gac410: If that concept (create on config save),  maybe we could even do that for the UsersWeb   so we don't have to ship that either.
(09:54:27 AM) gac410: To get really radical.  What if ...  if the "Main" web did not exist,  foswiki would redirect to the System/WebHome.   So after bootstrap, on first save,  HomeWeb, UsersWeb and ConfigWeb all get created on the fly.
(09:54:40 AM) vrurg: That's a huge change. Too huge.
(09:56:46 AM) gac410: hm,  anyway,  how to eat an elephant...  one small bite at a time.  :D   First step.   Separate out the ConfigWeb concept.   Then play with dynamically populating it.   I think the Checker "onSave" handler is a perfect spot for that.
(09:57:20 AM) gac410: *if* that experiment works,  then some of the other webs have a possibility of handling them the same way.
(09:58:49 AM) gac410: That's how we create / move the workingDir automatically.
(09:59:10 AM) gac410: So that's been handled dynamically for a long time.
(10:02:09 AM) vrurg: Looks like a promising feature, but I'd expect too much efforts been wasted on it. Does it worth it?
(10:02:18 AM) gac410: I've always hated that we ship topics that users have to edit, so we have to build a 2nd "upgrade" package with omitted files.
(10:03:31 AM) MichaelDaum: guys, have to run. back laters. will read logs.
(10:03:41 AM) gac410: Well, the concept has been proven with the workingDir.  as a utiliy function,  it knows how to populate, and/or move a missing/renamed directory.  The idea of doing that to a web is not that much of a leap.
(10:04:01 AM) gac410: MichaelDaum:   Thanks.    I won't last much longer either.
(10:05:08 AM) gac410: I think for 2.2,  separating Home, Config and Users webs into separate concepts is a very small leap.   That needs to be the priority mainly to help finish addressing some of the security concerns we patched in 2.1.6
(10:06:28 AM) gac410: The next nibble of the elephant.   will Foswiki function in bootstrap mode with any of those 3 webs missing.   I'm not sure.  
(10:07:52 AM) vrurg: I'm not talking about the separation which was accepted already. But templating them too? Too many changes, to high risk of new bugs. You've just mentioned just one possible area for them.
(10:08:54 AM) gac410: y,  well maybe template is too complicated a word.   What I was thinking of is just copying a "distribution" version of what we ship today into a different location.
(10:09:03 AM) vrurg: I would leave templating for later, when all the hype around separation would settle down and the code would stabilize.
(10:10:01 AM) vrurg: Something I miss here then. What is the advantage of just copying them? What problem does it solve?
(10:11:14 AM) gac410: I come at that from the packaging side.    If they are in the tarball, and you change anythig, you get them reverted if you upgrade.
(10:12:58 AM) gac410: I made a proposal for this years ago,  but before we had bootstrap, and config onSave handlers.
(10:14:00 AM) vrurg: Reasonable wether it's packaging or manual installation – both would suffer same issue. 
(10:14:16 AM) gac410: https://foswiki.org/Development/ShipCommonlyTailoredTopicsInConfigure     It got rejected.
(10:14:38 AM) gac410: well I rejected it myself
(10:14:47 AM) vrurg: Yet, I'd postpone it – simply because I know very well how hard is it to balance a couple of changes made at once.
(10:15:28 AM) ***vrurg remembers that proposal.
(10:16:27 AM) gac410: y.  I agree.   Maybe an  initial concept would be a rename capability in an onSave handler for those webnames.   So if you want your UsersWeb named "Shareholders"  it can be done from the Configure UI rather than requireing a CLI rename.
(10:16:49 AM) gac410: Right now we have all these configurable names with a warning that if you change them you break your site.
(10:17:48 AM) gac410: But,  really the priority needs to be splitting out the 3 web concepts.  So I won't go any further than that.  
(10:18:40 AM) vrurg: So, let's chunk up the elephant then.
(10:19:26 AM) gac410: The challenge is sites that want to upgrade.  New installs need to default to 3 separate webs  (Which is what would be in the Config.spec)   but existing sites need to recognize Main as the name for all 3 webs.  
(10:20:11 AM) gac410: So Configure::Load() probably needs to decide that if the HomeWeb and ConfigWeb are undefined,  set them based upon the value of UsersWeb
(10:20:12 AM) vrurg: Some of the things discussed today would be easier to implement in 3.0 with new dedicated support for callbacks – if the callbacks themselve are properly planned and planted.
(10:22:09 AM) gac410: y.   We *really* need to make 2.2 our last hurray for the 2.x stream and focus on trying to catch 3.x up with the 2.x features.
(10:22:40 AM) gac410: And tackle the really ugly questions on backwards compatiblitiy of plugins, etc.
(10:23:35 AM) vrurg: With regard to upgrade – how does upgrade package deal with specs? Would it be capable of distinguishing whether HomeWeb/ConfigWeb come from LSC or from spec defaults?
(10:24:32 AM) gac410: Once the LSC exists, the only thing copied from .spec files are keys missing from LSC.   And in most cases the default is sufficient.
(10:25:32 AM) gac410: There is a poorly known  config upgrade part of Configure::Load() which does a forEach over a list of keys, to convert older deprecated keys into a new value.
(10:26:13 AM) vrurg: But these two are missing in LSC, aren't they? So, when configure loads it fetches and merges specs, if memory serves me right.
(10:26:37 AM) gac410: Right.  It's the same issue for the new RootDir setting in 2.2.  
(10:27:04 AM) gac410: So Load needs an exception ... don't just merge, but make an intelligent setting.
(10:29:58 AM) vrurg: Ok, anyway – it's a minor technical issue. As far as I remember, readConfig can load LSC only without specs – so, additional check is possible. BTW, in the new specs model I addressed this issue by allowing loading LSC into a separate hash – on way. And another: when LSC is in specs mode it possible to find out if actual key value is set or default is used. Just boasting of myself... ;)
(10:30:21 AM) gac410: ;)
(10:34:22 AM) vrurg: Looks like it's all for today?
(10:34:30 AM) gac410: I think so.
(10:34:42 AM) gac410: I need to get going anyway
(10:35:42 AM) vrurg: Thank you!
(10:35:54 AM) gac410: Thank you too.