[tool] More tools reenginering

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[tool] More tools reenginering

Claude Brisson-2
This time, it's about tools reading external resources. I'd like to:

1) have ImportSupport and ImportTool also be available in generic tools
for absolute URLs, only the view versions being able to import a
relative URL

2) have XmlTool inherit from ImportSupport, to homogenize its behavior
with ImportTool

3) have a new generic JsonTool, in the same spirit

Example:

   #set($elements = $json.read("https://some.external.api/elements.json"))
   #foreach($element in $elements) ...

4) Put back XmlTool in the velocity-tools-generic module, using shading
for the dom4j dependency. This is motivated by the fact that although
dom4j-1.6.1.jar weights 446k, shading only needs 152k out of it. So it's
a little loss for users not using the XmlTool, and a little gain for
users using the XmlTool (see the exact figures in the appended annex).
Not mentioning the gain in simplicity, one less jar...

5) have view versions of XmlTool and JsonTool, with the extra feature of
being able to handle query POST parameters that are in text/json and
text/xml format.

Example:

   javascript part:
   obj.foo = 'bar';
   $.post({
     url: "some.template",
     data: JSON.serialize(obj),
     success: function(data) { do something },
     error: function() { console.log("error"); },
     contentType: "application/json"
   });

   template part in some.template:
   ... $json.post.foo ...

That's a typical client to server communication scheme. Of course, it
certainly needs sanitization somewhere, as every client-side submission,
but it's another story.

---------

Annex: dom4j shading experiment features (which also shows some jar
sizes evolution figures)

Here are the individual jar sizes:

(velocity-engine 1.5 383k)
---
velocity-engine 1.7 439k
dom4j-1.1 446k
velocity-tools 2.0 (generic) 189k
velocity-tools 2.0 (generic + view) 312k
---
velocity-engine 2.0 current RC 622k
velocity-tools-generic 3.0-SNAPSHOT 160k
velocity-tools-view 3.0-SNAPSHOT 99k
velocity-tools-xml 3.0-SNAPSHOT 14k
dom4j-1.6.1.jar 307k
velocity-tools-xml 3.0-SNAPSHOT with dom4j shading 166k

Here are Velocity + Tools configuration sizes:

Config engine 1.7 + generic tools 2.0 => 628k
Config engine 1.7 +    view tools 2.0 => 751k

Config engine 1.7 + generic tools 2.0 + dom4j 1.1 => 1079k
Config engine 1.7 +    view tools 2.0 + dom4j 1.1 => 1197k

Config engine 2.0 + generic tools 3.0                                  
=>  782k
Config engine 2.0 + generic tools & xml tool 3.0 + unshaded dom4j 1.6.1
=> 1103k
Config engine 2.0 + generic tools & xml tool 3.0 + shaded dom4j 1.6.1
=>  948k

Config engine 2.0 + view tools 3.0                                   =>  
881k
Config engine 2.0 + view tools & xml tool 3.0 + unshaded dom4j 1.6.1 =>
1202k
Config engine 2.0 + view tools & xml tool 3.0 + shaded dom4j 1.6.1 => 1047k

To put it in another way, with dom4j shading and XmlTool back into the
generic tools (compared with the engine 2.0 + tools 3.0 config):
  - people not using the XmlTool will loose 166k, about +21% for generic
tools users and +18% for view tools users
  - people     using the XmlTool will gain  155k, about -14% for generic
tools users and -13% for view tools users


   Claude


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[tool] More tools reenginering

Michael Osipov
> This time, it's about tools reading external resources. I'd like to:
>
> 1) have ImportSupport and ImportTool also be available in generic tools
> for absolute URLs, only the view versions being able to import a
> relative URL
>
> 2) have XmlTool inherit from ImportSupport, to homogenize its behavior
> with ImportTool
>
> 3) have a new generic JsonTool, in the same spirit
>
> Example:
>
>    #set($elements = $json.read("https://some.external.api/elements.json"))
>    #foreach($element in $elements) ...
>
> 4) Put back XmlTool in the velocity-tools-generic module, using shading
> for the dom4j dependency. This is motivated by the fact that although
> dom4j-1.6.1.jar weights 446k, shading only needs 152k out of it. So it's
> a little loss for users not using the XmlTool, and a little gain for
> users using the XmlTool (see the exact figures in the appended annex).
> Not mentioning the gain in simplicity, one less jar...
>
> 5) have view versions of XmlTool and JsonTool, with the extra feature of
> being able to handle query POST parameters that are in text/json and
> text/xml format.


While I understand you idea, I do think that introducing stuff like this
a bad idea for several reasons:

1. Processing of structured, low-level data like XML, JSON, form data
always belongs to a controller and not to the view layer. The view layer
should always receive a minimal set of high-level structures to do its task.
2. You are turning a template engine into a scripting engine which
Velocity is not.
3. By having opening the door for #read("http..") people will start to
ask for: How do I set credentials, how do I add request headers, how do
I add my company's proxy, etc? At the end, you will find yourself
implementing stuff people have already done several times.
4. and likely more.

The very same idea has been implemented on a lower level by the Plexus
Resouce Loader [1] which in turn is used in LICENSE.vm in Maven [2] and
guess what, every time someone tries to build Maven inside a company
with a proxy, he/she waste some time to figure out why the build is
hanging. One simple reason: the implementation so simple that you can't
provide a proxy host to reach external hosts. What a pain!

In that spirit, keep the tools as simple as possible and as offline as
possible. I would even go with the XML support available in Java. the
DOM4J support was added in times where Java's native XML support was lousy.

Michael

[1]
https://codehaus-plexus.github.io/plexus-resources/apidocs/org/codehaus/plexus/resource/loader/URLResourceLoader.html
[2]
https://github.com/apache/maven/blob/master/apache-maven/src/main/appended-resources/META-INF/LICENSE.vm 


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [tool] More tools reenginering

Claude Brisson-2
Hi.

On 01/12/2016 21:21, Michael Osipov wrote:
> While I understand you idea, I do think that introducing stuff like
> this a bad idea for several reasons:
>
> 1. Processing of structured, low-level data like XML, JSON, form data
> always belongs to a controller and not to the view layer. The view
> layer should always receive a minimal set of high-level structures to
> do its task.

I'm well aware of this MVC design paradigm. But the underlying purpose
is the separation of concerns, not the fact that the view should be fed
its data like a baby. View layers have grown up, and pull models have
proven their efficiency. Since they are more lightweight and flexible,
they are far more maintainable.

More specifically, let say for instance that someone wants to build a
webapp and a native mobile app on a common model layer. He would
typically publish his model in the form of an API returning JSON
structures. They cannot be called low-level, because the raw data model
has already been somehow digested in the API presentation logic. Each
API author is trying to guarantee minimal ergonomics. So if I have, say,
the following JSON:
   { '"books" : [ { "author": "Lewis Caroll", "title": "Alice in
Wonderland" } , { "author":"Saint-Exupéry", "title":"le Petit Prince" } ] }
what you're saying is that it's not high-level enough for the view? I
can't agree.

> 2. You are turning a template engine into a scripting engine which
> Velocity is not.

Since a bare scripting engine doesn't know how to do templating, I think
you intended to say that I'm empowering Velocity with scripting
abilities. Well... it's already the case. And another thing: I've seen
environments where people made a complete mess of their view layer
without using a single scripting functionality. My point here is that
although we ought to remain attentive to what the language offers,
trying to limit the features is not the right way to promote the
separation of concerns.

> 3. By having opening the door for #read("http..") people will start to
> ask for: How do I set credentials, how do I add request headers, how
> do I add my company's proxy, etc? At the end, you will find yourself
> implementing stuff people have already done several times.

If I understand correctly, you consider that the existing ImportTool is
already wrongful. The door is already open.

I already had in mind to disallow absolute URLs in safe mode (which is
on by default) for the view versions. This is already the case for the
actual XmlTool, but not for the ImportTool. My plan is to homogeneize
all that: the XmlTool generic version would be relaxed by my proposal,
while the view version of the ImportTool would be restricted to relative
URLs in safe mode.

I totally agree that fetching an external resource is dubious in the
view layer of an MVC webapp. But regarding the generic versions, where
only absolute URLs make sense, I think it's not our job to try to
prevent bad designs for some by refusing features to others.

> 4. and likely more.
>
> The very same idea has been implemented on a lower level by the Plexus
> Resouce Loader [1] which in turn is used in LICENSE.vm in Maven [2]
> and guess what, every time someone tries to build Maven inside a
> company with a proxy, he/she waste some time to figure out why the
> build is hanging. One simple reason: the implementation so simple that
> you can't provide a proxy host to reach external hosts. What a pain!

It's bad design. And you're story illustrates something: if we don't
provide people with such tools, they'll write them and it will be worse.

>
> In that spirit, keep the tools as simple as possible and as offline as
> possible.

Tools should do their job. That's the spirit. Their syntax and their use
should be kept as simple as possible, but I really don't see why they
shouldn't do complex things. I wonder how you'll react the day I will
want to integrate Velosurf as a ModelTool in a future version...

> I would even go with the XML support available in Java. the DOM4J
> support was added in times where Java's native XML support was lousy.

You may be right here, and that would be more easier than use DOM4J
shading, but I'm not sure at all I'll find the time and motivation to
rewrite the XmlTool.


   Claude



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [tool] More tools reenginering

Michael Osipov
Am 2016-12-02 um 10:01 schrieb Claude Brisson:

> Hi.
>
> On 01/12/2016 21:21, Michael Osipov wrote:
>> While I understand you idea, I do think that introducing stuff like
>> this a bad idea for several reasons:
>>
>> 1. Processing of structured, low-level data like XML, JSON, form data
>> always belongs to a controller and not to the view layer. The view
>> layer should always receive a minimal set of high-level structures to
>> do its task.
>
> I'm well aware of this MVC design paradigm. But the underlying purpose
> is the separation of concerns, not the fact that the view should be fed
> its data like a baby. View layers have grown up, and pull models have
> proven their efficiency. Since they are more lightweight and flexible,
> they are far more maintainable.
>
> More specifically, let say for instance that someone wants to build a
> webapp and a native mobile app on a common model layer. He would
> typically publish his model in the form of an API returning JSON
> structures. They cannot be called low-level, because the raw data model
> has already been somehow digested in the API presentation logic. Each
> API author is trying to guarantee minimal ergonomics. So if I have, say,
> the following JSON:
>   { '"books" : [ { "author": "Lewis Caroll", "title": "Alice in
> Wonderland" } , { "author":"Saint-Exupéry", "title":"le Petit Prince" } ] }
> what you're saying is that it's not high-level enough for the view? I
> can't agree.

I do not consider this good application design unless I see a specific
case where it truly makes sense. Having a browser/client-only app
implies using JavaScript, thus Velocity will be a little of use. E.g.,
Sonatype Nexus does that. Render skeleton by Velocity, rest is done in JS.

In this very example, this is not high-level. High-level for me is a
complete abstraction of the serialization format, Java structures only.
Pretty much like with Eclipse MOXy or Jackson, they support multiple
formats and the templater user/designer does not really care about the
format.

>> 2. You are turning a template engine into a scripting engine which
>> Velocity is not.
>
> Since a bare scripting engine doesn't know how to do templating, I think
> you intended to say that I'm empowering Velocity with scripting
> abilities. Well... it's already the case. And another thing: I've seen
> environments where people made a complete mess of their view layer
> without using a single scripting functionality. My point here is that
> although we ought to remain attentive to what the language offers,
> trying to limit the features is not the right way to promote the
> separation of concerns.

I agree here, my concern is rather not to introduce features which will
people encourage to abuse Velocity for tasks it has not been designed
and knowning that their are better suited tools for this.

>> 3. By having opening the door for #read("http..") people will start to
>> ask for: How do I set credentials, how do I add request headers, how
>> do I add my company's proxy, etc? At the end, you will find yourself
>> implementing stuff people have already done several times.
>
> If I understand correctly, you consider that the existing ImportTool is
> already wrongful. The door is already open.

Unfortunately, it is.

> I already had in mind to disallow absolute URLs in safe mode (which is
> on by default) for the view versions. This is already the case for the
> actual XmlTool, but not for the ImportTool. My plan is to homogeneize
> all that: the XmlTool generic version would be relaxed by my proposal,
> while the view version of the ImportTool would be restricted to relative
> URLs in safe mode.

Sound good.

> I totally agree that fetching an external resource is dubious in the
> view layer of an MVC webapp. But regarding the generic versions, where
> only absolute URLs make sense, I think it's not our job to try to
> prevent bad designs for some by refusing features to others.

While this is right, we still should try to promotive well-proven and
predictable approaches for the use of Velocity.

>> 4. and likely more.
>>
>> The very same idea has been implemented on a lower level by the Plexus
>> Resouce Loader [1] which in turn is used in LICENSE.vm in Maven [2]
>> and guess what, every time someone tries to build Maven inside a
>> company with a proxy, he/she waste some time to figure out why the
>> build is hanging. One simple reason: the implementation so simple that
>> you can't provide a proxy host to reach external hosts. What a pain!
>
> It's bad design. And you're story illustrates something: if we don't
> provide people with such tools, they'll write them and it will be worse.

That's absolutely true, though functionality should be still limited to
avoid frustration and unnecessary support from us.

>>
>> In that spirit, keep the tools as simple as possible and as offline as
>> possible.
>
> Tools should do their job. That's the spirit. Their syntax and their use
> should be kept as simple as possible, but I really don't see why they
> shouldn't do complex things. I wonder how you'll react the day I will
> want to integrate Velosurf as a ModelTool in a future version...

Just checked Velosurf, really sick stuff (meant positively).
Velosurf isn't probably the best name for that because it says not much
about its objective. I am quite certain that there is a usecase for that
where you have some command line tool to auto generate data from a
database, but not from a web application from my point of view. It
pretty much defeats any time of encapsulation and testing capabilities.

At the end, Velocity is a general purpose template engine, there are
tools which make sense in a web environment and which in a different
mode only.

>> I would even go with the XML support available in Java. the DOM4J
>> support was added in times where Java's native XML support was lousy.
>
> You may be right here, and that would be more easier than use DOM4J
> shading, but I'm not sure at all I'll find the time and motivation to
> rewrite the XmlTool.

You should at least track that as a JIRA issue.


Michael


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [tool] More tools reenginering

Sergiu Dumitriu
I agree with Michael here, this direction is against what "pure"
Velocity is about. The generic tools shouldn't be allowed to fetch
random resources off the internet.

I'm also against propagating the dependency on DOM4J, a library which
has been dead for ten years.

I'm OK with adding tools for working with JSON and XML, as long as they
perform safe operations on local data, and use modern libraries for the
generated data structures (org.json:json and the org.w3c.dom package,
although it's not very user friendly).

On 12/04/2016 07:57 AM, Michael Osipov wrote:

> Am 2016-12-02 um 10:01 schrieb Claude Brisson:
>> Hi.
>>
>> On 01/12/2016 21:21, Michael Osipov wrote:
>>> While I understand you idea, I do think that introducing stuff like
>>> this a bad idea for several reasons:
>>>
>>> 1. Processing of structured, low-level data like XML, JSON, form data
>>> always belongs to a controller and not to the view layer. The view
>>> layer should always receive a minimal set of high-level structures to
>>> do its task.
>>
>> I'm well aware of this MVC design paradigm. But the underlying purpose
>> is the separation of concerns, not the fact that the view should be fed
>> its data like a baby. View layers have grown up, and pull models have
>> proven their efficiency. Since they are more lightweight and flexible,
>> they are far more maintainable.
>>
>> More specifically, let say for instance that someone wants to build a
>> webapp and a native mobile app on a common model layer. He would
>> typically publish his model in the form of an API returning JSON
>> structures. They cannot be called low-level, because the raw data model
>> has already been somehow digested in the API presentation logic. Each
>> API author is trying to guarantee minimal ergonomics. So if I have, say,
>> the following JSON:
>>   { '"books" : [ { "author": "Lewis Caroll", "title": "Alice in
>> Wonderland" } , { "author":"Saint-Exupéry", "title":"le Petit Prince"
>> } ] }
>> what you're saying is that it's not high-level enough for the view? I
>> can't agree.
>
> I do not consider this good application design unless I see a specific
> case where it truly makes sense. Having a browser/client-only app
> implies using JavaScript, thus Velocity will be a little of use. E.g.,
> Sonatype Nexus does that. Render skeleton by Velocity, rest is done in JS.
>
> In this very example, this is not high-level. High-level for me is a
> complete abstraction of the serialization format, Java structures only.
> Pretty much like with Eclipse MOXy or Jackson, they support multiple
> formats and the templater user/designer does not really care about the
> format.
>
>>> 2. You are turning a template engine into a scripting engine which
>>> Velocity is not.
>>
>> Since a bare scripting engine doesn't know how to do templating, I think
>> you intended to say that I'm empowering Velocity with scripting
>> abilities. Well... it's already the case. And another thing: I've seen
>> environments where people made a complete mess of their view layer
>> without using a single scripting functionality. My point here is that
>> although we ought to remain attentive to what the language offers,
>> trying to limit the features is not the right way to promote the
>> separation of concerns.
>
> I agree here, my concern is rather not to introduce features which will
> people encourage to abuse Velocity for tasks it has not been designed
> and knowning that their are better suited tools for this.
>
>>> 3. By having opening the door for #read("http..") people will start to
>>> ask for: How do I set credentials, how do I add request headers, how
>>> do I add my company's proxy, etc? At the end, you will find yourself
>>> implementing stuff people have already done several times.
>>
>> If I understand correctly, you consider that the existing ImportTool is
>> already wrongful. The door is already open.
>
> Unfortunately, it is.
>
>> I already had in mind to disallow absolute URLs in safe mode (which is
>> on by default) for the view versions. This is already the case for the
>> actual XmlTool, but not for the ImportTool. My plan is to homogeneize
>> all that: the XmlTool generic version would be relaxed by my proposal,
>> while the view version of the ImportTool would be restricted to relative
>> URLs in safe mode.
>
> Sound good.
>
>> I totally agree that fetching an external resource is dubious in the
>> view layer of an MVC webapp. But regarding the generic versions, where
>> only absolute URLs make sense, I think it's not our job to try to
>> prevent bad designs for some by refusing features to others.
>
> While this is right, we still should try to promotive well-proven and
> predictable approaches for the use of Velocity.
>
>>> 4. and likely more.
>>>
>>> The very same idea has been implemented on a lower level by the Plexus
>>> Resouce Loader [1] which in turn is used in LICENSE.vm in Maven [2]
>>> and guess what, every time someone tries to build Maven inside a
>>> company with a proxy, he/she waste some time to figure out why the
>>> build is hanging. One simple reason: the implementation so simple that
>>> you can't provide a proxy host to reach external hosts. What a pain!
>>
>> It's bad design. And you're story illustrates something: if we don't
>> provide people with such tools, they'll write them and it will be worse.
>
> That's absolutely true, though functionality should be still limited to
> avoid frustration and unnecessary support from us.
>
>>>
>>> In that spirit, keep the tools as simple as possible and as offline as
>>> possible.
>>
>> Tools should do their job. That's the spirit. Their syntax and their use
>> should be kept as simple as possible, but I really don't see why they
>> shouldn't do complex things. I wonder how you'll react the day I will
>> want to integrate Velosurf as a ModelTool in a future version...
>
> Just checked Velosurf, really sick stuff (meant positively).
> Velosurf isn't probably the best name for that because it says not much
> about its objective. I am quite certain that there is a usecase for that
> where you have some command line tool to auto generate data from a
> database, but not from a web application from my point of view. It
> pretty much defeats any time of encapsulation and testing capabilities.
>
> At the end, Velocity is a general purpose template engine, there are
> tools which make sense in a web environment and which in a different
> mode only.
>
>>> I would even go with the XML support available in Java. the DOM4J
>>> support was added in times where Java's native XML support was lousy.
>>
>> You may be right here, and that would be more easier than use DOM4J
>> shading, but I'm not sure at all I'll find the time and motivation to
>> rewrite the XmlTool.
>
> You should at least track that as a JIRA issue.
>
>
> Michael
>

--
Sergiu Dumitriu
http://purl.org/net/sergiu

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [tool] More tools reenginering

Nathan Bubna
On Mon, Dec 5, 2016 at 9:49 AM, Sergiu Dumitriu <[hidden email]>
wrote:

> I agree with Michael here, this direction is against what "pure"
> Velocity is about. The generic tools shouldn't be allowed to fetch
> random resources off the internet.
>

I'm with Claude here. Making powerful tools available is a good thing. If
we were putting this in Engine, i'd protest. Even in VelocityView, clearly
for webapps, it would seem an inappropriate default. But in GenericTools, i
think it fits fine. They are to be generic, useful. Velocity is not just
about MVC, not just used in applications.


> I'm also against propagating the dependency on DOM4J, a library which
> has been dead for ten years.
>

Well, those who do the work should decide, but if i were doing the work,
yeah, dom4j does smell a little. ;) Though arguably we can hardly be too
critical of old, little-update libraries.


> I'm OK with adding tools for working with JSON and XML, as long as they
> perform safe operations on local data, and use modern libraries for the
> generated data structures (org.json:json and the org.w3c.dom package,
> although it's not very user friendly).
>
> On 12/04/2016 07:57 AM, Michael Osipov wrote:
> > Am 2016-12-02 um 10:01 schrieb Claude Brisson:
> >> Hi.
> >>
> >> On 01/12/2016 21:21, Michael Osipov wrote:
> >>> While I understand you idea, I do think that introducing stuff like
> >>> this a bad idea for several reasons:
> >>>
> >>> 1. Processing of structured, low-level data like XML, JSON, form data
> >>> always belongs to a controller and not to the view layer. The view
> >>> layer should always receive a minimal set of high-level structures to
> >>> do its task.
> >>
> >> I'm well aware of this MVC design paradigm. But the underlying purpose
> >> is the separation of concerns, not the fact that the view should be fed
> >> its data like a baby. View layers have grown up, and pull models have
> >> proven their efficiency. Since they are more lightweight and flexible,
> >> they are far more maintainable.
> >>
> >> More specifically, let say for instance that someone wants to build a
> >> webapp and a native mobile app on a common model layer. He would
> >> typically publish his model in the form of an API returning JSON
> >> structures. They cannot be called low-level, because the raw data model
> >> has already been somehow digested in the API presentation logic. Each
> >> API author is trying to guarantee minimal ergonomics. So if I have, say,
> >> the following JSON:
> >>   { '"books" : [ { "author": "Lewis Caroll", "title": "Alice in
> >> Wonderland" } , { "author":"Saint-Exupéry", "title":"le Petit Prince"
> >> } ] }
> >> what you're saying is that it's not high-level enough for the view? I
> >> can't agree.
> >
> > I do not consider this good application design unless I see a specific
> > case where it truly makes sense. Having a browser/client-only app
> > implies using JavaScript, thus Velocity will be a little of use. E.g.,
> > Sonatype Nexus does that. Render skeleton by Velocity, rest is done in
> JS.
> >
> > In this very example, this is not high-level. High-level for me is a
> > complete abstraction of the serialization format, Java structures only.
> > Pretty much like with Eclipse MOXy or Jackson, they support multiple
> > formats and the templater user/designer does not really care about the
> > format.
> >
> >>> 2. You are turning a template engine into a scripting engine which
> >>> Velocity is not.
> >>
> >> Since a bare scripting engine doesn't know how to do templating, I think
> >> you intended to say that I'm empowering Velocity with scripting
> >> abilities. Well... it's already the case. And another thing: I've seen
> >> environments where people made a complete mess of their view layer
> >> without using a single scripting functionality. My point here is that
> >> although we ought to remain attentive to what the language offers,
> >> trying to limit the features is not the right way to promote the
> >> separation of concerns.
> >
> > I agree here, my concern is rather not to introduce features which will
> > people encourage to abuse Velocity for tasks it has not been designed
> > and knowning that their are better suited tools for this.
> >
> >>> 3. By having opening the door for #read("http..") people will start to
> >>> ask for: How do I set credentials, how do I add request headers, how
> >>> do I add my company's proxy, etc? At the end, you will find yourself
> >>> implementing stuff people have already done several times.
> >>
> >> If I understand correctly, you consider that the existing ImportTool is
> >> already wrongful. The door is already open.
> >
> > Unfortunately, it is.
> >
> >> I already had in mind to disallow absolute URLs in safe mode (which is
> >> on by default) for the view versions. This is already the case for the
> >> actual XmlTool, but not for the ImportTool. My plan is to homogeneize
> >> all that: the XmlTool generic version would be relaxed by my proposal,
> >> while the view version of the ImportTool would be restricted to relative
> >> URLs in safe mode.
> >
> > Sound good.
> >
> >> I totally agree that fetching an external resource is dubious in the
> >> view layer of an MVC webapp. But regarding the generic versions, where
> >> only absolute URLs make sense, I think it's not our job to try to
> >> prevent bad designs for some by refusing features to others.
> >
> > While this is right, we still should try to promotive well-proven and
> > predictable approaches for the use of Velocity.
> >
> >>> 4. and likely more.
> >>>
> >>> The very same idea has been implemented on a lower level by the Plexus
> >>> Resouce Loader [1] which in turn is used in LICENSE.vm in Maven [2]
> >>> and guess what, every time someone tries to build Maven inside a
> >>> company with a proxy, he/she waste some time to figure out why the
> >>> build is hanging. One simple reason: the implementation so simple that
> >>> you can't provide a proxy host to reach external hosts. What a pain!
> >>
> >> It's bad design. And you're story illustrates something: if we don't
> >> provide people with such tools, they'll write them and it will be worse.
> >
> > That's absolutely true, though functionality should be still limited to
> > avoid frustration and unnecessary support from us.
> >
> >>>
> >>> In that spirit, keep the tools as simple as possible and as offline as
> >>> possible.
> >>
> >> Tools should do their job. That's the spirit. Their syntax and their use
> >> should be kept as simple as possible, but I really don't see why they
> >> shouldn't do complex things. I wonder how you'll react the day I will
> >> want to integrate Velosurf as a ModelTool in a future version...
> >
> > Just checked Velosurf, really sick stuff (meant positively).
> > Velosurf isn't probably the best name for that because it says not much
> > about its objective. I am quite certain that there is a usecase for that
> > where you have some command line tool to auto generate data from a
> > database, but not from a web application from my point of view. It
> > pretty much defeats any time of encapsulation and testing capabilities.
> >
> > At the end, Velocity is a general purpose template engine, there are
> > tools which make sense in a web environment and which in a different
> > mode only.
> >
> >>> I would even go with the XML support available in Java. the DOM4J
> >>> support was added in times where Java's native XML support was lousy.
> >>
> >> You may be right here, and that would be more easier than use DOM4J
> >> shading, but I'm not sure at all I'll find the time and motivation to
> >> rewrite the XmlTool.
> >
> > You should at least track that as a JIRA issue.
> >
> >
> > Michael
> >
>
> --
> Sergiu Dumitriu
> http://purl.org/net/sergiu
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [tool] More tools reenginering

Claude Brisson-2
On 05/12/2016 19:51, Nathan Bubna wrote:

> On Mon, Dec 5, 2016 at 9:49 AM, Sergiu Dumitriu <[hidden email]>
> wrote:
>
>> I agree with Michael here, this direction is against what "pure"
>> Velocity is about. The generic tools shouldn't be allowed to fetch
>> random resources off the internet.
>>
> I'm with Claude here. Making powerful tools available is a good thing. If
> we were putting this in Engine, i'd protest. Even in VelocityView, clearly
> for webapps, it would seem an inappropriate default. But in GenericTools, i
> think it fits fine. They are to be generic, useful. Velocity is not just
> about MVC, not just used in applications.

Exactly. Forbidding absolute URLs in view tools safe mode seems all we
need to do to me.

>
>> I'm also against propagating the dependency on DOM4J, a library which
>> has been dead for ten years.
>>
> Well, those who do the work should decide, but if i were doing the work,
> yeah, dom4j does smell a little. ;) Though arguably we can hardly be too
> critical of old, little-update libraries.

Maybe I'll find the motivation to try to migrate the XmlTool. Getting
rid of a sub-module would be nice.

>
>> I'm OK with adding tools for working with JSON and XML, as long as they
>> perform safe operations on local data, and use modern libraries for the
>> generated data structures (org.json:json and the org.w3c.dom package,
>> although it's not very user friendly).
>>
>> On 12/04/2016 07:57 AM, Michael Osipov wrote:
>>> Am 2016-12-02 um 10:01 schrieb Claude Brisson:
>>>> Hi.
>>>>
>>>> On 01/12/2016 21:21, Michael Osipov wrote:
>>>>> While I understand you idea, I do think that introducing stuff like
>>>>> this a bad idea for several reasons:
>>>>>
>>>>> 1. Processing of structured, low-level data like XML, JSON, form data
>>>>> always belongs to a controller and not to the view layer. The view
>>>>> layer should always receive a minimal set of high-level structures to
>>>>> do its task.
>>>> I'm well aware of this MVC design paradigm. But the underlying purpose
>>>> is the separation of concerns, not the fact that the view should be fed
>>>> its data like a baby. View layers have grown up, and pull models have
>>>> proven their efficiency. Since they are more lightweight and flexible,
>>>> they are far more maintainable.
>>>>
>>>> More specifically, let say for instance that someone wants to build a
>>>> webapp and a native mobile app on a common model layer. He would
>>>> typically publish his model in the form of an API returning JSON
>>>> structures. They cannot be called low-level, because the raw data model
>>>> has already been somehow digested in the API presentation logic. Each
>>>> API author is trying to guarantee minimal ergonomics. So if I have, say,
>>>> the following JSON:
>>>>    { '"books" : [ { "author": "Lewis Caroll", "title": "Alice in
>>>> Wonderland" } , { "author":"Saint-Exupéry", "title":"le Petit Prince"
>>>> } ] }
>>>> what you're saying is that it's not high-level enough for the view? I
>>>> can't agree.
>>> I do not consider this good application design unless I see a specific
>>> case where it truly makes sense. Having a browser/client-only app
>>> implies using JavaScript, thus Velocity will be a little of use. E.g.,
>>> Sonatype Nexus does that. Render skeleton by Velocity, rest is done in
>> JS.
>>> In this very example, this is not high-level. High-level for me is a
>>> complete abstraction of the serialization format, Java structures only.
>>> Pretty much like with Eclipse MOXy or Jackson, they support multiple
>>> formats and the templater user/designer does not really care about the
>>> format.

While we may certainly agree on what's a definitely wrong design, I
think there are several good designs and architectural styles, as there
are many use cases. Formatting json/xml data is widely in the scope of a
templating engine, whether client-side or server-side.

Yes, you can choose to use a third party library to provide an
abstraction of the serialization format, and in that case you won't need
any XmlTool or JsonTool, but is it pertinent?
- it's mainly a purely theoretical advantage: unless you are coding a
very generic application whose data sources are not known in advance,
you already know which data sources you use and in which format you get
its data, and this format is very unlikely to change in the future for a
given data source.
  - it does ignore format specificities (xml is a richer format than
json, the formats are not always interchangeable).
  - it does add an extra layer, which comes with all its implied
problematics (performances, dependencies, bugs, code generation, added
complexity, ...) whereas a minimalistic and format-specific publication
layer, because of its lightweight nature, doesn't impose all that.

>>>
>>>>> 2. You are turning a template engine into a scripting engine which
>>>>> Velocity is not.
>>>> Since a bare scripting engine doesn't know how to do templating, I think
>>>> you intended to say that I'm empowering Velocity with scripting
>>>> abilities. Well... it's already the case. And another thing: I've seen
>>>> environments where people made a complete mess of their view layer
>>>> without using a single scripting functionality. My point here is that
>>>> although we ought to remain attentive to what the language offers,
>>>> trying to limit the features is not the right way to promote the
>>>> separation of concerns.
>>> I agree here, my concern is rather not to introduce features which will
>>> people encourage to abuse Velocity for tasks it has not been designed
>>> and knowning that their are better suited tools for this.
>>>
>>>>> 3. By having opening the door for #read("http..") people will start to
>>>>> ask for: How do I set credentials, how do I add request headers, how
>>>>> do I add my company's proxy, etc? At the end, you will find yourself
>>>>> implementing stuff people have already done several times.
>>>> If I understand correctly, you consider that the existing ImportTool is
>>>> already wrongful. The door is already open.
>>> Unfortunately, it is.
>>>
>>>> I already had in mind to disallow absolute URLs in safe mode (which is
>>>> on by default) for the view versions. This is already the case for the
>>>> actual XmlTool, but not for the ImportTool. My plan is to homogeneize
>>>> all that: the XmlTool generic version would be relaxed by my proposal,
>>>> while the view version of the ImportTool would be restricted to relative
>>>> URLs in safe mode.
>>> Sound good.
>>>
>>>> I totally agree that fetching an external resource is dubious in the
>>>> view layer of an MVC webapp. But regarding the generic versions, where
>>>> only absolute URLs make sense, I think it's not our job to try to
>>>> prevent bad designs for some by refusing features to others.
>>> While this is right, we still should try to promotive well-proven and
>>> predictable approaches for the use of Velocity.
>>>
>>>>> 4. and likely more.
>>>>>
>>>>> The very same idea has been implemented on a lower level by the Plexus
>>>>> Resouce Loader [1] which in turn is used in LICENSE.vm in Maven [2]
>>>>> and guess what, every time someone tries to build Maven inside a
>>>>> company with a proxy, he/she waste some time to figure out why the
>>>>> build is hanging. One simple reason: the implementation so simple that
>>>>> you can't provide a proxy host to reach external hosts. What a pain!
>>>> It's bad design. And you're story illustrates something: if we don't
>>>> provide people with such tools, they'll write them and it will be worse.
>>> That's absolutely true, though functionality should be still limited to
>>> avoid frustration and unnecessary support from us.
>>>
>>>>> In that spirit, keep the tools as simple as possible and as offline as
>>>>> possible.
>>>> Tools should do their job. That's the spirit. Their syntax and their use
>>>> should be kept as simple as possible, but I really don't see why they
>>>> shouldn't do complex things. I wonder how you'll react the day I will
>>>> want to integrate Velosurf as a ModelTool in a future version...
>>> Just checked Velosurf, really sick stuff (meant positively).
>>> Velosurf isn't probably the best name for that because it says not much
>>> about its objective. I am quite certain that there is a usecase for that
>>> where you have some command line tool to auto generate data from a
>>> database, but not from a web application from my point of view. It
>>> pretty much defeats any time of encapsulation and testing capabilities.

Oh but there *is* encapsulation, since you can choose to use a specific
class as an instance at any time (or overwrite a property with a
complete SQL statement) and the VTL won't see the difference. Many
developers get the feeling that since Velosurf is a lightweight library,
it's not calibrated for big projects. But on the contrary I used it with
success in several ambitious web applications, and will continue to do
so, since I find it an heresy to let some complex library do code
generation, memory persistence or SQL statements generation for me.


   Claude


>>> At the end, Velocity is a general purpose template engine, there are
>>> tools which make sense in a web environment and which in a different
>>> mode only.
>>>
>>>>> I would even go with the XML support available in Java. the DOM4J
>>>>> support was added in times where Java's native XML support was lousy.
>>>> You may be right here, and that would be more easier than use DOM4J
>>>> shading, but I'm not sure at all I'll find the time and motivation to
>>>> rewrite the XmlTool.
>>> You should at least track that as a JIRA issue.
>>>
>>>
>>> Michael
>>>
>> --
>> Sergiu Dumitriu
>> http://purl.org/net/sergiu
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]