Monday, November 19, 2012

How to correctly escape webapp output

1, Escaping must be performed in the last possible place = when writing HTML output


Only when writing a java variable content to the output stream we know if we write the content into a javascript variable, html attribute or as a plain text (html body).

2, Every variable of a String type deserves escaping. Don't escape output only in case you NEED to print HTML.


You may never know how a possibly malicious content got into your variable. It may be persistent
XSS from DB, session attribute, request parameter.

3, Use the right escaping for the right situation. 


See OWASP's XSS_Prevention_Rules_Summary


Side note: Don't mix URL encoding with URL escaping. 

See HTML appendix B.2.1 and B.2.2


URL encoding
Though there can be UTF-8 characters in the URL, URLs should be transmitted in US-ASCII encoding.

Valid URL follows percent encoding of characters, described in Percent encoding of URI characters.

When URL is not encoded, UTF-8 characters are translated into US-ASCII OOTB by browsers. But if you want to be sure (you can't lose here) you can use new java.net.URI().toASCIIString(). In javascript there is a similar function window.encodeURI().

What MUST BE encoded is a URL parameter name and value. It's confusing but to encode the parameter use java.net.URLEncoder.encode(). Javascript has better name: window.encodeURIComponent().

Don't use URL encoding to perform HTML escaping! (Yes, I've seen that :) )


URL escaping
Let's say we already have a valid (correctly encoded) URL which we want to write into HTML.

Then there are 2 important steps:
1, Ampersand character (&), which may be a part of URL, is used to prefix html entity references. When writing a valid HTML we should escape it the same way as any other ampersand you want to write there. Use ampersand entity: &

2, To write safe HTML = to be sure that URL won't escape from HTML attributes or JavaScript variable (it is allowed to contain apostrophe character) it's good to escape the URL as described in  XSS_Prevention_Rules_Summary, not only the query string part.

When we don't have the URL properly encoded we must escape the whole URL - perform the step #2.

Sunday, November 4, 2012

Theme Template For a Specific Portlet

If you want to have your own template for Sign In portlet, you can simply:

  1. Go to your templates directory (theme/docroot/_diffs/templates/)
  2. Copy portlet.vm into portlet.58.vm (58 is a portlet id for Sign In portlet)
  3. Now, Sign In portlet will use portlet.58.vm, other portlets will use default portlet.vm
The same applies for instanceable portlets (e.g. you can have portlet.101_INSTANCE_ABCDE.vm).

You can find the nice piece of logic for loading a portlet template (both VM and FTL) inside com.liferay.taglib.util.ThemeUtil. It's simple:

  • Let's render portlet portlet with Id: 1_WAR_samplejsonportlet_INSTANCE_UgOvIB6Y3Ppb
  1. Portal first tries to find portlet.1_WAR_samplejsonportlet_INSTANCE_UgOvIB6Y3Ppb.vm
  2. Then tries portlet.1_WAR_samplejsonportlet.vm
  3. Finally use portlet.vm



Monday, October 15, 2012

Linux Dummy Mail Server for Debugging

Recently I found useful console command for creating a local mail server, echoing incoming mail messages to std out.

Enjoy the simplicity and effectiveness:
sudo python -m smtpd -n -c DebuggingServer localhost:25

// made it to my .bash_aliases file as:
alias mail_server='sudo python -m smtpd -n -c DebuggingServer localhost:25'

Friday, October 12, 2012

Liferay JRebel PermGen IntelliJ Idea

Problem:
  1. IntelliJ use JAVA_OPTS for setting -XX:MaxPermSize in Run/Debug Configurations
  2. catalina.sh overwrites JAVA_OPTS with CATALINA_OPTS
  3. Liferay+Tomcat bundle defines
    CATALINA_OPTS as "... -Xmx1024m -XX:MaxPermSize=256m"
  4. => settings in the IDE are overwritten by the bundle default configuration :(

Result:
/usr/lib/jvm/java-6-sun/bin/java -Djava.util.logging.config.file=/opt/liferay.git/bundles/tomcat-7.0.27/conf/logging.properties -javaagent:/home/topolik/.IntelliJIdea11/config/plugins/jr-ide-idea/lib/jrebel/jrebel.jar -Xdebug -Xrunjdwp:transport=dt_socket,address=127.0.0.1:52374,suspend=y,server=n -javaagent:/opt/idea-IU-117.117/plugins/Groovy/lib/agent/gragent.jar -Xms512m -Xmx1024m -XX:MaxPermSize=512m -Dfile.encoding=UTF8 -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dfile.encoding=UTF8 -Dorg.apache.catalina.loader.WebappClassLoader.ENABLE_CLEAR_REFERENCES=false -Duser.timezone=GMT -Xmx1024m -XX:MaxPermSize=256m -Djava.endorsed.dirs=/opt/liferay.git/bundles/tomcat-7.0.27/endorsed -classpath /opt/liferay.git/bundles/tomcat-7.0.27/bin/bootstrap.jar:/opt/liferay.git/bundles/tomcat-7.0.27/bin/tomcat-juli.jar -Dcatalina.base=/opt/liferay.git/bundles/tomcat-7.0.27 -Dcatalina.home=/opt/liferay.git/bundles/tomcat-7.0.27 -Djava.io.tmpdir=/opt/liferay.git/bundles/tomcat-7.0.27/temp org.apache.catalina.startup.Bootstrap start


Workaround:
Hack one of Tomcat's env. properties to include the MaxPermSize settings AFTER CATALINA_OPTS:
  1. In the IntelliJ Run/Debug configurations dialog add a new env. variable named CATALINA_TMPDIR with the following value (don't forget the quotes!):
    $CATALINA_BASE/temp" -XX:MaxPermSize=512m -Dwhatever="
Note: If you are running Windows, use %CATALINA_BASE%/temp instead of $CATALINA_BASE/temp in the variable value

Result:

/usr/lib/jvm/java-6-sun/bin/java -Djava.util.logging.config.file=/opt/liferay.git/bundles/tomcat-7.0.27/conf/logging.properties -javaagent:/home/topolik/.IntelliJIdea11/config/plugins/jr-ide-idea/lib/jrebel/jrebel.jar -Xdebug -Xrunjdwp:transport=dt_socket,address=127.0.0.1:52374,suspend=y,server=n -javaagent:/opt/idea-IU-117.117/plugins/Groovy/lib/agent/gragent.jar -Xms512m -Xmx1024m -XX:MaxPermSize=724m -Dfile.encoding=UTF8 -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dfile.encoding=UTF8 -Dorg.apache.catalina.loader.WebappClassLoader.ENABLE_CLEAR_REFERENCES=false -Duser.timezone=GMT -Xmx1024m -XX:MaxPermSize=256m -Djava.endorsed.dirs=/opt/liferay.git/bundles/tomcat-7.0.27/endorsed -classpath /opt/liferay.git/bundles/tomcat-7.0.27/bin/bootstrap.jar:/opt/liferay.git/bundles/tomcat-7.0.27/bin/tomcat-juli.jar -Dcatalina.base=/opt/liferay.git/bundles/tomcat-7.0.27 -Dcatalina.home=/opt/liferay.git/bundles/tomcat-7.0.27 -Djava.io.tmpdir=/opt/liferay.git/bundles/tomcat-7.0.27/temp -XX:MaxPermSize=512m -Dwhatever= org.apache.catalina.startup.Bootstrap start


Wednesday, September 12, 2012

Running Liferay Unit and Integration Tests


Liferay source code contains many JUnit and Integration tests. It wasn't easy to setup my environment to run the tests so here is little guide for those how want to try it too.

Basic info is at http://www.liferay.com/community/wiki/-/wiki/Main/Liferay+Testing+Infrastructure

I have Liferay sources in /opt/liferay.git/portal, let's call this directory SRC_ROOT for now.

Prerequisites:
* I have MySQL installed and run with Ubuntu defaults (at localhost, user root, empty password)
* Download run-tests.sh from https://gist.github.com/3706003
* Save it into SRC_ROOT
* Change permissions of the script file: chmod u+x run-tests.sh
* Note: I don't have any extra settings in any *.properties file

Running:
* Execute ./run-tests.sh inside SRC_ROOT
* Wait, it takes minutes to finish
* Ignore any exceptions during the tests, the important ones will be shown at the end
* At the end you should see

BUILD SUCCESSFUL
Total time: 6 minutes 42 seconds

**************************************
*           FOUND ERRORS             *
**************************************

./portal-impl/test-results/integration/TEST-com.liferay.portlet.wiki.trash.WikiNodeTrashHandlerTest.xml:    <error message="No AssetEntry exists with the key {classNameId=10149, classPK=18905}" type="com.liferay.portlet.asset.NoSuchEntryException">com.liferay.portlet.asset.NoSuchEntryException: No AssetEntry exists with the key {classNameId=10149, classPK=18905}
./portal-impl/test-results/integration/TEST-com.liferay.portlet.wiki.trash.WikiNodeTrashHandlerTest.xml:    <error message="No AssetEntry exists with the key {classNameId=10149, classPK=18910}" type="com.liferay.portlet.asset.NoSuchEntryException">com.liferay.portlet.asset.NoSuchEntryException: No AssetEntry exists with the key {classNameId=10149, classPK=18910}
./portal-impl/test-results/integration/TEST-com.liferay.portlet.wiki.trash.WikiNodeTrashHandlerTest.xml:    <error message="No AssetEntry exists with the key {classNameId=10149, classPK=18915}" type="com.liferay.portlet.asset.NoSuchEntryException">com.liferay.portlet.asset.NoSuchEntryException: No AssetEntry exists with the key {classNameId=10149, classPK=18915}
./portal-impl/test-results/integration/TEST-com.liferay.portlet.wiki.trash.WikiPageTrashHandlerTest.xml:    <error message="No WikiNode exists with the primary key 19001" type="com.liferay.portlet.wiki.NoSuchNodeException">com.liferay.portlet.wiki.NoSuchNodeException: No WikiNode exists with the primary key 19001
./portal-impl/test-results/integration/TEST-com.liferay.portlet.wiki.trash.WikiPageTrashHandlerTest.xml:    <error message="No WikiNode exists with the primary key 19006" type="com.liferay.portlet.wiki.NoSuchNodeException">com.liferay.portlet.wiki.NoSuchNodeException: No WikiNode exists with the primary key 19006
./portal-impl/test-results/integration/TEST-com.liferay.portlet.wiki.trash.WikiPageTrashHandlerTest.xml:    <error message="No WikiNode exists with the primary key 19011" type="com.liferay.portlet.wiki.NoSuchNodeException">com.liferay.portlet.wiki.NoSuchNodeException: No WikiNode exists with the primary key 19011


******************************************
* FOUND TEST FAILURES *
******************************************

./portal-impl/test-results/integration/TEST-com.liferay.portal.staging.StagingImplTest.xml:    <failure message="expected:&lt;Title[]en_US&gt; but was:&lt;Title[2]en_US&gt;" type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: expected:&lt;Title[]en_US&gt; but was:&lt;Title[2]en_US&gt;

Which means success! :)

Monday, September 10, 2012

When was a line added to a file - git bisect + grep

When do you want to know in which commit was a text added into a file, here are steps for bash + grep + git

export TEXT_TO_SEARCH="jcr.fetch.delay"
export FILE="portal-impl/src/portal.properties"

git bisect start "$FILE"
git bisect bad
git bisect good `git rev-list --parents HEAD "$FILE" | egrep "^[a-f0-9]{40}$"`

while
    ((grep "$TEXT_TO_SEARCH" "$FILE" > /dev/null && git bisect bad) || git bisect good) | grep Bisecting
do
    continue
done

git bisect visualize

git bisect reset