Wednesday, October 1, 2014

Recover Deleted Fles From Canon SD Card (FAT32) on OpenSUSE

It happens. You delete files on your SD Card with family films to realize you don't have the backup @#$@(#@*&$!@(#$!!!!

Luckily the SD Card use old FAT32 => when I won't write any new data on the card I should be able to simplly un-delete the files.

Here are my 8 steps to the rescue:
=========================

1. Put SD card into the card reader

2. Unmount the card (find it using mount command)
unmount /dev/sde1
3. Get the card raw data (has anyone tried to create a boot-able SD card?)
sudo dd if=/dev/sde of=sde.raw
4. Display the image structure:
fdisk -l sde.raw

Disk sde.raw: 31.9 GB, 31914983424 bytes, 62333952 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00000000

  Device Boot      Start         End      Blocks   Id  System
sde.raw1            8192    62333951    31162880    c  W95 FAT32 (LBA)
5. Install SleuthKit (I had it installed)

6. Browse the image using fls tool (notice I start from offset 8192 with 512 bytes/sector) to find the deleted files inodes:
fls -b 512 -o 8192 -f fat sde.raw

r/r 3: CANON       (Volume Label Entry)
d/d 4: PRIVATE
d/d 5: DCIM
v/v 996950019: $MBR
v/v 996950020: $FAT1
v/v 996950021: $FAT2
d/d 996950022: $OrphanFiles

fls -b 512 -o 8192 -f fat sde.raw 4
d/d 1029: MY_MUSIC
d/d 1030: MY_PICT
d/d 1031: CVINFO
d/d 1032: AVCHD

fls -b 512 -o 8192 -f fat sde.raw 1032
d/d 5125: BDMV
d/d 5126: CANON

fls -b 512 -o 8192 -f fat sde.raw 5125
d/d 6149: BACKUP
d/d 6150: PLAYLIST
d/d 6151: CLIPINF
d/d 6152: STREAM
r/r 6153: INDEX.BDM
r/r 6154: MOVIEOBJ.BDM

fls -b 512 -o 8192 -f fat sde.raw 6152
r/r * 10245: _0000.MTS
r/r * 10246: _0001.MTS
r/r * 10247: _0002.MTS
r/r * 10248: _0003.MTS
r/r * 10249: _0004.MTS

...
Bingo!

7. Extract the files using icat tool
icat -b 512 -o 8192 -f fat -r sde.raw 10245 > _0000.MTS
icat -b 512 -o 8192 -f fat -r sde.raw 10246 > _0001.MTS
icat -b 512 -o 8192 -f fat -r sde.raw 10247 > _0002.MTS
icat -b 512 -o 8192 -f fat -r sde.raw 10248 > _0003.MTS
icat -b 512 -o 8192 -f fat -r sde.raw 10249 > _0004.MTS
...
8. Voilà! Marriage is rescued again! ;)

Friday, September 12, 2014

CoolPeople critical vulnerabilities fixed (affecting 100k+ users)

Finally I found some time to make a blog post :)

CoolPeople, world-wide Czech-based sourcing company, fixed recently many critical flaws in their apps.

Freelancers registers and work through them for customers, CoolPeople takes care of the business and payment part.

I won't list all flaws here, only the categories:
  • Arbitrary (Remote) Code Execution
  • Unrestricted File Upload
  • OS Command Injection
  • Wrong Credentials Management
  • XSS
  • CSRF
  • Missing Authorization
  • Other less important issues (https config, cookie flags, etc.)
Beside others, all invoices, bank account details, signatures, contacts, CVs and all other uploaded files into system were freely accessible by anyone using URL manipulation.

----

Their apps run on AstraSystems framework with the same issues affecting all their customers and their private data:
  • Customer accounts (CRM)
  • Customer apps configuration (incl. passwords) and DB dumps
  • Internal business documents
  • etc.

Some screenshots for fun:

Plain-text password in DB
PHP shell
Stored XSS

-----
Timeline:
July 3, 2014 ... August 14, 2014 – Working together to fix the flaws
August 15, 2014 – The most critical flaws are fixed, the rest will come with site redesign (I hope)

Thursday, September 4, 2014

How Apache Directory Listings Can Compromise Your Server

Do you think allowed Directory Listings is a minor security issue or no issue at all?

It's not.

Why?

See:


(DB backup with site passwords and others)

Thursday, August 28, 2014

Liferay ERROR SERVLET_CONTEXT_/html/themes/classic/templates/portlet.vm does not exist

Problem:

10:03:16,205 ERROR [RuntimePageImpl-6][ThemeUtil:426] _SERVLET_CONTEXT_/html/themes/classic/templates/portlet.vm does not exist
10:03:16,212 ERROR [RuntimePageImpl-5][ThemeUtil:426] _SERVLET_CONTEXT_/html/themes/classic/templates/portlet.vm does not exist
10:03:16,248 ERROR [RuntimePageImpl-4][ThemeUtil:426] _SERVLET_CONTEXT_/html/themes/classic/templates/portlet.vm does not exist
10:03:16,249 ERROR [ajp-bio-8009-exec-8][ThemeUtil:426] _SERVLET_CONTEXT_/html/themes/classic/templates/portal_normal.vm does not exist


Solution

  1. Build and Deploy themes:

    liferay-portal-src/portal-web $ ant clean build-themes deploy
     
    Buildfile: /opt/liferay.git/portal.git/portal-web/build.xml
    
    build-themes:
    
    build-alloy:
    
    build-alloy-bootstrap:
    
    build-alloy-font-awesome:
         [java] Loading file:/opt/liferay.git/portal.git/portal-impl/classes/system.properties
         [java] Loading file:/opt/liferay.git/portal.git/portal-impl/classes/system-ext.properties
         [java] Loading file:/opt/liferay.git/portal.git/portal-impl/classes/system.properties
         [java] Loading file:/opt/liferay.git/portal.git/portal-impl/classes/system-ext.properties
         [java] Loading file:/opt/liferay.git/portal.git/portal-impl/classes/system.properties
         [java] Loading file:/opt/liferay.git/portal.git/portal-impl/classes/system-ext.properties
         [java] Loading file:/opt/liferay.git/portal.git/portal-impl/classes/system.properties
         [java] Loading file:/opt/liferay.git/portal.git/portal-impl/classes/system-ext.properties
         [copy] Copying 671 files to /opt/liferay.git/portal.git/portal-web/docroot/html/themes/classic
         [copy] Copied 56 empty directories to 6 empty directories under /opt/liferay.git/portal.git/portal-web/docroot/html/themes/classic
         [copy] Copying 13 files to /opt/liferay.git/portal.git/portal-web/docroot/html/themes/classic
         [copy] Copying 21 files to /opt/liferay.git/portal.git/portal-web/docroot/html/themes/classic
         [java] Loading file:/opt/liferay.git/portal.git/portal-impl/classes/system.properties
         [java] Loading file:/opt/liferay.git/portal.git/portal-impl/classes/system-ext.properties
         [copy] Copying 671 files to /opt/liferay.git/portal.git/portal-web/docroot/html/themes/control_panel
         [copy] Copied 56 empty directories to 6 empty directories under /opt/liferay.git/portal.git/portal-web/docroot/html/themes/control_panel
         [copy] Copying 13 files to /opt/liferay.git/portal.git/portal-web/docroot/html/themes/control_panel
         [copy] Copying 45 files to /opt/liferay.git/portal.git/portal-web/docroot/html/themes/control_panel
         [copy] Copying 1 file to /opt/liferay.git/portal.git/portal-web/docroot
     
    deploy-fast:
         [copy] Copying 1272 files to /opt/liferay.git/bundles/tomcat-7.0.42/webapps/ROOT
    
    BUILD SUCCESSFUL
    Total time: 3 seconds
    
    
  2. Restart portal

Wednesday, July 2, 2014

Notes on Publishing Equinox OSGi services as JAX-WS services using Apache CXF with WS-Security* support

... This are my notes in case I forget during the process ...

Goal: Publish Liferay services as JAX-WS API
=================================

Problem: JAX-WS is supported by each app server differently.

Imagine having:
* JAX-WS 2.0, 2.1 and 2.2
* GlassFish, Tomcat + TCat, JBoss, IBM WAS, Oracle WLS
* Several version of each app server/container are certified / supported (see https://www.liferay.com/services/support/support-matrix)

To expose Liferay services we would need to tune each app server, version and write specific deployment instructions.

Solution: Isolate JAX-WS implementation using ClassLoaders.

Here comes OSGi which can load whatever JAR files we want and separate the implementation from app server class loaders.

------------------------------------------------------------

Decisions:
* use OSGi

------------------------------------------------------------

Problem: Find the best JAX-WS framework for OSGi

Not all JAX-WS frameworks can work correctly in OSGi and can be loaded by OSGi correctly.

+ we need WS-Security* enabled JAX-WS framework

Solution: Small analysis, CXF wins

Most common JAX-WS frameworks are (alphabetically):
* Axis2
* CXF
* Metro (reference implementation)

Prepared for OSGi are:
1, CXF
2, Axis2

Both support WS-Security into some degree, CXF seems to be better, has full JUnit security tests.

Google popularity:
axis2 problem: 445 000 hits
* cxf problem:  594 000 hits
jax-ws problem: 4 200 000 hits :D

Trends:

------------------------------------------------------------

Decisions:
* use OSGi
* use CXF

------------------------------------------------------------

Problem: CXF doesn't work in other OSGi implementations

CXF 2.x works correctly with Apache Karaf + Apache Aries for Blueprint.

CXF 2.x declares many extension points using Blueprint XML namespaces, if Apache Aries is not available, CXF fails to load.

See: https://issues.apache.org/jira/browse/CXF-5703

Solution: Use CXF 3.0 Beta

It's rather a workaround, CXF still tries to register custom namespaces using Aries.

If Aries is not found then continue, but CXF extensions are not available from Blueprint XML:

Jul 02, 2014 2:48:44 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
WARNING: Aries Blueprint packages not available. So namespaces will not be registered
java.lang.NoClassDefFoundError: org/apache/aries/blueprint/NamespaceHandler
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.defineClass(DefaultClassLoader.java:188)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.defineClassHoldingLock(ClasspathManager.java:638)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.defineClass(ClasspathManager.java:613)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findClassImpl(ClasspathManager.java:574)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClassImpl(ClasspathManager.java:492)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass(ClasspathManager.java:465)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.findLocalClass(DefaultClassLoader.java:216)
at org.eclipse.osgi.internal.loader.BundleLoader.findLocalClass(BundleLoader.java:395)
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:464)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:421)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:412)


=>  CXF must be configured programatically.

------------------------------------------------------------

Decisions:
* use OSGi
* use CXF
* use CXF version >= 3.0
* configure CXF programatically

------------------------------------------------------------

Problem: Some JAR files are not OSGi bundles, installation of CXF into container is complicated


Worth noting - works only on my environment :D

------------------------------------------------------------

Decisions:
* use OSGi
* use CXF
* use CXF version >= 3.0
* configure CXF programatically
* use install scripts

------------------------------------------------------------

Problem: Unresolved optional dependencies, that are not optional :)

CXF with JAX-WS and WS-Security = many bundles.

Because WS-Security is optional, it's not loaded when it's missing / installed later.

To resolve all bundles it takes non-trivial effort that ends with unresolved optional Import package references.

Solution: Trial-error bundle loading to satisfy all optional import packages

Recommendation: 
1, use "diag" command from gogo console!
2, enable debug in Equinox to see what has been imported and what keeps unresolved
osgi.debug=/opt/liferay.git/bundles/portal-ext.properties
The file content:
org.eclipse.osgi/resolver/debug=true
org.eclipse.osgi/resolver/wiring=true
org.eclipse.osgi/resolver/imports=true
org.eclipse.osgi/resolver/requires=true
org.eclipse.osgi/resolver/generics=true
org.eclipse.osgi/resolver/uses=true
org.eclipse.osgi/resolver/cycles=true
Result: (for now) gogo-console script

Should be: a one-time install package like Apache Karaf features

------------------------------------------------------------

Decisions:
* use OSGi
* use CXF
* use CXF version >= 3.0
* configure CXF programatically
* use install scripts

------------------------------------------------------------

Problem: To declare security used by JAX-WS we need WS-Policy but we don't have WSDL and  existing OSGi services doesn't have JAX-WS @Policy annotations.

Solution: Use External Policy Attachments for WS-Policy

Let's configure CXF to map external policies onto JAX-WS services.

------------------------------------------------------------

Decisions:
* use OSGi
* use CXF
* use CXF version >= 3.0
* configure CXF programatically
* use install scripts
* use WS-Policy external attachments

------------------------------------------------------------

Problem: CXF doesn't support WS-Policy external attachments to be configured programatically

CXF's ExternalAttachmentProvider class doesn't have public constructor and is created only as Spring bean (called by Apache Aries).

Solution: Create a OSGi fragment and fix the provider so that it can be used outside the package.

------------------------------------------------------------

Decisions:
* use OSGi
* use CXF
* use CXF version >= 3.0
* configure CXF programatically
* use install scripts
* use WS-Policy external attachments
* use a OSGi fragment to fix ExternalAttachmentsProvider

------------------------------------------------------------

Prototype: How dynamically apply external policy XMLs to existing services?

1, Configure external policy file using patched ExternalAttachmentsProvider
2, Enrich default external policy to be able to dynamically match services

=>  Create CXF extension
* CXF extension description META-INF/cxf/bus-extensions.txt
* + DomainExpression builder prototype that is able to apply policies on the service / service parts

------------------------------------------------------------

Decisions:
* use OSGi
* use CXF
* use CXF version >= 3.0
* configure CXF programatically
* use install scripts
* use WS-Policy external attachments
* use a OSGi fragment to fix ExternalAttachmentsProvider
* use a custom CXF extension to dynamically map services to external policy file

------------------------------------------------------------

Problem: OSGi services are not complatible with JAX-WS

Services  don't have JAX-WS @WebService annotation which is required by JAX-WS

Solution: Use Javassist to generate new interface with @WebService annotation and dynamic proxy to apply the annotated interfaces on the existing service

------------------------------------------------------------

Decisions:
* use OSGi
* use CXF
* use CXF version >= 3.0
* configure CXF programatically
* use install scripts
* use WS-Policy external attachments
* use a OSGi fragment to fix ExternalAttachmentsProvider
* use a custom CXF extension to dynamically map services to external policy file
* use Javassist and dynamic proxies to add @WebService annotation

------------------------------------------------------------

Problem: OSGi services are not JAXB friendly.

Types that services use may not be easily marshallable, but OSGi services doesn't support JAXB annotation.

JAXB is not able to generate XSD into WSDL to correctly describe types and interfaces used in the methods.


Solution: Generate type wrappers so that JAXB is able to marshall interfaces and unknown types.

The type wrappers are used instead of original types. When the type cannot be wrapped I ignored the type => the whole web method.


------------------------------------------------------------

Decisions:
* use OSGi
* use CXF
* use CXF version >= 3.0
* configure CXF programatically
* use install scripts
* use WS-Policy external attachments
* use a OSGi fragment to fix ExternalAttachmentsProvider
* use a custom CXF extension to dynamically map services to external policy file
* use Javassist and dynamic proxies to add @WebService annotation
* use Javassist to generate type wrappers for unmarshallable types

------------------------------------------------------------

Problem: CXF is not able to generate correct WSDL from java proxies

The whole solution with dynamic proxies is bad. CXF doesn't want to create WSDL based on the annotation interface. It inspects the actual class and defined methods ... doesn't work with dynamic proxies.

Solution: Throw out dynamic proxies and generate services wrapper class using Javassist.

The prototype is semi-working, see further.

------------------------------------------------------------

Decisions:
* use OSGi
* use CXF
* use CXF version >= 3.0
* configure CXF programatically
* use install scripts
* use WS-Policy external attachments
* use a OSGi fragment to fix ExternalAttachmentsProvider
* use a custom CXF extension to dynamically map services to external policy file
* use Javassist and dynamic proxies to add @WebService annotation
* use Javassist to generate type wrappers for unmarshallable types
* don't use dynamic proxies and generate wrappers for both services and unmarshallable types

------------------------------------------------------------

Problem: ClassLoading problems

* ClassNotFoundException ... generated classes are loaded by a different classloader
* ClassCastException ... bundle classes are loaded twice, once inside generated classes and once using bundle classloader

Solution: We need custom ClassPool and class loader for defining classes that see all services & bundles classes.

Liferay AggregateClassLoader with WeekReferences is the perfect fit for bundle classloaders that will be dynamically added and removed when OSGi service is registered / removed.

We also need strictly delegating ClassLoader for instantiating CtClasses, because we don't override existing classes.

=> JaxWsClassPool with custom ClassLoader having AggregateClassLoader as parent.

This way generated classses are stored in the custom ClassLoader, existing classes are found using parent classloader - AggregateClassLoader which holds references to bundles' ClassLoaders.

Problem: method.getClass().getPackage() == null for generated classes???

CtClass.toClass() only creates the class object, but not package object. We can use javassist.Loader class loader to generate the package. See https://community.jboss.org/thread/93814?start=0&tstart=0

But, CtClass.toClass() calls directly ClassLoader.define() methods using reflection API => javassist.Loader.findClass() that creates the package object is not called => package is not created.

Solution:
1. We cannot call CtClass.toClass() directly, but ClassPool.getClassLoader().findClass(CtClass.getName()).
2, We need our strict loading ClassLoader to be javassist.Loader:

JaxWsClassPool -> DelegatingLoader (extends javasssist.loader) -> AggregatedClassLoader (references bundles class loaders)

Problem: Primitive conversion: auto-boxing doesn't work using Javassist compiler.

Solution: We need to correctly cast types and convert them so that they match method signatures :/

Ends up in another working prototype

------------------------------------------------------------

Decisions:
* use OSGi
* use CXF
* use CXF version >= 3.0
* configure CXF programatically
* use install scripts
* use WS-Policy external attachments
* use a OSGi fragment to fix ExternalAttachmentsProvider
* use a custom CXF extension to dynamically map services to external policy file
* use Javassist and dynamic proxies to add @WebService annotation
* use Javassist to generate type wrappers for unmarshallable types
* don't use dynamic proxies and generate wrappers for both services and unmarshallable types
* use custom class pool with custom class loading architecture
* use type casting for primitives, auto-box primitives manually

------------------------------------------------------------

Problem: Type wrappers with auto-boxed primitives cannot be used as original type instances.

There is a clash between a type wrapper implementing original type (model class) and the original type.

Having primitive field "int", the type Wrapper has Integer => getInt() must return Integer for JAXB and "int" to satisfy the implementation.

Problem: Unmarshallable types inside collections and maps cannot be handled by JAXB.

Having an ArrayList with Objects, we don't wrap instances of the objects -> JAXB doesn't know how to marshall the objects, because we don't convert them into type wrappers.

Solution: 
1, Avoid type wrappers conversion at all, delegate types conversion on JAXB.
2, Generate Type Wrappers as now, but implement the type <-> type wrapper conversion inside custom XMLAdapters.
2,  Dynamically generate the XMLAdapter classes for types that cannot be marshalled.
3,  Use @XmlJavaTypeAdapter on package level to define supported type conversion

This way the type can be converted in any situation.

... to be implemented

Friday, June 20, 2014

LinkedIn Stored XSS Vulnerability

TL;DR: I found a stored XSS on LinkedIn, that's all folks, nothing special.

----

I was invited to publish on LinkedIn.

I wondered how good is LinkedIn at multi-byte UTF-8 support (fixed some bugs recently in Liferay related to escaping and surrogate pairs) and noticed a strange thing.

When processing URLs, LinkedIn exchanges escaped characters with their unescaped form = removes the escaping.

&#34; → "

Then it was easy to create some vectors to try the stored XSS:





----

Timeline:
June 17, 2014 10:43 PM CEST – Reported to Linkedin Security Team
June 17, 2014 11:09 PM CEST – ACKed they received it
June 18, 2014 01:04 AM CEST – Reproduced
June 19, 2014 04:12 AM CEST – Got email that it's fixed

----

After this I was a bit scared so I quickly looked also at other LinkedIn features that I use. And found another vulnerability in the feature I trust and would be a good victim for :/

But, for now, please stay tuned until they fix it. You know, I'm the white-hat = harmless ;)

Friday, February 14, 2014

How is it with URL encoding + XSS

Correct URLs


All URIs need to be encoded using percent encoding as specified in rfc3986.

URLs may consist of:
* reserved characters - when used with their special meaning: !*'();:@&=+$,/?#[]
* unreserved characters - don't need to be encoded: a-z A-Z 0-9 -._~
* any other characters must be encoded using %xx hex escaping sequence using UTF-8 bytes

This means:
http://example.com/příliš/žluťoučký;kůň=úpěl?ďábelské=ódy is a nice invalid URL.

Correct is:
http://example.com/p%C5%99%C3%ADli%C5%A1/%C5%BElu%C5%A5ou%C4%8Dk%C3%BD;k%C5%AF%C5%88=%C3%BAp%C4%9Bl?%C4%8F%C3%A1belsk%C3%A9=%C3%B3dy

Question: Are URLs in my web app broken?
Answer: Unless you always go with ASCII ... probably yes :)
Better answer: It doesn't matter, browsers are smart enough to correctly parse the simple ones

Semantics of URL parts

Important to know ⁉
  • URL = decode(encode(URL))
  • encode(URL) != encode(decode(encode(URL)))
Decoding depends on correct knowledge of the parts being encoded!
http://example.com/evaluate/3%2B2%2F5 ⇝ http://example.com/evaluate/3+2/5
http://example.com/evaluate/3%2B2/5 ⇝ http://example.com/evaluate/3+2/5
=> ENCODING is responsible to transfer the knowledge into correctly constructed URL
=> only then DECODING is able to correctly identify the parts

Exercise from [2]:
http://example.com/:@-._~!$&'()*+,=;:@-._~!$&'()*+,=:@-._~!$&'()*+,==?/?:@-._~!$'()*+,;=/?:@-._~!$'()*+,;==#/?:@-._~!$&'()*+,;=

The important are delimiters - reserved characters.

URLs & HTML = XSS ?


URLs in HTML = replace & for &amp; But what about URLs and XSS?

Q: Can there be a XSS in URL? 
A: Inside not encoded URL ... YES!
Example URL: http://localhost/'"><img src=x onerror=alert(1)>
⇝ <a href="http://localhost/'"><img src=x onerror=alert(1)>">link</a>

Q: Can there be a XSS in correctly encoded URL?
A: Yes!
Example URL: http://localhost/'+alert(1)+'
Correctly encoded: http://localhost/'+alert(1)+'
<script>location.href='http://localhost/'+alert(1)+'';</script>

Summary

Q: Does it matter when I don't encode URLs using percent-encoding syntax?
A: Only when parts contain reserved characters.

=> ENCODE user-supplied parts of the URL

http://example.com/evaluate/ + encode("3+2/5") ⇝ http://example.com/evaluate/3%2B2%2F5

Q: Can there be a XSS in URL?
A: Yes.

=> ESCAPE user-supplied URLs based on the surrounding context (see OWASP cheet sheet)!

⇝ <a href="http&#x3a;&#x2f;&#x2f;localhost&#x2f;&#x27;&#x22;&#x3e;&#x3c;img&#x20;src&#x3d;x&#x20;onerror&#x3d;alert&#x28;1&#x29;&#x3e;">link</a>

<script>location.href='http\x3a\x2f\x2flocalhost\x2f\x27\x22\x3e\x3cimg\x20src\x3dx\x20onerror\x3dalert\x281\x29\x3e';</script>

Where to continue...


(for serious readers, ordered by simplicity)