Marketplace Tutorial

To use the command line interface, it must be installed on your machine. Please use the referenced package stratuslab-cli-user-1.21* or later for working with the Marketplace. Note: The StratusLab command line interface requires Python 2.6 or later.

• Marketplace Technical Note
• Tutorial at EGI UF (see the Marketplace presentation in the “slides” link)

Creating an image is a time-consuming process so for this tutorial we will cheat.

Create a fake machine image file with random contents:

$ dd of=dummy.img if=/dev/random bs=1024 count=10
10+0 records in
10+0 records out
10240 bytes transferred in 0.001461 secs (7008759 bytes/sec)

The file would also need to be uploaded to a server that will allow access to it via http or https. The location URL would normally be included as part of the metadata uploaded to the MarketPlace.

The metadata description contains information about the machine or disk image in an RDF-XML format. This is not particularly friendly for human editing, so StratusLab provides a command to help create the necessary file.

Use the following command:

$ stratus-build-metadata \
  --author='Charles Loomis' \
  --os=ttylinux \
  --os-version=10.0 \
  --os-arch=i486 \
  --version=1.0 \
  dummy.img 

This will create a file called “ttylinux-10.0-i486-base-1.0.xml” with a skeleton of the image metadata.

The skeleton already contains the image identifier (27 character base-64 string) based on the image's SHA-1 checksum. It also contains the size of the image in bytes and several other checksums. These are used to verify the correspondance between the metadata description and the actual machine or disk image file.

Any additions or modifications to the file must be done before the metadata entry is signed. The following is a fairly complete metadata description and should give you an idea of what you can add.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
        xmlns:dcterms="http://purl.org/dc/terms/" xmlns:slreq="http://mp.stratuslab.eu/slreq#"
        xmlns:slterms="http://mp.stratuslab.eu/slterms#" xml:base="http://mp.stratuslab.eu/"
        xmlns:ex="http://example.org/">

        <rdf:Description rdf:about="#MMZu9WvwKIro-rtBQfDk4PsKO7_">

                <dcterms:identifier>MMZu9WvwKIro-rtBQfDk4PsKO7_</dcterms:identifier>

                <slreq:bytes>100</slreq:bytes>

                <slreq:checksum rdf:parseType="Resource">
                        <slreq:algorithm>SHA-1</slreq:algorithm>
                        <slreq:value>c319bbd5afc0a22ba3eaed0507c39383ec28eeff</slreq:value>
                </slreq:checksum>

                <slreq:endorsement rdf:parseType="Resource" />

                <dcterms:type>machine</dcterms:type>

                <dcterms:valid>2011-07-23T10:59:42Z</dcterms:valid>

                <dcterms:publisher>StratusLab</dcterms:publisher>
                <dcterms:title>linux-with-my-apps</dcterms:title>
                <dcterms:description>A 32-bit ttylinux...</dcterms:description>

                <slterms:serial-number>0</slterms:serial-number>
                <slterms:version>1.0</slterms:version>

                <slterms:hypervisor>kvm</slterms:hypervisor>

                <slterms:inbound-port>443</slterms:inbound-port>
                <slterms:outbound-port>25</slterms:outbound-port>
                <slterms:icmp>8</slterms:icmp>

                <slterms:os>ttylinux</slterms:os>
                <slterms:os-version>10.0</slterms:os-version>
                <slterms:os-arch>i486</slterms:os-arch>
                
                <slterms:location>http://example.org/my-image</slterms:location>

                <slterms:deprecated>security issue with app</slterms:deprecated>

                <ex:other-info>additional metadata</ex:other-info>
                <ex:yet-more>still more info</ex:yet-more>

                <ex:relatedImages rdf:parseType="Resource">
                        <dcterms:identifier>MMZu9WvwKIro-rtBQfDk4PsKO7_</dcterms:identifier>
                        <dcterms:identifier>NMZu9WvwKIro-rtBQfDk4PsKO7_</dcterms:identifier>
                        <dcterms:identifier>OMZu9WvwKIro-rtBQfDk4PsKO7_</dcterms:identifier>
                        <dcterms:identifier>PMZu9WvwKIro-rtBQfDk4PsKO7_</dcterms:identifier>
                </ex:relatedImages>

        </rdf:Description>
</rdf:RDF>

In particular, you will probably want to add one or more location elements containing the URL(s) where the image can be found. The MarketPlace technical note contains the standard elements. You may also add elements from other namespaces (“ex” in this document) before the end of the description. Note, however, that the file must remain a valid RDF description so you may have to use “rdf:parseType” attributes on elements with children.

Try to validate the unsigned file.

$ stratus-validate-metadata ttylinux-10.0-i486-base-1.0.xml 
Invalid: ttylinux-10.0-i486-base-1.0.xml
no signature

As you can see, this fails (as it should) indicating that there is no signature in this file.

To sign the file, you will need a PKSC12-formatted certificate. Any certificate will do, although if you want a system administrator down the like to accept the signature, you should use a valid grid certificate.

To sign the file, use the stratus-sign-metadata command:

$ stratus-sign-metadata \
    --p12-cert grid.p12 \
    --p12-password xxxxxx \
    --output ttylinux-10.0-i486-base-1.0.xml.sign.xml \
    ttylinux-10.0-i486-base-1.0.xml
    Metadata file successfully signed: ttylinux-10.0-i486-base-1.0.xml.sign.xml

If you don't provide a password on the command line (or in the STRATUSLAB_P12_PASSWORD variable), you will be prompted for it. If the command was successful, it will write the result to the same file name as the input. Use the –output option (as here) to write the output to a different file.

Now try to validate the signed file:

$ stratus-validate-metadata ttylinux-10.0-i486-base-1.0.xml.sign.xml
Valid: ttylinux-10.0-i486-base-1.0.xml.sign.xml

If everything has worked correctly, the command should indicate that the file is valid.

Now you will want to upload the metadata description. Only valid, signed descriptions will be accepted by the MarketPlace server. Use the following command:

$ stratus-upload-metadata \
    --marketplace-endpoint=http://marketplace.stratuslab.eu
    ttylinux-10.0-i486-base-1.0.xml.sign.xml
 http://marketplace.stratuslab.eu/metadata/MMZu9WvwKIro-rtBQfDk4PsKO7_/email address/2011-06-29T10:14:41Z

This should return the URL of the metadata entry. You should be able to navigate to the metadata listings and see your metadata entry.

Note: Depending on the configuration of the server, it may validate the email address in the metadata description before it is made visible. Check with the system administrator for the service to see if email validation is turned on.

You can also upload the signed metadata from the web interface. Just click on the “upload” link, search for the metadata description to upload, and click the upload button.

On the query page, you can use SPARQL queries to search for metadata entries. For example the following will list the identifiers and operating systems for all of the metadata descriptions:

PREFIX dcterms: <http://purl.org/dc/terms/>
PREFIX slterms: <http://mp.stratuslab.eu/slterms#>

select

  distinct ?os ?identifier 

  where { 
    ?x dcterms:identifier ?identifier . 
    ?x slterms:os ?os
  }

Similarly the following will search for entries signed by an endorser with the email address “me@example.com”.

PREFIX dcterms: <http://purl.org/dc/terms/>
PREFIX slreq: <http://mp.stratuslab.eu/slreq#>

select
  distinct ?identifier ?email ?created 
  where 
  {
    ?x dcterms:identifier ?identifier;

    slreq:endorsement ?endorsement . 
    ?endorsement slreq:endorser ?endorser;

    dcterms:created ?created .
    ?endorser slreq:email ?email .
    FILTER (?email = "me@example.com")
  }

Generally, the search boxes on the metadata listing page would be more convenient for humans. The above queries are better for automated queries from scripts.

To use an image described in the Marketplace, simply give the image identifier to the usual stratus-run-instance command. The reference can be used for machine images or for static disk images.