Skip to main content

Hello world image

In this tutorial you will recreate the steps needed to build and run the tutorial/hello-world image used in the Quick start section.

Source code​

To create the tutorial/hello-world image you will require the following file structure:

batchx-hello-world/
├── Dockerfile
├── manifest
│   └── manifest.json
└── hello-world.py

Dockerfile​

Docker builds images automatically by reading lines of instructions from a Dockerfile. A Dockerfile is a text document that contains all the commands needed to assemble an image. Learn more about Docker files here.

Create a file with the content shown below, make sure you save this file inside the project folder with the name: Dockerfile.

info

These configuration files can have any name, however, naming them as Dockerfileis a standard practice. This also simplifies the command to build the image, given that it's the default name expected by the Docker client.

FROM python:3.6.8-alpine3.8
COPY hello-world.py hello-world.py
RUN chmod +x hello-world.py
ENTRYPOINT python hello-world.py
LABEL io.batchx.manifest=11
COPY manifest /batchx/manifest/

Line explanations:​

  1. The FROM instruction sets the base image this will extend. This one is a official python image for Alpine (a very lightweight Linux distribution).
  2. Copies the python script to the image file-system, within the working directory and with the same name.
  3. Changes the permissions of the image script, so it's executable.
  4. Set the scripts as the command to run when the container is started.
  5. Declares that next instruction will attach the manifest to the image file-system.
  6. Copies the manifest to image file-system.

Manifest​

The manifest defines the purpose, usage and other important information about the image. See The BatchX contract for detailed information.

manifest/manifest.json
{
  "name": "tutorial/hello-world",
  "version": "0.0.1",
  "title": "BatchX Hello World",
  "schema": {
    "input": {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "yourName": {
          "type": "string",
          "required": true,
          "title": "Your name."
        }
      }
    },
    "output": {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "responseFile": {
          "required": true,
          "type": "string",
          "format": "file",
          "title": "Hello World response file."
        }
      }
    }
  },
  "author": "BatchX",
  "changeLog": "Initial version",
  "runtime": {
    "minMem": 1000
  }
}

Entry-point​

Docker enables polyglot development and BatchX maintains this same philosophy, working seamlessly with any programming language. Below is the entry-point script hello-world.py, which is written in Python but any programming language is supported (JavaScript, Java, C++, R, Bash, ...).

info

Since the hello-world.py script will be executed inside the Docker container, you are not required to have Python installed in your system, although it might be useful for debugging.

hello-world.py
#!/usr/bin/python3
import json
import os

# Print environment
print("Environment:")
print(os.environ)

# Parse the input JSON message
with open("/batchx/input/input.json", "r") as inputFile:
    inputJson = inputFile.read()
parsedJson = json.loads(inputJson)

# Get "yourName" property from the input
yourName = parsedJson["yourName"]

# Create an output directory
outputDir = "/batchx/output/response/"
os.mkdir(outputDir)

# Write response to output file
outputFile = outputDir + "response.txt"
responseFile= open(outputFile,"w+")
responseFile.write("Hello " + yourName + ". Welcome to BatchX!\n")

# Write the output JSON message (pointing to the output file)
outputJson = {
    'responseFile': outputFile
}
with open('/batchx/output/output.json', 'w+') as json_file:
    json.dump(outputJson, json_file)

Building the image​

NOTE: If you are using a Mac computer with an Apple Silicon chip (not Intel), to avoid errors when importing the image later, you must first execute the following command:

export DOCKER_DEFAULT_PLATFORM=linux/amd64

This will modify the environment variable that indicates the default platform that docker will use temporarily (only lasts until you close the terminal session). To see how to set an environment variable permanently, you can check it here.

Now that you have created all the necessary files is time to build the image. For this you will use thedocker build command as shown below, adapting it to work with our hello-world image. Read more about this command here.

docker build -t <image-name> .

Basically, this command tells the Docker client to build an image from the Dockerfile in the current directory (that’s what the . at the end represent).

Therefore, the build command should look similar to the one shown below:

docker build -t batchx-hello-world .

Running the command from above will build our image. It might take a few minutes the first time. When it finishes you will obtain an output similar to the one shown below.

$ docker build -t batchx-hello-world .
Sending build context to Docker daemon   7.68kB
Step 1/6 : FROM python:3.6.8-alpine3.8
 ---> b6bfe2b71f9d
Step 2/6 : COPY hello-world.py hello-world.py
 ---> Using cache
 ---> af8b4bfc3e55
Step 3/6 : RUN chmod +x hello-world.py
 ---> Using cache
 ---> de77e1d51199
Step 4/6 : ENTRYPOINT python hello-world.py
 ---> Using cache
 ---> a2f875761c9b
Step 5/6 : LABEL io.batchx.manifest=10
 ---> Using cache
 ---> eaf6dfaef098
Step 6/6 : COPY manifest /batchx/manifest/
 ---> Using cache
 ---> d859a29a8850
Successfully built d859a29a8850
Successfully tagged batchx-hello-world:latest

The last two lines of this output inform us that the building and the tagging processes have ran successfully. You are now ready to test this image locally.

Testing locally​

Now that you have built the hello-world image, it is time for using the bx run-local command to test it locally.

This command mimics a job executed in BatchX using the run command. Therefore, is a reliable mechanism to test Docker images before uploading them to BatchX.

Running this image locally will produce an output similar to the one below:

$ bx run-local batchx-hello-world '{"yourName":"Michael Jordan"}'
[batchx] [2022/08/11 19:08:06] Running job locally...
[batchx] [2022/08/11 19:08:06] Storing job information at /home/user/.bx/fs/jobs/1660237686170 for 4 days
[batchx] [2022/08/11 19:08:06] Non-pipeline image found. Running container without access to BatchX
[batchx-hello-world] Environment:
[batchx-hello-world] environ({'HOSTNAME': '23cfcbb4a9ff', 'PYTHON_PIP_VERSION': '19.1.1', 'SHLVL': '1', 'HOME': '/root', 'GPG_KEY': '0D96DF4D4110E5C43FBFB17F2D347EA6AA65421D', 'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'BX_MEMORY': '1000', 'LANG': 'C.UTF-8', 'PYTHON_VERSION': '3.6.8', 'BX_VCPUS': '1', 'PWD': '/'})
[batchx] [2022/08/11 19:08:07] Container exit code 0
{"responseFile":"/home/user/.bx/fs/jobs/1660237686170/output/response/response.txt"}

As shown in the last line, outputs generated when running images with the run-local command will be stored under the /home/<user_name>/.bx/fs/jobs/ path.

Print the content of the response.txt file with Linux cat command to verify our image worked as it suppose to. If you obtain an output similar to the one shown below you are ready to push this image to your Docker Hub account.

$ cat /home/user/.bx/fs/jobs/1660237686170/output/response/response.txt
Hello Michael Jordan. Welcome to BatchX!

Importing to BatchX​

At this point you have successfully build the image and tested it locally. Now it is time to import it into BatchX.

First of all, if you are not logged into the platform yet, you must do it running bx login command:

$ bx login

This will open a new window in the browser where you can login, and then return to the already logged in terminal.

Next step before importing the image, will be to select the environment in which we are going to import it. To will do this running bx env command:

$ bx env <environment>

After this, you will import the image by running bx import command.

$ bx import <image-hash>

This command will import a local image into BatchX using the <image-hash>, this hash is shown in the second to last line printed when building the image. These are the last two lines printed when building the hello-world image:

Successfully built d859a29a8850
Successfully tagged batchx-hello-world:latest

NOTE: If you're using a Mac computer, the hash may not appear this way. To obtain it, you can use the command below. The IMAGE ID column contains the hash of the images stored in your machine. Use the one corresponding to the hello-world image.

$ docker images

Running the import command using the image hash (d859a29a8850) will produce an output similar to the one below:

$ bx import d859a29a8850
[batchx] [2022/08/11 19:26:58] Pushing local image
The push refers to repository [079621346490.dkr.ecr.eu-west-1.amazonaws.com/user/tutorial/hello-world]
f037debdf0ac: Preparing
0792a0a8114f: Preparing
eec9707b92f9: Preparing
9eb30c835f51: Preparing
14d6a951dd6d: Preparing
8384d6abc572: Preparing
9797041513df: Preparing
d9ff549177a9: Preparing
8384d6abc572: Waiting
9797041513df: Waiting
d9ff549177a9: Waiting
eec9707b92f9: Pushed
0792a0a8114f: Pushed
f037debdf0ac: Pushed
14d6a951dd6d: Pushed
9eb30c835f51: Pushed
9797041513df: Pushed
d9ff549177a9: Pushed
8384d6abc572: Pushed
user-1660238818792: digest: sha256:6b4bae63b601ae1e14d3a7795c02be50d185e8231874929e133a7295bbb5c234 size: 1989
[batchx] [2022/08/11 19:27:15] Importing image into BatchX
[batchx] [2022/08/11 19:27:16] Getting image BatchX manifest
[batchx] [2022/08/11 19:27:16] 'batchx/manifest/readme.md' not found in manifest layer: sha256:7c4e880cb761f77990a87260d2ab43d23d7511c9a4367a4cf695573aa3ab9b9a
[batchx] [2022/08/11 19:27:16] Importing 079621346490.dkr.ecr.eu-west-1.amazonaws.com/user/tutorial/hello-world:user-1660238818792@sha256:6b4bae63b601ae1e14d3a7795c02be50d185e8231874929e133a7295bbb5c234 as user@tutorial/hello-world:0.0.1
[batchx] [2022/08/11 19:27:16] Image imported successfully

The last line of this log informs us that the image has been imported successfully. You can verify this image is now available in your BatchX registry by using the bx images command.

$ bx images
IMAGE                             CREATED            SIZE
<user>@tutorial/hello-world:0.0.1  22 seconds ago  27.5 MB
info

BatchX vs Docker coordinates. Notice that once imported into BatchX, the original (Docker) image coordinates become irrelevant. From this point only the BatchX coordinates (specified in the manifest) will be used.

Running a BatchX job​

Finally you can run the image in BatchX. Use the bx run command to run the image (provide any name you want in the yourName parameter).

$ bx run <user>@tutorial/hello-world:0.0.1 '{"yourName":"Rocky Balboa"}'

Logs should start appearing in your screen as soon as the job is submitted. These logs provide information of the job id, transitions between states and logs generated by the running container.

info

If the environment had no previous activity the job might take a few minutes to start.

$ bx run <user>@tutorial/hello-world:0.0.1 '{"yourName":"Rocky Balboa"}'
[batchx] [INFO] [2020/03/25 13:53:26] Submitting job...
[batchx] [INFO] [2020/03/25 13:53:27] Job submitted with id 2
[batchx] [INFO] [2020/03/25 13:53:27] Attaching to job 2
[batchx] [INFO] [2020/03/25 13:53:28] Job status: SUBMITTED
[batchx] [INFO] [2020/03/25 13:53:37] Job status: STARTING
[batchx] [INFO] [2020/03/25 13:53:37] Job status: DOWNLOADING_INPUT
[batchx] [INFO] [2020/03/25 13:53:37] Job status: RUNNING
[david@tutorial/hello-world:0.0.1] [2020/03/25 13:53:40] Environment:
[david@tutorial/hello-world:0.0.1] [2020/03/25 13:53:40] environ({'HOSTNAME': 'batchx-container', 'PYTHON_PIP_VERSION': '19.1.1', 'SHLVL': '1', 'HOME': '/root', 'GPG_KEY': '0D96DF4D4110E5C43FBFB17F2D347EA6AA65421D', 'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'BX_MEMORY': '1000', 'LANG': 'C.UTF-8', 'BX_GPUS': '0', 'PYTHON_VERSION': '3.6.8', 'BX_VCPUS': '1', 'PWD': '/batchx'})
[batchx] [INFO] [2020/03/25 13:53:47] Job status: UPLOADING_OUTPUT
[batchx] [INFO] [2020/03/25 13:53:49] Job status: SUCCEEDED
{"responseFile":"bx://jobs/2/output/response/response.txt"}
info

All the logs lines generated so far have been written to stderr. If the job finishes successfully though, the last line will be written to stdout containing the job output message. This help piping jobs, as we will see in the next tutorial.

Given that the output contains a reference to a file, you can use bx cat to inspect its contents:

$ bx cat bx://jobs/2/output/response/response.txt
Hello Rocky Balboa. Welcome to BatchX!

Congratulations! You completed this tutorial!
We have covered a lot of ground here: You created a Docker image, tested it locally, uploaded to Docker Hub, imported to BatchX and ran it!

Please give yourself an applause!