Disclaimer:
- It is not about storing python packages in Artifactory.
- It is not about creating python packages in TravisCI and publishing them to Artifactory.
In this article I’ll show you how to set up Artifactory in Travis CI for using it in your python project’s tests.
Why do I (and may be you) need this?
Imagine you develop an application, which talks to Artifactory. If you can’t – take a look at Enot – advanced Erlang package manager. In a few words – it is a hybrid of maven and pip but for Erlang.
One of the things it can do: is to compile Erlang project into a package and load it to a remote repository. Once loaded, it can be used everywhere without compiling. Enot will just download package and link it to your project. Compile once – use many.
Great! How is it related to Artifactory? It’s simple – Enot use several backend types for remote repositories. Artifactory is one of them. And I have tests for a class which works with Artifactory, to ensure that everything is ok.
All my tests are run through make tests and I set up Travis easily by placing .travis.yml in my repository root.
sudo: required dist: trusty language: python python: - "3.3" - "3.4" - "3.5" - "3.6" before_install: - sudo apt-get -y install erlang install: "python setup.py install" script: - make tests
Now Travis will run tests against 4 python versions on push to every branch and pull request opened on github.
Problem – no local Artifactory service
If your tests use local Artifactory there can be a problem. Travis doesn’t have such service locally. You need to add it somehow. Actualy Travis is poor for existent connection services. There are ready some, but only to a few popular databases. So you need to install, run and configure artifactory before running tests.
Solution – installing Artifactory before tests
Option 1 – installing Artifactory service from deb
dist: trusty tells Travis to create Ubuntu Trusty build environment. It is quite easy to install Artifactory on Ubuntu Trusty. From JFrogs installation guide:
echo "deb https://jfrog.bintray.com/artifactory-pro-debs xenial main" | sudo tee -a /etc/apt/sources.list curl https://bintray.com/user/downloadSubjectPublicKey?username=jfrog | sudo apt-key add - sudo apt-get update sudo apt-get install jfrog-artifactory-oss
But this means every time you run tests – Artifactory will be downloaded and installed. This is not cool, as it takes some time to download artifactory and about 2 minutes for thre first run.
Option 2 – manual Artifactory installation from zip
As I didn’t want to download and set-up Artifactory every time I run tests, I decided to download Artifactory zip package and cache it between test runs.
Travis allows caching some data in directories, specified in .travis.yml file. The content of these directories is packed in tar and uploaded to S3. Before building Travis check if a cached archive exists. If it does, it will be pulled and unpacked to the specified location. Travis has one cache per branch and language version/compiler version/JDK version/Gemfile location/etc.
Modify .travis.yml to set up cache directory $HOME/artifactory and include install_deps.sh running before installation. Also you will need to add Artifactory starting in a background via `$HOME/artifactory/artifactory/bin/artifactory.sh &` and 2 min delay to wait for the first start. Later delay can be decreased to 10 sec.
sudo: required dist: trusty language: python python: - "3.3" - "3.4" - "3.5" - "3.6" before_install: - sudo apt-get -y install erlang - bash install_deps.sh install: "python setup.py install" addons: cache: directories: - $HOME/artifactory script: - $HOME/artifactory/artifactory/bin/artifactory.sh & - sleep 120 - make tests
Add install_deps.sh which should download and untar Artifactory zip package if cache directory doesn’t exsist.
#!/usr/bin/env bash
if [ ! -d "$HOME/artifactory/artifactory" ]; then
mkdir -p $HOME/artifactory
artifactorywget https://bintray.com/jfrog/artifactory/download_file?file_path=jfrog-artifactory-oss-5.3.0.zip \
-O $HOME/artifactory/artifactory.zip
unzip $HOME/artifactory/artifactory.zip -d $HOME/artifactory
mv $HOME/artifactory/artifactory-oss-* $HOME/artifactory/artifactory
else
echo "Using cached artifactory."
fi
If $HOME/artifactory exists in system – it will be used:
Zip package config makes Artifactory treat dir it was downloaded to (cache dir in our case) as installation dir. So – first run configuration results are also cached!
Any other time it takes about 9 seconds for Artifactory to start:
As for configuration, provided in etc/artifactory.config.xml shipped with artifactory.zip – generic example-repo-local will be created with default credentials mentioned in README.md:
username: admin password: password
That is what I exactly need! Now Artifactory test repo is available for Travis at http://localhost:8081/artifactory/example-repo-local and my Artifactory tests run without problem.
Kate Belova