Skip to content

DockerHub publish through CI#412

Merged
mcserep merged 5 commits intoEricsson:masterfrom
mcserep:ci_docker
Aug 4, 2020
Merged

DockerHub publish through CI#412
mcserep merged 5 commits intoEricsson:masterfrom
mcserep:ci_docker

Conversation

@mcserep
Copy link
Collaborator

@mcserep mcserep commented Jul 26, 2020

This PR extends the continuous integration workflow to following the successful compiling and testing of CodeCompass, build the Docker images and publish them to DockerHub, to:
https://hub.docker.com/r/modelcpp/codecompass

The following images are built and published:

  • modelcpp/codecompass:dev, the development image (ready!)
  • modelcpp/codecompass:runtime-sqlite, the runtime image containing CodeCompass binaries built against SQLite (can be done after Docker runtime image #409 merged)
  • modelcpp/codecompass:runtime-pgsql, the runtime image containing CodeCompass binaries built against PostrgreSQL (can be done after Docker runtime image #409 merged)
  • modelcpp/codecompass:web-sqlite, the webserver executing container image built against SQLite (ready!)
  • modelcpp/codecompass:web-pgsql, the webserver executing container image built against PostgreSQL (can be done after Use PostgreSQL in Docker containers #410 merged)

I propose that the default :latest tag should be an alias to :runtime-pgsql.

@mcserep mcserep added Kind: Enhancement 🌟 Status: WIP 👷 Issue or PR under development - feel free to review, though! Target: Developer environment Developer environment issues consist of CodeCompass or 3rd-party build tooling, configuration or CI. labels Jul 26, 2020
@mcserep mcserep added this to the Release Flash milestone Jul 26, 2020
@mcserep mcserep requested review from bruntib and whisperity July 26, 2020 08:22
@mcserep mcserep force-pushed the ci_docker branch 2 times, most recently from 3eae4c2 to a24f331 Compare July 26, 2020 08:44
.travis.yml Outdated
Comment on lines 383 to 384
- compile
- test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the separation of these two stages for? Remember #396, I had seen tangible issues with caching between two jobs of the same build... Even if these are the same build chain, the two jobs are entirely separate virtual machines running, and it seemed that -- for the dependencies, as #396 was meant to cache that -- there is some weird nondeterminism...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default stages in Travis are: compile, test and deploy. Until now we have not used stages and all jobs were put into the test stage by default.

Stages are sequential, so e.g. the jobs in the deploy stage are only executed when all jobs in the previous test are finished (and passed). That was my aim here: to only build the Docker images and publish them if the previous jobs have succeeded, because otherwise something might be off and we shall not roll out the new Docker images to the public.
This deploy stage in our case is also configured that it is only executed in the original repository (not in the forked ones) and only upon pushes to the master branch (pull requests to the master are also excluded that way).

The compile stage can be omitted and removed, because we do not use it. Initially I have just listed the default stages and it was left there.

Copy link
Contributor

@whisperity whisperity Jul 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright. That distinction is okay with me. I just wanted to get ahead of the possibility of trying to share things between the stages, as I have experienced it not working in the recent past. (We can't really split the "compile" and "test" stages without sharing the build between the two...)

And I agree with running deploy only if we're comfortable with the test stage.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, Travis has only a limited support to transfer, well, anything between stages. Only solvable with external providers, e.g. with Amazon S3.
https://docs.travis-ci.com/user/build-stages/#data-persistence-between-stages-and-jobs
https://docs.travis-ci.com/user/build-stages/share-files-s3/

Fortunately we do not need this functionality now, the Docker images are built from scratch each time.
(GitLab e.g. performs way better in this aspect, supporting to transfer so called artifacts between jobs and stages.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, should we remove "compile" from the list, and just keep "test"?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed it now to avoid any possible confusion.

@whisperity
Copy link
Contributor

So runtime is the image that contains both parsing and webserver operations, and web is the one that is only the web server? Or am I missing something?

@mcserep
Copy link
Collaborator Author

mcserep commented Jul 26, 2020

So runtime is the image that contains both parsing and webserver operations, and web is the one that is only the web server? Or am I missing something?

Basically yes. To be more precise:

  • the runtime image contains the binaries (and their dependencies), and that is all.
  • the web image contains the binaries (and only dependencies of the webserver) and it is configured that upon starting a container created from it, it will launch the webserver as a non-root user.

The runtime image will be available after #409 .

@whisperity
Copy link
Contributor

As for the web image, as I mentioned in #398 already, perhaps, as it is once again not in the scope of this patch, but you should add a TODO on having the web image not contain the parser binary's stuff and install. (I think the current build system is not capable of building just the web stuff.)

@mcserep mcserep requested a review from whisperity July 31, 2020 13:49
@mcserep mcserep removed the Status: WIP 👷 Issue or PR under development - feel free to review, though! label Jul 31, 2020
@mcserep
Copy link
Collaborator Author

mcserep commented Jul 31, 2020

PR completed, please review.

Copy link
Contributor

@whisperity whisperity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. I assume we'll have to set Travis up with the appropriate creds.

@mcserep
Copy link
Collaborator Author

mcserep commented Aug 4, 2020

Looks good to me. I assume we'll have to set Travis up with the appropriate creds.

Already configured in Travis :)

@mcserep mcserep requested review from zporky and removed request for zporky August 4, 2020 09:21
@mcserep mcserep merged commit 8875bfb into Ericsson:master Aug 4, 2020
@mcserep mcserep deleted the ci_docker branch August 26, 2020 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Kind: Enhancement 🌟 Target: Developer environment Developer environment issues consist of CodeCompass or 3rd-party build tooling, configuration or CI.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants