Enable KAS by antt1995 · Pull Request #2598 · sameersbn/docker-gitlab

antt1995 · 2022-06-30T11:58:03Z

Enabling KAS

Not sure what to put or where in the Readme

antt1995 · 2022-06-30T12:04:59Z

wulfdj · 2022-07-01T06:47:45Z

Are these settings all I need? I've tried to change on the running container but still doesn't work. On the clusters page I get the error message that GRPC is not available

blattersturm · 2022-07-02T09:02:52Z

This seems to only set the configuration settings in case you're trying to use an external KAS install.

The Omnibus repo seems to use some more steps to deploy the built-in KAS instance.

kkimurak

@antt1995 Thank you for your contribution.
Those settings seems fine, but we need some step to make them accessible.

Please add function (like gitlab_configure_gitlab_kas()) to assets/runtime/functions
Maybe simplest reference is gitlab_configure_monitoring():

docker-gitlab/assets/runtime/functions

Lines 264 to 273 in b34d48a

    
           gitlab_configure_monitoring() { 
        
             echo "Configuring gitlab::monitoring..." 
        
             update_template ${GITLAB_CONFIG} \ 
        
               GITLAB_MONITORING_UNICORN_SAMPLER_INTERVAL \ 
        
               GITLAB_MONITORING_IP_WHITELIST \ 
        
               GITLAB_MONITORING_SIDEKIQ_EXPORTER_ENABLED \ 
        
               GITLAB_MONITORING_SIDEKIQ_EXPORTER_ADDRESS \ 
        
               GITLAB_MONITORING_SIDEKIQ_EXPORTER_PORT 
        
           }

So the implementation will be like:

gitlab_configure_gitlab_kas() {
  echo "Configuring gitlab::KAS..."

  update_template ${GITLAB_CONFIG} \
    GITLAB_KAS_ENABLED \
    GITLAB_KAS_SECRET \
    GITLAB_KAS_EXTERNAL \
    GITLAB_KAS_INTERNAL \
    GITLAB_KAS_PROXY
}

Call the function in configure_gitlab(). This function will be called on container startup.

docker-gitlab/assets/runtime/functions

Lines 1814 to 1877 in b34d48a

    
           configure_gitlab() { 
        
             echo "Configuring gitlab..." 
        
             update_template ${GITLAB_CONFIG} \ 
        
               GITLAB_INSTALL_DIR \ 
        
               GITLAB_SHELL_INSTALL_DIR \ 
        
               GITLAB_DATA_DIR \ 
        
               GITLAB_REPOS_DIR \ 
        
               GITLAB_DOWNLOADS_DIR \ 
        
               GITLAB_SHARED_DIR \ 
        
               GITLAB_HOME \ 
        
               GITLAB_HOST \ 
        
               GITLAB_PORT \ 
        
               GITLAB_RELATIVE_URL_ROOT \ 
        
               GITLAB_HTTPS \ 
        
               GITLAB_SSH_HOST \ 
        
               GITLAB_SSH_LISTEN_PORT \ 
        
               GITLAB_SSH_PORT \ 
        
               GITLAB_SIGNUP_ENABLED \ 
        
               GITLAB_IMPERSONATION_ENABLED \ 
        
               GITLAB_PROJECTS_LIMIT \ 
        
               GITLAB_USERNAME_CHANGE \ 
        
               GITLAB_DEFAULT_THEME \ 
        
               GITLAB_CREATE_GROUP \ 
        
               GITLAB_ISSUE_CLOSING_PATTERN 
        
             gitlab_configure_database 
        
             gitlab_configure_redis 
        
             gitlab_configure_actioncable 
        
             gitlab_configure_secrets 
        
             gitlab_configure_sidekiq 
        
             gitlab_configure_gitaly 
        
             gitlab_configure_monitoring 
        
             gitlab_configure_gitlab_workhorse 
        
             gitlab_configure_relative_url 
        
             gitlab_configure_trusted_proxies 
        
             gitlab_configure_puma 
        
             gitlab_configure_timezone 
        
             gitlab_configure_rack_attack 
        
             gitlab_configure_ci 
        
             gitlab_configure_artifacts 
        
             gitlab_configure_packages 
        
             gitlab_configure_terraform_state 
        
             gitlab_configure_lfs 
        
             gitlab_configure_uploads 
        
             gitlab_configure_mattermost 
        
             gitlab_configure_project_features 
        
             gitlab_configure_mail_delivery 
        
             gitlab_configure_mailroom 
        
             gitlab_configure_oauth 
        
             gitlab_configure_ldap 
        
             gitlab_configure_gravatar 
        
             gitlab_configure_cron_jobs 
        
             gitlab_configure_analytics 
        
             gitlab_configure_backups 
        
             generate_registry_certificates 
        
             gitlab_configure_registry 
        
             gitlab_configure_pages 
        
             gitlab_configure_sentry 
        
             generate_healthcheck_script 
        
             gitlab_configure_content_security_policy 
        
             # remove stale gitlab.socket 
        
             rm -rf ${GITLAB_INSTALL_DIR}/tmp/sockets/gitlab.socket 
        
           }

(optional) The variable naming rule is not strictly defined, but it seems converting all yaml key name into UPPER_CASE is common approach for this repository.
- GITLAB_KAS_SECRET -> GITLAB_KAS_SECRET_FILE
- GITLAB_KAS_EXTERNAL -> GITLAB_KAS_EXTERNAL_URL
- GITLAB_KAS_INTERNAL -> GITLAB_KAS_INTERNAL_URL
- GITLAB_KAS_PROXY -> GITLAB_KAS_K8S_PROXY_URL

antt1995 · 2022-07-04T13:13:38Z

@kkimurak
I have added that to the Functions

kkimurak

Passed build and test! Thanks for your contribution.

Sorry for forgetting to check in the last review, but could you please fill these additional tasks?

add a description of the variables you added to the "Available Configuration Parameters" section of the README.md
check default values for the variables

assets/runtime/env-defaults

antt1995 · 2022-07-05T13:33:49Z

Building Now with basic info in readme

antt1995 · 2022-07-05T15:46:50Z

Not sure what to put or where in the Readme

Still not sure what to put but have put the vars and their defaults

kkimurak

Sorry for late, I have added review for README and fixed typo in my previous suggestion. Could you confirm them?

Best,

README.md

assets/runtime/env-defaults

antt1995 · 2022-07-13T14:15:43Z

@kkimurak Confirmed

Updated to Latest 15.1.2

kkimurak · 2022-07-13T14:24:46Z

@antt1995 Do you mean you confirmed KAS works as expected applying this PR? Then I can approve. Thank you for your contribution and sorry for my rough review.

By the way I have no permission to merge. Please ask for maintainer (@sachilles is active, I think)

antt1995 · 2022-07-13T14:47:35Z

@kkimurak

Sorry for late, I have added review for README and fixed typo in my previous suggestion. Could you confirm them?

That's what I was Confirming

kkimurak · 2022-07-13T15:12:47Z

@antt1995 ok Thank you. I approve the code changes themselves, but I'm not sure if KAS works.
I would like to find time to see if kas works. As mentioned above, using the internal KAS should require more work. If you are ready to use external KAS, please ask @sachilles to merge this (It is nice to confirm if it works building image by youself).

tiehfood · 2022-07-26T13:58:56Z

Would love this feature to be included

sachilles · 2022-07-28T11:01:07Z

Hi there, sorry for being late. I guess @kkimurak did a great job with his valuable hints. However, I don't use KAS and thus, I can't confirm that this PR works.

Nevertheless, I'd like to ask if the configuration file for secrets (GITLAB_KAS_SECRET=${GITLAB_INSTALL_DIR}/.gitlab_kas_secret}) will be generated dynamically? If not, I guess the README.md has to be extended in order to explain the configuration for KAS in detail.

kkimurak

Hi @antt1995 , Sorry for the tedious review process, but after being pointed out by @sachilles I reviewed the README again. Could you please confirm it?

edit: Oops sorry but I have added comment to "resolved" thread (and hidden by default). I apologize for the inconvenience, but for the second item, press "show resolved" and see the last comment.

README.md

eslamhossam23 · 2022-08-16T08:14:16Z

Hello @antt1995 , I tested this pull request on my Gitlab instance but I haven't been able to make it work.

The KAS was detected as installed by the UI as shown here:

However, when adding a new agent, I always got the error message: GRPC::Unavailable.
Upon inspecting the container via ss -lp, I also noticed that the KAS service was never started at port 8153 as it should have been.

Should the service start by itself automatically by setting GITLAB_KAS_ENABLED to true or is there something that needs to be done after that? Thanks!

antt1995 · 2022-08-16T10:33:05Z

@eslamhossam23 Did you map the port in your docker config "-p 8153:8153" or in docker-compose "- 8153:8153"

eslamhossam23 · 2022-08-16T13:19:19Z

Hello @antt1995 yes, I had the port correctly mapped.

The container itself haven't had started the KAS service inside it.
I checked using ss -lp and verified that it was not started.

tiehfood · 2022-08-24T07:52:16Z

@eslamhossam23 I deployed the KAS service with separate docker image and set the variables from this PR corresponding. Then it was working for me

kkimurak · 2022-08-24T08:56:16Z

@eslamhossam23 As already reported above (#2598 (comment)), this image currently does not provides built-in KAS. This PR requires external KAS is exists.

If you have time to do that, could you please try to build and test my support-builtin-kas branch ?
In addition to the work done by @antt1995, I've made sure to build gitlab-kas when building the image and add setting for supervisord to launch kas if enabled. Also I have added configuration file for KAS and minimal configuration parameters.

I have never used Kubernetes so never tested if the registration success, but made sure the gitlab-kas service starts without exiting.
Here are new entries for env of docker-compose.yml on test:

services:
  gitlab:
    environment:
      GITLAB_KAS_ENABLED: 'true'
      # If we don't set this, built-in gitlab-kas will exit with an error
      # See https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/8069d9b79763859b5bcead437a28f4dc73b66ae8/cmd/kas/kasapp/configured_app.go#L96-98
      OWN_PRIVATE_API_URL: grpc://127.0.0.1:8155

sachilles · 2022-09-11T10:40:56Z

Hi,

is this PR considered to be ready to merge?

I guess (and based on this discussion) there should be some comments on using KAS with this image (something like if you want using KAS you'll need to deploy an additional docker image), isn't it?

defp · 2023-09-12T04:21:21Z

@eslamhossam23 I deployed the KAS service with separate docker image and set the variables from this PR corresponding. Then it was working for me

Is there any tutorial? @tiehfood

antt1995 marked this pull request as ready for review June 30, 2022 12:05

kkimurak suggested changes Jul 2, 2022

View reviewed changes

kkimurak suggested changes Jul 5, 2022

View reviewed changes

assets/runtime/env-defaults Outdated Show resolved Hide resolved

kkimurak suggested changes Jul 11, 2022

View reviewed changes

README.md Outdated Show resolved Hide resolved

assets/runtime/env-defaults Outdated Show resolved Hide resolved

kkimurak mentioned this pull request Jul 20, 2022

Microsoft Modern Auth Email's #2607

Open

kkimurak reviewed Jul 28, 2022

View reviewed changes

README.md Outdated Show resolved Hide resolved

README.md Outdated Show resolved Hide resolved

antt1995 force-pushed the master branch 2 times, most recently from 76860a0 to 452ae97 Compare September 3, 2022 17:27

kkimurak mentioned this pull request Sep 11, 2022

Sync upstream configuration #2570

Draft

4 tasks

antt1995 closed this Dec 22, 2022

antt1995 force-pushed the master branch from 4a84c24 to 8e0320a Compare December 22, 2022 12:53

yougotborked mentioned this pull request Mar 24, 2023

Include builtin-kas in docker image provided by @kkimurak #2736

Open

kkimurak mentioned this pull request Oct 24, 2025

Support KAS #3180

Open

1 task

	gitlab_configure_monitoring() {
	echo "Configuring gitlab::monitoring..."

	update_template ${GITLAB_CONFIG} \
	GITLAB_MONITORING_UNICORN_SAMPLER_INTERVAL \
	GITLAB_MONITORING_IP_WHITELIST \
	GITLAB_MONITORING_SIDEKIQ_EXPORTER_ENABLED \
	GITLAB_MONITORING_SIDEKIQ_EXPORTER_ADDRESS \
	GITLAB_MONITORING_SIDEKIQ_EXPORTER_PORT
	}

	configure_gitlab() {
	echo "Configuring gitlab..."
	update_template ${GITLAB_CONFIG} \
	GITLAB_INSTALL_DIR \
	GITLAB_SHELL_INSTALL_DIR \
	GITLAB_DATA_DIR \
	GITLAB_REPOS_DIR \
	GITLAB_DOWNLOADS_DIR \
	GITLAB_SHARED_DIR \
	GITLAB_HOME \
	GITLAB_HOST \
	GITLAB_PORT \
	GITLAB_RELATIVE_URL_ROOT \
	GITLAB_HTTPS \
	GITLAB_SSH_HOST \
	GITLAB_SSH_LISTEN_PORT \
	GITLAB_SSH_PORT \
	GITLAB_SIGNUP_ENABLED \
	GITLAB_IMPERSONATION_ENABLED \
	GITLAB_PROJECTS_LIMIT \
	GITLAB_USERNAME_CHANGE \
	GITLAB_DEFAULT_THEME \
	GITLAB_CREATE_GROUP \
	GITLAB_ISSUE_CLOSING_PATTERN

	gitlab_configure_database
	gitlab_configure_redis
	gitlab_configure_actioncable
	gitlab_configure_secrets
	gitlab_configure_sidekiq
	gitlab_configure_gitaly
	gitlab_configure_monitoring
	gitlab_configure_gitlab_workhorse
	gitlab_configure_relative_url
	gitlab_configure_trusted_proxies
	gitlab_configure_puma
	gitlab_configure_timezone
	gitlab_configure_rack_attack
	gitlab_configure_ci
	gitlab_configure_artifacts
	gitlab_configure_packages
	gitlab_configure_terraform_state
	gitlab_configure_lfs
	gitlab_configure_uploads
	gitlab_configure_mattermost
	gitlab_configure_project_features
	gitlab_configure_mail_delivery
	gitlab_configure_mailroom
	gitlab_configure_oauth
	gitlab_configure_ldap
	gitlab_configure_gravatar
	gitlab_configure_cron_jobs
	gitlab_configure_analytics
	gitlab_configure_backups
	generate_registry_certificates
	gitlab_configure_registry
	gitlab_configure_pages
	gitlab_configure_sentry
	generate_healthcheck_script
	gitlab_configure_content_security_policy

	# remove stale gitlab.socket
	rm -rf ${GITLAB_INSTALL_DIR}/tmp/sockets/gitlab.socket
	}

Conversation

antt1995 commented Jun 30, 2022

Uh oh!

antt1995 commented Jun 30, 2022

Uh oh!

wulfdj commented Jul 1, 2022

Uh oh!

blattersturm commented Jul 2, 2022

Uh oh!

kkimurak left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

antt1995 commented Jul 4, 2022

Uh oh!

kkimurak left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

antt1995 commented Jul 5, 2022

Uh oh!

antt1995 commented Jul 5, 2022

Uh oh!

kkimurak left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

antt1995 commented Jul 13, 2022

Uh oh!

kkimurak commented Jul 13, 2022

Uh oh!

antt1995 commented Jul 13, 2022

Uh oh!

kkimurak commented Jul 13, 2022

Uh oh!

tiehfood commented Jul 26, 2022

Uh oh!

sachilles commented Jul 28, 2022

Uh oh!

kkimurak left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

eslamhossam23 commented Aug 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

antt1995 commented Aug 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eslamhossam23 commented Aug 16, 2022

Uh oh!

tiehfood commented Aug 24, 2022

Uh oh!

kkimurak commented Aug 24, 2022

Uh oh!

sachilles commented Sep 11, 2022

Uh oh!

defp commented Sep 12, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

kkimurak left a comment •

edited

Loading

kkimurak left a comment •

edited

Loading

eslamhossam23 commented Aug 16, 2022 •

edited

Loading

antt1995 commented Aug 16, 2022 •

edited

Loading