Docker Registry Build via Harbor
Build your own Docker registry

1. Background
In the previous article, we have GitLab and GitLab Runner deployed on our server. In the next step, let’s build our own cloud native registry, using Harbor.
Harbor is an open source trusted cloud native registry project that stores, signs, and scans content. Harbor extends the open source Docker Distribution by adding the functionalities usually required by users such as security, identity and management. Having a registry closer to the build and run environment can improve the image transfer efficiency. Harbor supports replication of images between registries, and also offers advanced security features such as user management, access control and activity auditing.
For more details, please refer to Harbor GitHub
2. Premise and Structure
- Assume you have already finished deployment and configuration in GitLab and GitLab Runner
- Harbor will be deployed on a different server to reduce server load (Where the GitLab and GitLab Runner are located)
- To make full use of our server, an extra service will also be deployed on the same server, using Nginx SNI, just like the way we proceed with GitLab and GitLab Runner.
- Although Harbor has its own official docker image and scripts, it DOES NOT supports ARM64 architecture. In this case, we will use Harbor packaged by Bitnami
- The goal is to reuse the 443 port and make minimum modification of original conf file to save our effort when it comes to major version upgrades.
- Overall, the structure will be: (Visitor) - (Harbor server Nginx, https) - (SNI) - (Bitnami Harbor build-in Nginx, local port A) - (Harbor docker, local port B)
3. Harbor Setup
- Create Harbor folder
cd /your/path
mkdir harbor && cd harbor
- Get docker-compose and preset config
curl -LO https://raw.githubusercontent.com/bitnami/containers/main/bitnami/harbor-portal/docker-compose.yml
curl -L https://github.com/bitnami/containers/archive/main.tar.gz | tar xz --strip=2 containers-main/bitnami/harbor-portal && cp -RL harbor-portal/config . && rm -rf harbor-portal
- Modify docker-compose.yml
nano docker-compose.yml
To make it easy to maintain, I set all local volumes under the harbor folder (eg. ./folder:/docker/folder) And I’ll put comments on where you have to modify. Also, please change and match the passwords in the yml file.
version: '2.1'
services:
registry:
image: docker.io/bitnami/harbor-registry:2
environment:
- REGISTRY_HTTP_SECRET=CHANGEME
volumes:
- ./registry_data:/storage
- ./config/registry/:/etc/registry/:ro
registryctl:
image: docker.io/bitnami/harbor-registryctl:2
environment:
- CORE_SECRET=CHANGEME
- JOBSERVICE_SECRET=CHANGEME
- REGISTRY_HTTP_SECRET=CHANGEME
volumes:
- ./registry_data:/storage
- ./config/registry/:/etc/registry/:ro
- ./config/registryctl/config.yml:/etc/registryctl/config.yml:ro
postgresql:
image: docker.io/bitnami/postgresql:13
container_name: harbor-db
environment:
- POSTGRESQL_PASSWORD=bitnami
- POSTGRESQL_DATABASE=registry
volumes:
- ./postgresql_data:/bitnami/postgresql
core:
image: docker.io/bitnami/harbor-core:2
container_name: harbor-core
depends_on:
- registry
- chartmuseum
environment:
- CORE_KEY=change-this-key
- _REDIS_URL_CORE=redis://redis:6379/0
- SYNC_REGISTRY=false
- CHART_CACHE_DRIVER=redis
- _REDIS_URL_REG=redis://redis:6379/1
- PORT=8080
- LOG_LEVEL=info
- EXT_ENDPOINT=https://harbor.example.com # Put your site URL with https here
- DATABASE_TYPE=postgresql
- REGISTRY_CONTROLLER_URL=http://registryctl:8080
- POSTGRESQL_HOST=postgresql
- POSTGRESQL_PORT=5432
- POSTGRESQL_DATABASE=registry
- POSTGRESQL_USERNAME=postgres
- POSTGRESQL_PASSWORD=bitnami
- POSTGRESQL_SSLMODE=disable
- REGISTRY_URL=http://registry:5000
- TOKEN_SERVICE_URL=http://core:8080/service/token
- HARBOR_ADMIN_PASSWORD=bitnami
- CORE_SECRET=CHANGEME
- JOBSERVICE_SECRET=CHANGEME
- ADMIRAL_URL=
- WITH_NOTARY=False
- WITH_CHARTMUSEUM=True
- CHART_REPOSITORY_URL=http://chartmuseum:8080
- CORE_URL=http://core:8080
- JOBSERVICE_URL=http://jobservice:8080
- REGISTRY_STORAGE_PROVIDER_NAME=filesystem
- REGISTRY_CREDENTIAL_USERNAME=harbor_registry_user
- REGISTRY_CREDENTIAL_PASSWORD=harbor_registry_password
- READ_ONLY=false
- RELOAD_KEY=
volumes:
- ./core_data:/data
- ./config/core/app.conf:/etc/core/app.conf:ro
- ./config/core/private_key.pem:/etc/core/private_key.pem:ro
portal:
image: docker.io/bitnami/harbor-portal:2
container_name: harbor-portal
depends_on:
- core
jobservice:
image: docker.io/bitnami/harbor-jobservice:2
container_name: harbor-jobservice
depends_on:
- redis
- core
environment:
- CORE_SECRET=CHANGEME
- JOBSERVICE_SECRET=CHANGEME
- CORE_URL=http://core:8080
- REGISTRY_CONTROLLER_URL=http://registryctl:8080
- REGISTRY_CREDENTIAL_USERNAME=harbor_registry_user
- REGISTRY_CREDENTIAL_PASSWORD=harbor_registry_password
volumes:
- ./jobservice_data:/var/log/jobs
- ./config/jobservice/config.yml:/etc/jobservice/config.yml:ro
redis:
image: docker.io/bitnami/redis:7.0
environment:
# ALLOW_EMPTY_PASSWORD is recommended only for development.
- ALLOW_EMPTY_PASSWORD=yes
harbor-nginx:
image: docker.io/bitnami/nginx:1.23
container_name: nginx
volumes:
- ./config/proxy/nginx.conf:/opt/bitnami/nginx/conf/nginx.conf:ro
ports:
- '127.0.0.1:22222:8080' # To reuse local port 443, let’s point it to a local port here, 22222
depends_on:
- postgresql
- registry
- core
- portal
chartmuseum:
container_name: chartmuseum
image: docker.io/bitnami/chartmuseum:0
environment:
- CACHE=redis
- CACHE_REDIS_ADDR=redis:6379
- CACHE_REDIS_DB=1
- DEPTH=1
- PORT=8080
- STORAGE=local
- STORAGE_LOCAL_ROOTDIR=/bitnami/data
- ALLOW_OVERWRITE=true
- INDEX_LIMIT=0
volumes:
- ./chartmuseum_data:/bitnami/data
volumes:
registry_data:
driver: local
core_data:
driver: local
jobservice_data:
driver: local
postgresql_data:
driver: local
chartmuseum_data:
driver: local
IMPORTANT: For the value of EXT_ENDPOINT
, you have to set it to https here (https://harbor.example.com), because another Nginx will proxy pass through the build-in Nginx. If you leave the original http value here, you will not be able to docker login harbor.example.com
, with error message with “unauthorized: authentication required.”
- Modify Bitnami Harbor build in Nginx conf
cd /your/path/harbor/config/proxy
nano nginx.conf
Since we will use Nginx on our server to take over port 443 with SNI, in this case we will have to comment out the X-Forwarded-Proto
in the conf.
worker_processes auto;
error_log "/opt/bitnami/nginx/logs/error.log";
pid "/opt/bitnami/nginx/tmp/nginx.pid";
events {
worker_connections 1024;
use epoll;
multi_accept on;
}
http {
tcp_nodelay on;
# this is necessary for us to be able to disable request buffering in all cases
proxy_http_version 1.1;
upstream core {
server core:8080;
}
upstream portal {
server portal:8080;
}
log_format timed_combined '$remote_addr - '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" '
'$request_time $upstream_response_time $pipe';
client_body_temp_path "/opt/bitnami/nginx/tmp/client_body" 1 2;
proxy_temp_path "/opt/bitnami/nginx/tmp/proxy" 1 2;
fastcgi_temp_path "/opt/bitnami/nginx/tmp/fastcgi" 1 2;
scgi_temp_path "/opt/bitnami/nginx/tmp/scgi" 1 2;
uwsgi_temp_path "/opt/bitnami/nginx/tmp/uwsgi" 1 2;
server {
listen 8080;
server_tokens off;
# disable any limits to avoid HTTP 413 for large image uploads
client_max_body_size 0;
# costumized location config file can place to /opt/bitnami/nginx/conf with prefix harbor.http. and suffix .conf
include /opt/bitnami/conf/nginx/conf.d/harbor.http.*.conf;
location / {
proxy_pass http://portal/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
# proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
location /c/ {
proxy_pass http://core/c/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
# proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
location /api/ {
proxy_pass http://core/api/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
# proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
location /chartrepo/ {
proxy_pass http://core/chartrepo/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
# proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
location /v1/ {
return 404;
}
location /v2/ {
proxy_pass http://core/v2/;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
# proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
location /service/ {
proxy_pass http://core/service/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
# proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
location /service/notifications {
return 404;
}
}
}
Note: There are six places to be commented out, including location /
, location /c/
, location /api/
, location /chartrepo/
, location /v2/
, location /service/
.
- Bring up the container
cd /your/path/harbor
docker-compose up -d
4. BookStack Deployment
This is just an example of a second application to be setup.
cd /your/path
mkdir bookstack && cd bookstack
nano docker-compose.yml
version: "3.8"
services:
bookstack:
image: ghcr.io/linuxserver/bookstack
container_name: bookstack
environment:
- PUID=1000
- PGID=1000
- APP_URL=https://bookstack.example.com
- DB_HOST=bookstack_db
- DB_USER=bookstack
- DB_PASS=password
- DB_DATABASE=bookstackapp
volumes:
- ./data:/config
ports:
- 127.0.0.1:22223:80 # We have Harbor application pointed docker internal port from 8080 to local port 22222. So for this second app, we point BookStack from docker internal port from 80 to local port 22223.
restart: unless-stopped
depends_on:
- bookstack_db
bookstack_db:
image: ghcr.io/linuxserver/mariadb
container_name: bookstack_db
environment:
- PUID=1000
- PGID=1000
- MYSQL_ROOT_PASSWORD=password
- TZ=Europe/London
- MYSQL_DATABASE=bookstackapp
- MYSQL_USER=bookstack
- MYSQL_PASSWORD=password
volumes:
- ./db:/config
restart: unless-stopped
- Bring up the container
docker-compose up -d
5. Nginx Configuration
This is an “outer” Nginx to cover both Bitnami Harbor built-in Nginx and BookStack
nano /etc/nginx/nginx.conf
- Add a
stream
block at the end
stream {
map $ssl_preread_server_name $example_multi {
harbor.henrywithu.com harbor;
bookstack.henrywithu.com bookstack;
}
upstream harbor {
server 127.0.0.1:30001; # This port will be linked to local port 22222, as Harbor Registry service
}
upstream bookstack {
server 127.0.0.1:30002; # This port will be linked to local port 22223 as BookStack service
}
server {
listen 443 reuseport;
listen [::]:443 reuseport;
proxy_pass $example_multi;
ssl_preread on;
#proxy_protocol on;
}
}
- harbor.conf
nano /etc/nginx/conf.d/harbor.conf
server {
listen 127.0.0.1:30001 ssl http2;
ssl_certificate /your/path/fullchain.pem;
ssl_certificate_key /your/path/privkey.pem;
#ssl_dhparam /your/path/dhparams.pem; #optional
#Authenticated Origin Pull is optional. Please refer to https://developers.cloudflare.com/ssl/origin/authenticated-origin-pull/
#ssl_client_certificate /etc/ssl/origin-pull-ca.pem;
#ssl_verify_client on;
ssl_session_timeout 1d;
ssl_session_cache shared:MozSSL:10m;
ssl_session_tickets off;
proxy_read_timeout 300;
proxy_connect_timeout 300;
proxy_send_timeout 300;
client_max_body_size 10G; # define your max_upload_file_size
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
ssl_prefer_server_ciphers off;
server_name harbor.example.com;
add_header Content-Security-Policy upgrade-insecure-requests;
location / {
proxy_redirect off;
proxy_pass http://127.0.0.1:22222; # port 22222 matches the built-in Nginx service port
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
- bookstack.conf
nano /etc/nginx/conf.d/bookstack.conf
server {
listen 127.0.0.1:30002 ssl http2;
ssl_certificate /your/path/fullchain.pem;
ssl_certificate_key /your/path/privkey.pem;
#ssl_dhparam /your/path/dhparams.pem; #optional
#Authenticated Origin Pull is optional. Please refer to https://developers.cloudflare.com/ssl/origin/authenticated-origin-pull/
#ssl_client_certificate /etc/ssl/origin-pull-ca.pem;
#ssl_verify_client on;
ssl_session_timeout 1d;
ssl_session_cache shared:MozSSL:10m;
ssl_session_tickets off;
proxy_read_timeout 300;
proxy_connect_timeout 300;
proxy_send_timeout 300;
client_max_body_size 10G; # define your max_upload_file_size
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
ssl_prefer_server_ciphers off;
server_name bookstack.example.com;
add_header Content-Security-Policy upgrade-insecure-requests;
location / {
proxy_redirect off;
proxy_pass http://127.0.0.1:22223; # port 22223 matches the built-in Nginx service port
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
- httpsredirect.conf
nano /etc/nginx/conf.d/httpsredirect.conf
server {
listen 80;
server_name bookstack.example.com;
if ($host = bookstack.example.com) {
return 301 https://$host$request_uri;
}
server_name harbor.example.com;
if ($host = harbor.example.com) {
return 301 https://$host$request_uri;
}
return 404;
}
- Restart Nginx
nginx -t
systemctl restart nginx
6. Harbor Post-Configuration
Now you can login to Harbor in your browser. The initial credentials can be found in your Harbor docker-compose.yml, which is admin/bitnami
. Once you successfully logged in, please change the password immediately. Or you can create a new user with an admin role and then remove the default admin user.
- Assume you created a user and created a new project named
test
Now let’s try to push an docker image to the Harbor Registry. - Login another server
# On a non-Harbor server
docker login harbor.example.com
- You will be prompted to input username and password
USERNAME=<username_you_createed_in_Harbor>
PASSWORD=<password_you_set_in_Harbor>
- The successful output will be:
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
- Now check your local docker images
docker images
- For example, let’s choose
NODE:19-alpine
REPOSITORY | TAG | IMAGE ID | CREATED | SIZE |
---|---|---|---|---|
NODE | 19-alpine | 31724419e3ea | 10 days ago | 177MB |
- Put a tag on your local docker image with the remote Harbor Registry (Project name test has been created earlier)
docker tag NODE:19-alpine harbor.example.com/test/NODE:19-alpine
- Push the docker image to Harbor Registry
docker push harbor.example.com/test/NODE:19-alpine
Now you can check the image in Harbor Registry from web.
For general docker commands, please refer to: The base command for the Docker
Enjoy
Copyright statement: Unless otherwise stated, all articles on this blog adopt the CC BY-NC-SA 4.0 license agreement. For non-commercial reprints and citations, please indicate the author: Henry, and original article URL. For commercial reprints, please contact the author for authorization.