Desplegar app en Okteto con Gitlab CI III
Daniel J. Saldaña- 11 de junio de 2023
Continuamos con la tercera parte de esta serie de tutoriales de Gitlab CI. En este tutorial nos enfocaremos en realizar un refactor para cambiar el flujo de los pipelines y añadir unos nuevos jobs y steps.
Aquí os dejo los objetivos que abordaremos:
- Añadimos un job para ejecutar test de integración, “en este tutorial solo usaremos unos test de Cypress, en un futuro añadiremos unos test unitarios”.
- Integrar Cypress para validar test e2e.
- Analizaremos con Trivy el contenido del repositorio para detectar posibles usuarios y contraseñas
- Analizaremos la imagen con Trivy para detectar el tipo de licencia que dispone nuestro software.
- Redefinir flujo de CI, se añade el entorno de Staging.
Como siempre aquí os dejo el repositorio que utilizaremos.
Daniel J. Saldaña / app-okteto · GitLab
Recordamos que el tutorial lo podrá encontrar en la rama main, ya que verá que actualmente existen más ramas, las cuales están enfocadas en los siguientes labs.
stages: - test - scanning - review - build - staging - production
variables: IMAGE_NODE: node:latest IMAGE_GITGUARDIAN: gitguardian/ggshield:latest IMAGE_SONAR: sonarsource/sonar-scanner-cli:latest IMAGE_OKTETO: okteto/okteto:1.13.4 IMAGE_CYPRESS: cypress/browsers:node18.12.0-chrome107 IMAGE_DOCKER: docker:stable IMAGE_KANIKO: gcr.io/kaniko-project/executor:v1.9.0-debug IMAGE_TRIVY: docker.io/aquasec/trivy:latest IMAGE_HELM: alpine/helm RELEASE:
cache: key: ${CI_COMMIT_REF_SLUG} paths: - node_modules/ - .npm/
.charts_rules: rules: - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS when: never - if: $CI_COMMIT_BRANCH != 'main' when: never - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops' when: always - if: $CI_COMMIT_BRANCH != 'devops' changes: - charts/* when: always.code_rules: rules: - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS when: never - if: $CI_COMMIT_BRANCH != 'main' when: never - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops' when: always - if: $CI_COMMIT_BRANCH != 'devops' changes: - src/* - index.html - package* - tsconfig* - vite.config.ts - Dockerfile when: always.code_rules_stop: rules: - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS when: never - if: $CI_COMMIT_BRANCH != 'main' when: never - if: $CI_COMMIT_BRANCH != 'devops' changes: - src/* - index.html - package* - tsconfig* - vite.config.ts - Dockerfile when: delayed start_in: 30 minutes - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops' when: always.tests_rules: rules: - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS when: never - if: $CI_COMMIT_BRANCH != 'main' when: never - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops' when: always - if: $CI_COMMIT_BRANCH != 'devops' changes: - cypress/* when: always.tests_rules_stop: rules: - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS when: never - if: $CI_COMMIT_BRANCH != 'main' when: never - if: $CI_COMMIT_BRANCH != 'devops' changes: - cypress/* when: delayed start_in: 30 minutes - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops' when: always.devops_rules: rules: - if: $CI_COMMIT_BRANCH == 'devops' && $RELEASE == 'yes' when: always - if: $CI_COMMIT_BRANCH != 'main' when: never - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'main' when: always.devops_rules_stop: rules: - if: $CI_COMMIT_BRANCH == 'devops' && $RELEASE == 'yes' when: delayed start_in: 30 minutes - if: $CI_COMMIT_BRANCH != 'main' when: never - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'main' when: always.go_to_production_rules: rules: - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'main'.production_rules: rules: - if: $CI_COMMIT_BRANCH == 'main'
.deploy: image: name: $IMAGE_HELM entrypoint: [''] script: - export KUBECONFIG=${ENV_KUBECONFIG}:${KUBECONFIG:-$HOME/.kube/config} - sed -i 's/_CI_PROJECT_TITLE/'"${CI_PROJECT_TITLE}"'/' "charts/templates/NOTES.txt" - sed -i 's/_VA_ENV/'""'/' "charts/templates/NOTES.txt" - sed -i 's/_CI_COMMIT_REF_SLUG/'""'/' "charts/templates/NOTES.txt" - sed -i 's/_OKTETO_USERNAME/'"-${OKTETO_USERNAME}"'/' "charts/templates/NOTES.txt" - helm upgrade $CI_PROJECT_NAME ./charts --values=./charts/values.yaml --namespace ${VA_ENV}-${OKTETO_USERNAME} --install --atomic
kubelinter: stage: test image: $IMAGE_DOCKER services: - docker:dind script: - docker run -v $PWD/charts:/dir -v $PWD/lint/config.yaml:/etc/config.yaml stackrox/kube-linter lint /dir --config /etc/config.yaml rules: - !reference [.charts_rules, rules] - !reference [.devops_rules, rules]
sonarcloud-check: stage: test image: name: $IMAGE_SONAR variables: SONAR_USER_HOME: '${CI_PROJECT_DIR}/.sonar' GIT_DEPTH: '0' cache: key: '${CI_JOB_NAME}' paths: - .sonar/cache script: - sonar-scanner rules: - !reference [.code_rules, rules] - !reference [.devops_rules, rules]
integration-testing: stage: test image: $IMAGE_CYPRESS script: - npm ci - npm run integration-testing & - npx cypress run --env baseUrl=http://localhost:3000 --config-file cypress/integration-testing.config.ts --browser chrome --reporter junit --reporter-options "mochaFile=results/test-integration-[hash].xml" artifacts: when: always paths: - results/test-integration-*.xml - cypress/videos/**/*.mp4 - cypress/screenshots/**/*.png reports: junit: results/test-integration-*.xml expire_in: 1 day rules: - !reference [.tests_rules, rules] - !reference [.devops_rules, rules]
gitguardian: stage: scanning image: name: $IMAGE_GITGUARDIAN script: - ggshield secret scan ci rules: - !reference [.code_rules, rules] - !reference [.tests_rules, rules] - !reference [.devops_rules, rules]
type-detection: stage: scanning image: name: $IMAGE_TRIVY entrypoint: [''] script: - trivy conf --severity HIGH,CRITICAL . cache: paths: - .trivycache/ artifacts: expire_in: 1 week name: 'gl-type-detection-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json' paths: - ${CI_PROJECT_DIR} rules: - !reference [.code_rules, rules] - !reference [.devops_rules, rules]
deploy-review: stage: review image: $IMAGE_OKTETO variables: APP: $CI_COMMIT_REF_SLUG VA_ENV: review script: - okteto preview deploy $VA_ENV-$CI_COMMIT_REF_SLUG-$OKTETO_USERNAME --scope personal --branch $CI_COMMIT_REF_NAME --repository $CI_REPOSITORY_URL --wait environment: name: review/$CI_COMMIT_REF_SLUG url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${CI_COMMIT_REF_SLUG}-${OKTETO_USERNAME}.cloud.okteto.net on_stop: stop-review rules: - !reference [.code_rules, rules] - !reference [.tests_rules, rules]
e2e: stage: review image: $IMAGE_CYPRESS needs: ['deploy-review'] variables: APP: $CI_COMMIT_REF_SLUG VA_ENV: review script: - npm ci - npx cypress run --env baseUrl=https://${CI_PROJECT_TITLE}-${VA_ENV}-${CI_COMMIT_REF_SLUG}-${OKTETO_USERNAME}.cloud.okteto.net --config-file cypress/e2e.config.ts --browser chrome --reporter junit --reporter-options "mochaFile=results/test-e2e-[hash].xml" artifacts: when: always paths: - results/test-e2e-*.xml - cypress/videos/**/*.mp4 - cypress/screenshots/**/*.png reports: junit: results/test-e2e-*.xml expire_in: 1 day rules: - !reference [.code_rules, rules] - !reference [.tests_rules, rules]
stop-review: stage: review image: $IMAGE_OKTETO needs: ['e2e'] variables: APP: $CI_COMMIT_REF_SLUG VA_ENV: review environment: name: review/$CI_COMMIT_REF_SLUG action: stop script: - okteto preview destroy $VA_ENV-$CI_COMMIT_REF_SLUG-$OKTETO_USERNAME rules: - !reference [.code_rules_stop, rules] - !reference [.tests_rules_stop, rules]
build-docker: stage: build image: $IMAGE_DOCKER services: - docker:dind variables: REGISTRY_HOST: docker.io DOCKER_HUB_IMAGE: app-okteto REGISTRY_IMAGE: index.docker.io/danieljesussp/app-okteto script: - docker login -u "$DOCKER_HUB_USER" -p "$DOCKER_HUB_PASSWORD" $REGISTRY_HOST - > docker build --no-cache --tag $REGISTRY_IMAGE:$CI_COMMIT_SHA . - docker push $REGISTRY_IMAGE:$CI_COMMIT_SHA rules: - !reference [.devops_rules, rules]
build-kaniko: stage: build image: name: $IMAGE_KANIKO entrypoint: [''] variables: DOCKER_HUB_REGISTRY: registry.gitlab.com script: - /kaniko/executor --context ${CI_PROJECT_DIR} --dockerfile ${CI_PROJECT_DIR}/Dockerfile --destination ${DOCKER_HUB_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHA} rules: - !reference [.devops_rules, rules]
container_scanning: stage: build image: name: $IMAGE_TRIVY entrypoint: [''] needs: - job: build-kaniko variables: DOCKER_HUB_REGISTRY: registry.gitlab.com FULL_IMAGE_NAME: ${DOCKER_HUB_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHA} script: - trivy --version - trivy image --clear-cache - trivy image --exit-code 0 --cache-dir .trivycache/ --no-progress --security-checks vuln "$FULL_IMAGE_NAME" - trivy image --exit-code 0 --cache-dir .trivycache/ --severity CRITICAL --no-progress --security-checks vuln "$FULL_IMAGE_NAME" cache: paths: - .trivycache/ artifacts: expire_in: 1 week name: 'gl-container-scanning-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json' paths: - ${CI_PROJECT_DIR} rules: - !reference [.devops_rules, rules]
checks-license: stage: build image: name: $IMAGE_TRIVY entrypoint: [''] needs: - job: build-kaniko variables: DOCKER_HUB_REGISTRY: registry.gitlab.com FULL_IMAGE_NAME: ${DOCKER_HUB_REGISTRY}/${CI_PROJECT_PATH}:latest script: - trivy --version - trivy image --clear-cache - trivy image --security-checks license --severity UNKNOWN,HIGH,CRITICAL --license-full "$FULL_IMAGE_NAME" cache: paths: - .trivycache/ artifacts: expire_in: 1 week name: 'gl-license-scanning-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json' paths: - ${CI_PROJECT_DIR} rules: - !reference [.devops_rules, rules]
deploy-staging: stage: staging extends: .deploy variables: VA_ENV: staging environment: name: staging url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${OKTETO_USERNAME}.cloud.okteto.net rules: - !reference [.devops_rules, rules]
uninstall-staging: stage: staging needs: ['deploy-staging'] image: name: $IMAGE_HELM entrypoint: [''] variables: VA_ENV: staging environment: name: staging url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${OKTETO_USERNAME}.cloud.okteto.net script: - export KUBECONFIG=${ENV_KUBECONFIG}:${KUBECONFIG:-$HOME/.kube/config} - helm uninstall $CI_PROJECT_NAME --namespace ${VA_ENV}-${OKTETO_USERNAME} --wait rules: - !reference [.devops_rules_stop, rules]
deploy-production: stage: production extends: .deploy variables: VA_ENV: production environment: name: production url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${OKTETO_USERNAME}.cloud.okteto.net rules: - !reference [.production_rules, rules]Ahora vamos a explicar los cambios que hemos introducido en este nuevo labs.
integration-testing: stage: test image: $IMAGE_CYPRESS script: - npm ci - npm run integration-testing & - npx cypress run --env baseUrl=http://localhost:3000 --config-file cypress/integration-testing.config.ts --browser chrome --reporter junit --reporter-options "mochaFile=results/test-integration-[hash].xml" artifacts: when: always paths: - results/test-integration-*.xml - cypress/videos/**/*.mp4 - cypress/screenshots/**/*.png reports: junit: results/test-integration-*.xml expire_in: 1 day rules: - !reference [.tests_rules, rules] - !reference [.devops_rules, rules]Lo primero es que este job se desarrollara en el próximo y seguramente último laboratorio, donde integraremos test unitarios. Pero nos vale para explicar la esencia de este job, el cual el objetivo mismo será realizar test unitarios.
type-detection: stage: scanning image: name: $IMAGE_TRIVY entrypoint: [''] script: - trivy conf --severity HIGH,CRITICAL . cache: paths: - .trivycache/ artifacts: expire_in: 1 week name: 'gl-type-detection-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json' paths: - ${CI_PROJECT_DIR} rules: - !reference [.code_rules, rules] - !reference [.devops_rules, rules]Ahora hemos agregado otro job para detectar contraseñas o token, esta solución es parecida a GitGuardian, con la diferencia de que esta solución está basada en Trivy y es totalmente gratuita.
e2e: stage: review image: $IMAGE_CYPRESS needs: ['deploy-review'] variables: APP: $CI_COMMIT_REF_SLUG VA_ENV: review script: - npm ci - npx cypress run --env baseUrl=https://${CI_PROJECT_TITLE}-${VA_ENV}-${CI_COMMIT_REF_SLUG}-${OKTETO_USERNAME}.cloud.okteto.net --config-file cypress/e2e.config.ts --browser chrome --reporter junit --reporter-options "mochaFile=results/test-e2e-[hash].xml" artifacts: when: always paths: - results/test-e2e-*.xml - cypress/videos/**/*.mp4 - cypress/screenshots/**/*.png reports: junit: results/test-e2e-*.xml expire_in: 1 day rules: - !reference [.code_rules, rules] - !reference [.tests_rules, rules]Este job se ejecuta justo después de desplegar nuestra aplicación y nos permite probar nuestra aplicación sin necesidades de ejecutar pruebas manuales. Tal como funciona el pipeline actualmente, vinculamos el reporte en Gitlab CI y guardamos evidencias y videos en el pipeline.
checks-license: stage: build image: name: $IMAGE_TRIVY entrypoint: [''] needs: - job: build-kaniko variables: DOCKER_HUB_REGISTRY: registry.gitlab.com FULL_IMAGE_NAME: ${DOCKER_HUB_REGISTRY}/${CI_PROJECT_PATH}:latest script: - trivy --version - trivy image --clear-cache - trivy image --security-checks license --severity UNKNOWN,HIGH,CRITICAL --license-full "$FULL_IMAGE_NAME" cache: paths: - .trivycache/ artifacts: expire_in: 1 week name: 'gl-license-scanning-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json' paths: - ${CI_PROJECT_DIR} rules: - !reference [.devops_rules, rules]Este job lo considero bastante interesante y creo que cada vez existe una mayor necesidades por parte de las empresas de saber qué tipo de licenciamiento tiene sus productos de software. En este paso, analizamos y generamos un reporte de tipos de licenciamiento que tiene nuestra imagen.
deploy-staging: stage: staging extends: .deploy variables: VA_ENV: staging environment: name: staging url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${OKTETO_USERNAME}.cloud.okteto.net rules: - !reference [.devops_rules, rules]Ahora hablaremos de unos de los cambios en el flujo del CI. En este lab, hemos creado el entorno de Staging, para ello hemos establecido un flujo de trabajo. Cada programamos creará una rama de trabajo con la tarea que vaya a desarrollar. Una vez finalizada la tarea, se abrirá una pull request de la rama de su tarea a la rama devops. Una vez que tenemos todo preparado para ir a producción, abriremos otra pull request de la rama devops a main. Cuando se mergea a main, es cuando se realizara el despliegue a producción.
