db-website/docs/Walkthrough.md

Walkthrough

Summary

• Overview
• Pipeline definition
• Pipeline concurrency
• Triggers
• Steps
  • Replace hosts and user variables
  • Install frontend dependencies and build it
  • Run the frontend
  • Test frontend
  • Check ansible syntax
  • Apply ansible playbook
    • Playbook definition
    • Delete old spt-items-api
    • Copy the project
    • Copy PHP env file
    • Get JavaScript chunks name
    • Get file names from find output
    • Copy app.blade.php file
    • Download and install composer dependencies

Overview

• The project is split between the frontend and the backend
  • the backend is a submodule located in api that points towards https://dev.sp-tarkov.com/Rev/spt-items-api.git
  • the frontend is a submodule located in frontend that points towards https://dev.sp-tarkov.com/shirito/item-finder-website-frontend.git
• There are two Ansible pipelines
  • A docker pipeline drone-docker.yml
  • A kubernetes pipeline drone-kubernetes.yml
• All ansible playbook files are located in .ansible
• The documentation is located in documentation

Pipeline definition

kind: pipeline
type: kubernetes
name: default

The pipeline is defined either as Docker or Kubernetes depending on .drone-docker.yml or .drone-kubernetes.yml. The name is set as default.

Pipeline Concurrency

concurrency:
  limit: 1

The pipeline is set to only one build at a time (every subsequent build with be pending).

Triggers

trigger:
  event:
    - push

The pipeline is run on every push We want to check that every development on development branch is correct and deploy automatically when merged in master/main.

Steps

Replace hosts and user variables

- name: replace hosts and user variables
  image: ubuntu:impish
  environment:
    SPT_ITEMS_HOSTNAME:
      from_secret: spt_items_hostname
    DEPLOY_HOSTNAME:
      from_secret: deploy_hostname
    DEPLOY_USER:
      from_secret: deploy_username
    DEPLOY_PRIVATE_KEY:
      from_secret: deploy_ssh_key
    DEPLOY_SSH_KEY_PASSPHRASE:
      from_secret: deploy_ssh_key_passphrase
  commands:
    - sed -i 's/{{ SPT_ITEMS_HOSTNAME }}/'"$SPT_ITEMS_HOSTNAME"'/g' ./frontend/.env.example
    - mv ./frontend/.env.example ./frontend/.env
    - echo "$DEPLOY_PRIVATE_KEY" > private.key && chmod 600 private.key
    - sed -i 's/{{ DEPLOY_HOSTNAME }}/'"$DEPLOY_HOSTNAME"'/g' ./.ansible-items/inventory
    - sed -i 's/{{ DEPLOY_SSH_KEY_PASSPHRASE }}/'"$DEPLOY_SSH_KEY_PASSPHRASE"'/g' ./.ansible-items/inventory
    - sed -i 's/{{ DEPLOY_USER }}/'"$DEPLOY_USER"'/g' ./.ansible-items/inventory

Executed on every push.
The following environment variables are injected using Drone secrets: * SPT_ITEMS_HOSTNAME is used by the frontend to call the backend. * DEPLOY_HOSTNAME is used by Ansible to connect to the remote server via SSH. * DEPLOY_USER is used by Ansible to connect to the remote server via SSH. * DEPLOY_PRIVATE_KEY is the SSH key used to connect to the remote server via SSH * DEPLOY_SSH_KEY_PASSPHRASE is the SSH key passphrase Using sed makes temporary changes in the container/pod instead of commiting secrets in the repo in plain text.
The changes are never pushed and are discarded when the container/pod is terminated.

Install frontend dependencies and build it

- name: install dependencies and build frontend
  image: node:lts-alpine3.14
  commands:
    - node -v
    - npm -v
    - yarn --version
    - yarn --cwd ./frontend install
    - yarn --cwd ./frontend build --pure-lockfile
    - rm -rf ./api/public/static/*
    - cp -r ./frontend/build/* ./api/public
    - rm ./api/public/index.html
  depends_on:
    - replace hosts and user variables

Executed on every push.
Since the PHP backend serves the ReactJS frontend, the former is built and moved in the latter. Notes:

• yarn --cwd <path> <command> executes the command in the specified file
• rm -rf ./api/public/static/* deletes the static files to make sure no old JavaScript files remain
• rm ./api/public/invdex.html ReactJS is bundled with a index.html. It is discarded to use instead.

Run the frontend

- name: frontend
  image: nginx:1.21.4-alpine
  commands:
    - cp -r ./frontend/build/* /usr/share/nginx/html
    - cp ./frontend/src/cypress/nginx_config/default.conf /etc/nginx/conf.d/default.conf
    - nginx -g "daemon off;"
  detach: true
  depends_on:
    - install dependencies and build frontend

Copies the frontend and the nginx conf in the container to be able to test it. The frontend is run and detach is specified so the End-to-End tests (using cypress) can run on it

Test frontend

- name: test frontend
  image: cypress/browsers:node16.5.0-chrome94-ff93
  commands:
    - node -v
    - npm -v
    - yarn --version
    - yarn --cwd ./frontend cy:ci
  depends_on:
    - install dependencies and build frontend

Run frontend tests using Cypress

Check ansible syntax

- name: check ansible syntax
  image: plugins/ansible:3
  settings:
    playbook: ./.ansible-items/playbook.yml
    inventory: ./.ansible-items/inventory
    galaxy: ./.ansible-items/requirements.yml
    syntax_check: true
  when:
    branch:
      - development

Executed on every push.
Check the Ansible syntax in playbook.yml, inventory and requirements.yml. The check is executed on every push since we want to detect any error before validating the build using the promotion.

Apply ansible playbook

- name: apply ansible playbook
  image: plugins/ansible:3
  settings:
    playbook: ./.ansible-items/playbook.yml
    inventory: ./.ansible-items/inventory
    galaxy: ./.ansible-items/requirements.yml
    timeout: 60
    verbose: 2
  environment:
    SPT_ITEMS_HOSTNAME:
      from_secret: spt_items_hostname
    DEPLOY_HOSTNAME:
      from_secret: deploy_hostname
    DEPLOY_USER:
      from_secret: deploy_username
    DEPLOY_USER_GROUP:
      from_secret: deploy_user_group
    SPT_ITEMS_PATH:
      from_secret: deploy_path
  depends_on:
    - test frontend
  when:
    branch:
      - master
      - main

Executed only on promotion to production.
This step actually deploys to the server.
This step is idempotent.
The following environment variables are injected using Drone secrets: * SPT_ITEMS_HOSTNAME is used by the PHP env file. * DEPLOY_HOSTNAME is used to connect to the remote server via SSH. * DEPLOY_USER is used to connect to the remote server via SSH. * DEPLOY_USER_GROUP is the user group, used to give read/write/execute permissions to the whole group. It must be the same as Nginx's user. * SPT_ITEMS_PATH is the path on the remote server where the files will be copyed to.

Playbook definition

hosts: sptarkov

Uses the host defined in inventory. Remember, the step Replace hosts and user variables already replaced the variables at this point.

Delete old spt-items-api

- name: Delete spt-items-api before adding everything again
  file:
    state: absent
    path: "{{ lookup('env', 'SPT_ITEMS_PATH') }}"

Since the copy does not override the folder, this step takes care of it.
SPT_ITEMS_PATH is injected in the environments properties (see Apply ansible playbook)

Copy the project

- name: Copy the project
  copy:
    src: ../api/
    dest: "{{ lookup('env', 'SPT_ITEMS_PATH') }}"

Copies the whole project (frontend and backend) from the api folder into the server.

Copy PHP env file

- name: Copy PHP .env file
  template:
    src: ./templates/.php-env.j2
    dest: "{{ lookup('env', 'SPT_ITEMS_PATH') }}/.env"

Uses Jinja2 to resolve the template for the PHP .env file.
SPT_ITEMS_PATH is injected in the environments properties (see Apply ansible playbook)
SPT_ITEMS_HOSTNAME is injected in the environments properties (see Apply ansible playbook)

Get JavaScript chunks name

- name: Get Chunk 2 name
  shell:
    cmd: find "{{ lookup('env', 'SPT_ITEMS_PATH') }}" -type f -name "*chunk.js" -printf "%f\n"
  register: find_output

Prepare a find of all JavaScript chunk files for the app.blade.php.j2 template.
SPT_ITEMS_PATH is injected in the environments properties (see Apply ansible playbook)

Get file names from find output

- name: Get file names from find output
  set_fact:
    chunk_list: "{{ find_output['stdout'].split('\n') }}"

Splits the string containing the list of all JavaScript chunk files for the app.blade.php.j2 template.

Copy app.blade.php file

- name: Copy app.blade.php file
  template:
    src: ./templates/app.blade.php.j2
    dest: "{{ lookup('env', 'SPT_ITEMS_PATH') }}/resources/views/app.blade.php"

Uses Jinja2 to resolve the template for the PHP app.blade.php file.
SPT_ITEMS_PATH is injected in the environments properties (see Apply ansible playbook).

Download and install composer dependencies

- name: Download and installs all composer libs and dependencies
  community.general.composer:
    command: install
    working_dir: "{{ lookup('env', 'SPT_ITEMS_PATH') }}"

Reset files permissions

- name: Reset files permissions
  file:
    path: "{{ lookup('env', 'SPT_ITEMS_PATH') }}"
    owner: "{{ lookup('env', 'DEPLOY_USER') }}"
    group: "{{ lookup('env', 'DEPLOY_USER_GROUP') }}"
    mode: 0774
    recurse: yes

Permissions 0644: * user: read/write/execute * group: read/write/execute (for Nginx to execute the php as well as write in the Laravel logs) * other: read SPT_ITEMS_PATH is injected in the environments properties (see Apply ansible playbook).

Initialize database

- name: Initialize database
  uri:
    url: "https://{{ lookup('env', 'SPT_ITEMS_HOSTNAME') }}/api/refresh"
    method: GET
    status_code: [200, 204]
    timeout: 60

The call to /api/refresh fetches the data from AKI Server repository, development branch.