2021年11月28日 星期日

[GitHub] Github Actions - Workflow dependencies

 Github Action   Workflow   Dependency 

 

Introduction


 

The jobs in a GitHub Actions: workflow by default run in parallel at the same time.

To run a job only when another job has completed, we can use needs keyword as following,

 

(The sample manifest comes from GitHub Docs: Creating dependent jobs)

jobs:
  setup:
    runs-on: ubuntu-latest
    steps:
      - run: ./setup_server.sh
  build:
    needs: setup
    runs-on: ubuntu-latest
    steps:
      - run: ./build_server.sh
  test:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - run: ./test_server.sh

 

However, it needs some tricks on workflows dependencies, let us see the following example.

 

 

Implement


Scenario

I am going to create 2 workflows:

1.  Publish Docker Image

2.  Run Docker Container (to execute some commands)

 

The second workflow depends on the result of the first one.

We will run the container after the Docker image publish successfully. In other words, if publishing fails, we won’t run the container.

 

Constraints

 

1.  Currently (2021-11) GitHub Actions only supports putting the second workflow (that will be triggered by first one) on the “default branch”.

2.  To get the previous workflow’s state, use the value of github.event.workflow_run.conclusion.

 

 

Workflow: Publish Docker Image

 

This is the first workflow and it is not what we will focus in the article, see [GitHub] Github Actions - Publish Docker images for more details.

 

publish_docker_image.yml

---
name: Publish Docker Image
on:
  push:
    branches: [ master ]
jobs:
  push_images_to_acr:
    name: Push images to ACR
    runs-on: ubuntu-18.04
    steps:
      - name: Checkout repo
        uses: actions/checkout@v2
      - name: Login to ACR
        uses: azure/docker-login@v1
        with:
          login-server: ${{ secrets.ACR_REGISTRY }}
          username: ${{ secrets.ACR_USERNAME }}
          password: ${{ secrets.ACR_PASSWORD }}
      - name: Build and push
        id: build_publish
        uses: docker/build-push-action@v2
        with:
          # context: .
          file: ./docker/dockerfile
          push: true
          tags: ${{ secrets.ACR_REGISTRY }}/my-demo:latest

 

 

 

Workflow: Run Docker Container

The second workflow will need to watch the if the dependent workflow has been closed by

on:
  workflow_run:
    workflows:
      - The dependent workflow name
    types:
      - completed

 

run_docker_constainer.yml


---
name: Run Docker Container
on:
  workflow_run:
    workflows:
      - Publish Docker Image
    types:
      - completed
jobs:
  run_docker_container:
    name: Run
    if: ${{ github.event.workflow_run.conclusion == 'success' }}
    runs-on: ubuntu-18.04
    steps:
      - name: Run
        uses: addnab/docker-run-action@v3
        with:
          username: ${{ secrets.ACR_USERNAME }}
          password: ${{ secrets.ACR_PASSWORD }}
          registry: ${{ secrets.ACR_REGISTRY }}
          image: ${{ secrets.ACR_REGISTRY }}/my-demo:latest
          # options:
          run: |
            echo "The container is running."

 

Like we mentioned in Constraints, we have to enable the workflow (put the manifest file) in the default branch, e.q. master or main.

Notice that the above manifest used addnab/docker-run-action as the “Docker Run Action” Or we can use the default Action like this,

---
name: Run Docker Container
on:
  workflow_run:
    workflows:
      - Publish Docker Image
    types:
      - completed
jobs:
  run_docker_container:
    name: Run
    if: ${{ github.event.workflow_run.conclusion == 'success' }}
    runs-on: ubuntu-18.04
    container: karatejb/my-demo:latest
    steps:
      - name: Dcoker run
        run: |
          echo "The container is running."

 

 

Reference


GitHub Docs: workflow_run

addnab/docker-run-action

Using Docker Containers In Jobs - GitHub Actions

Workflow_run not working as expected

How to use the GitHub Actions `workflow_run` event?

actions/runner: Secrets cannot be used to condition job runs #520

Pass Github secrets to a docker github action

Workflow_run completed event triggered by failed workflow

 

 

 

 

 

 

 

 

2021年11月25日 星期四

[Notes] 2021 DevOpsDays Taipei

 DevOps   iThome  

下半年因為轉職,工作上忙碌了不少,還好當天可以排開其他會議,順利地參加今年的DevOpsDays




TSMC DevOps Journey

台積電也在做數位轉型;可以想像在這種大型跟根深蒂固的文化下,要推廣DevOps文化一定是相對更困難的,這邊有提到一個重點是TSMC聘了一位矽谷回來的CIO,是個很正確及聰明的作法。從上做改革及轉型,會比從下來的更有力和快速。

 

談假設思維之下的開發者體驗

  • DX: Developer Experience
  • 新人犯錯,誰的責任? 答案是整個團隊。
  • 良好的DX,是透過團隊持續的協作、回饋和經驗傳承。
  • 進行任務,需要退一步去看見全貌。
  • 假設思維
    • 頻繁低去做假設,並驗證它。
    • 聆聽在該項目有經驗的人的意見,無論他是資深或資淺。
    • 做出最接近目標的假設。
    • 任務一定要訂出Milestone,而Milestone是建立在假設之上。
  • 任何事情在完成到一定程度之後,會遇到"效率的邊界"(也就是即使在施力,它根本不會動),這時候可以再透過假設,去找到更適合施力的點。

雖然投影片內容好像跳來跳去,Ruddy老師其實在傳授心法。 在管理或是帶更大的團隊時,讓團隊有正確的思維和方向,會比鉅細靡遺的去管理任何小地方,來的更有效率。

 

 

Running and Operating Large Scale Real-time Data system on Cloud

 

  • 如何開始建構大型Streaming的系統
    • POC 100K
    • 迭代改進
    • 可視性 => 可觀察整個系統的效能和瓶頸
    • End to End
    • Bottom-up thinking
  • 建構團隊
    • 初期要先以多職能團隊為主,除非緊急任務,否則不應該只讓特定人執行特定工作。
    • 若是緊急的任務,再安排固定資源。
    • 角色應該rotate
    • 初期應以讓團隊了解產品或系統的整個脈絡及流程,但是到了更大(如100人)團隊的時候,再來分職責及角色。
  • 可視性及觀察性
    • 大型分散式系統已不可能再一台一台機器看LOG
    • 講者有展示他們Monitor的工具及如何從中抓出一個高風險的BUG




如何實踐品質內建之企業級DevOps流程架構

 

這一場主要是玉山銀行提出他們如何在內部整合包含派工、程式弱掃、容器安全性檢查、CI/CDMonitor的流程。 印象比較深刻的是,他們大量使用JIRA作為整合的工具,這點價值非常高,因為等於ITUSER都可以在同一平台作業。

 



 

 

大型團隊落實 CI/CD 的挑戰

 

我每次都會參加安德魯大大的場子,因為分享內容除了技術,更著重於在處理人和事的看法和經驗上。 詳細的內容可參考 HackMD

 

 

後記

這次的贈品都滿不錯的,我拿到了隨身碟、一條毛巾、旅行包和T-Shirt,也參加了一場Hands-on lab 來彌補幾個月鮮少寫code的時光  XDDD