JSONNET 入門

0x00 TL;DR

jsonnet 是個啥?

jsonnet是google開源的代碼模版化工具, 擴展了JSON,支持OO.
可以通過編寫jsonnet腳本, 生成json/yaml的等配置文件, 應用場景: 生成k8s部署文件等.

本文通過記錄官方指引的方式, 來學習jsonnet的一般用法.

0x01 jsonnet 語法

• 首先, 一個合法的json文檔,也是一個有效的jsonnet程序
• jsonnet 中的變量(field) 不用引號
• 數組裏最後一個對象的後面, 要多加一個 逗號
• 單引號, 雙引號, 作用一樣都表示字符串, 但是 雙引號中的 單引號 可以不用轉義.
• |||塊 可以表達多行字符串
• @'foo', @"foo", 帶@號的字符串, 只能表示單行字符串.
• 可以使用 + 連接數據元素

/* A C-style comment. */
# A Python-style comment.
{
  cocktails: {
    // Ingredient quantities are in fl oz.
    "Tom C'ollins": {
      ingredients: [
        { kind: "Farmer's Gin", qty: 1.5 },
        { kind: 'Lemon', qty: 1 },
        { kind: 'Simple Syrup', qty: 0.5 },
        { kind: 'Soda', qty: 2 },
        { kind: 'Angostura', qty: 'dash' },
      ],
      garnish: 'Maraschino Cherry',
      served: 'Tall',
      description: |||
        The Tom Collins is essentially gin and
        lemonade.  The bitters add complexity.
        1212
      |||,
    },
    Manhattan: {
      ingredients: [
        { kind: 'Rye', qty: 2.5 },
        { kind: 'Sweet Red Vermouth', qty: 1 },
        { kind: 'Angostura', qty: 'dash' },
      ],
      garnish: 'Maraschino Cherry',
      served: 'Straight Up',
      description: @'A clear \ red drink.',
    },
  },
}

0x02 變量

• 使用local關鍵字聲明一個變量
• 如果變量定義後面有其他字段, 則需要在聲明後面加 逗號, 否則加 分號

// A regular definition.
local house_rum = 'Banks Rum';

{
  // A definition next to fields.
  local pour = 1.5,

  Daiquiri: {
    ingredients: [
      { kind: house_rum, qty: pour },
      { kind: 'Lime', qty: 1 },
      { kind: 'Simple Syrup', qty: 0.5 },
    ],
    served: 'Straight Up',
  },
  Mojito: {
    ingredients: [
      {
        kind: 'Mint',
        action: 'muddle',
        qty: 6,
        unit: 'leaves',
      },
      { kind: house_rum, qty: pour },
      { kind: 'Lime', qty: 0.5 },
      { kind: 'Simple Syrup', qty: 0.5 },
      { kind: 'Soda', qty: 3 },
    ],
    garnish: 'Lime wedge',
    served: 'Over crushed ice',
  },
}

0x03 使用變量

• self 指向當前對象
• $ 指向最外層對象
• ['foo'] 查找一個字段
• .f 查找一個對象下面的指定字段
• [10] 查找一個數組對象元素
• 允許任意長路徑, 即: 可以 a.b.c 一直往下取
• 數組切片 , arr[10:20:2], 同 python
• 字符串支持unicode 查找/分隔

{
  'Tom Collins': {
    ingredients: [
      { kind: "Farmer's Gin", qty: 1.5 },
      { kind: 'Lemon', qty: 1 },
      { kind: 'Simple Syrup', qty: 0.5 },
      { kind: 'Soda', qty: 2 },
      { kind: 'Angostura', qty: 'dash' },
    ],
    garnish: 'Maraschino Cherry',
    served: 'Tall',
  },
  Martini: {
    ingredients: [
      {
        // Use the same gin as the Tom Collins.
        kind:
          $['Tom Collins'].ingredients[0].kind,
        qty: 2,
      },
      { kind: 'Dry White Vermouth', qty: 1 },
    ],
    garnish: 'Olive',
    served: 'Straight Up',
  },
  // Create an alias.
  'Gin Martini': self.Martini,
}

0x04 計算表達式

• 支持浮點運算,位運算,邏輯運算
• 和字符串進行+運算時, 會對其他非字符串對象進行隱式轉換
• 字符串支持比較大小
• 對象支持 + 運算
• 支持in 方法來判斷一個對象是否包含字段
• == 判斷 兩個對象所有字段都相等
• 支持% 操作像python一樣處理字符串格式化

{
  haha: 'a' in self,
  test2: '姓名:%s ,身高:%.2f米' % ['James', 2.35],
  concat_array: [1, 2, 3] + [4],
  concat_string: '123' + 4,
  equality1: 1 == '1',
  equality2: [{}, { x: 3 - 1 }]
             == [{}, { x: 2 }],
  ex1: 1 + 2 * 3 / (4 + 5),
  // Bitwise operations first cast to int.
  ex2: self.ex1 | 3,
  // Modulo operator.
  ex3: self.ex1 % 2,
  // Boolean logic
  ex4: (4 > 3) && (1 <= 3) || false,
  // Mixing objects together
  obj: { a: 1, b: 2 } + { b: 3, c: 4 },
  // Test if a field is in an object
  obj_member: 'foo' in { foo: 1 },
  // String formatting
  str1: 'The value of self.ex2 is '
        + self.ex2 + '.',
  str2: 'The value of self.ex2 is %g.'
        % self.ex2,
  str3: 'ex1=%0.2f, ex2=%0.2f'
        % [self.ex1, self.ex2],
  // By passing self, we allow ex1 and ex2 to
  // be extracted internally.
  str4: 'ex1=%(ex1)0.2f, ex2=%(ex2)0.2f'
        % self,
  // Do textual templating of entire files:
  str5: |||
    ex1=%(ex1)0.2f
    ex2=%(ex2)0.2f
  ||| % self,
}

0x05 方法

和python一樣, 方法參數可以省略,支持默認值, 還有閉包.

// Define a local function.
// Default arguments are like Python:
local my_function(x, y=10) = x + y;

// Define a local multiline function.
local multiline_function(x) =
  // One can nest locals.
  local temp = x * 2;
  // Every local ends with a semi-colon.
  [temp, temp + 1];

local object = {
  // A method
  my_method(x): x * x,
};

{
  // Functions are first class citizens.
  call_inline_function:
    (function(x) x * x)(5),

  call_multiline_function: multiline_function(4),

  // Using the variable fetches the function,
  // the parens call the function.
  call: my_function(2),

  // Like python, parameters can be named at
  // call time.
  named_params: my_function(x=2),
  // This allows changing their order
  named_params2: my_function(y=3, x=2),

  // object.my_method returns the function,
  // which is then called like any other.
  call_method1: object.my_method(3),

  standard_lib:
    std.join(' ', std.split('foo/bar', '/')),
  len: [
    std.length('hello'),
    std.length([1, 2, 3]),
  ],
}

0x06 條件計算

if a then b else c
不寫else的話, 返回 null

local Mojito(virgin=false, large=false) = {
  // A local next to fields ends with ','.
  local factor = if large then 2 else 1,
  // The ingredients are split into 3 arrays,
  // the middle one is either length 1 or 0.
  ingredients: [
    {
      kind: 'Mint',
      action: 'muddle',
      qty: 6 * factor,
      unit: 'leaves',
    },
  ] + (
    if virgin then [] else [
      { kind: 'Banks', qty: 1.5 * factor },
    ]
  ) + [
    { kind: 'Lime', qty: 0.5 * factor },
    { kind: 'Simple Syrup', qty: 0.5 * factor },
    { kind: 'Soda', qty: 3 * factor },
  ],
  // Returns null if not large.
  garnish: if large then 'Lime wedge',
  served: 'Over crushed ice',
};

{
  Mojito: Mojito(),
  'Virgin Mojito': Mojito(virgin=true),
  'Large Mojito': Mojito(large=true),
}

0x07 計算屬性

Jsonnet objects can be used like a std::map or similar datastructures from regular languages.

• Recall that a field lookup can be computed with obj[e]
• The definition equivalent is {[e]: ... }
• self or object locals cannot be accessed when field names are being computed, since the object is not yet constructed.
• If a field name evaluates to null during object construction, the field is omitted. This works nicely with the default false branch of a conditional (see below).

local Margarita(salted) = {
  ingredients: [
    { kind: 'Tequila Blanco', qty: 2 },
    { kind: 'Lime', qty: 1 },
    { kind: 'Cointreau', qty: 1 },
  ],
  [if salted then 'garnish']: 'Salt',
};
{
  Margarita: Margarita(true),
  'Margarita Unsalted': Margarita(false),

}

0x08 數組對象表達式

在運行期, 給數組/對象添加元素/字段時, 可以使用 for in, 或 if 語法.

local arr = std.range(5, 8);
{
  array_comprehensions: {
    higher: [x + 3 for x in arr],
    lower: [x - 3 for x in arr],
    evens: [x for x in arr if x % 2 == 0],
    odds: [x for x in arr if x % 2 == 1],
    evens_and_odds: [
      '%d-%d' % [x, y]
      for x in arr
      if x % 2 == 0
      for y in arr
      if y % 2 == 1
    ],
  },
  object_comprehensions: {
    evens: {
      ['f' + x]: true
      for x in arr
      if x % 2 == 0
    },
    // Use object composition (+) to add in
    // static fields:
    mixture: {
      f: 1,
      g: 2,
    } + {
      [x]: 0
      for x in ['a', 'b', 'c']
    },
  },
}

0x09 引用

可以從其他文件引用代碼 或 原始數據

• import效果就是 複製被引用的代碼
• 被引用的文件後綴應該是: .libsonnet
• 原始的json 也可以被引用
• 可以使用 importstr 引用 UTF-8 文本

local martinis = import 'martinis.libsonnet';

{
  'Vodka Martini': martinis['Vodka Martini'],
  Manhattan: {
    ingredients: [
      { kind: 'Rye', qty: 2.5 },
      { kind: 'Sweet Red Vermouth', qty: 1 },
      { kind: 'Angostura', qty: 'dash' },
    ],
    garnish: importstr 'garnish.txt',
    served: 'Straight Up',
  },
}

下面是一個引用方法的例子:

• utils.libsonnet

{
  equal_parts(size, ingredients)::
    // Define a function-scoped variable.
    local qty = size / std.length(ingredients);
    // Return an array.
    [
      { kind: i, qty: qty }
      for i in ingredients
    ],
}

• negroni.jsonnet

local utils = import 'utils.libsonnet';
{
  Negroni: {
    // Divide 3oz among the 3 ingredients.
    ingredients: utils.equal_parts(3, [
      'Farmers Gin',
      'Sweet Red Vermouth',
      'Campari',
    ]),
    garnish: 'Orange Peel',
    served: 'On The Rocks',
  },
}

0x10 異常

Errors can arise from the language itself (e.g. an array overrun) or thrown from Jsonnet code. Stack traces provide context for the error.

• To raise an error: error "foo"
• To assert a condition before an expression: assert "foo";
• A custom failure message: assert "foo" : "message";
• Assert fields have a property: assert self.f == 10,
• With custom failure message: assert "foo" : "message",

Try modifying the code below to trigger the assertion failures, and observe the error messages and stack traces that result.

You might wonder why the divide-by-zero is not triggered in the equal_parts function, since that code occurs before the std.length() check. Jsonnet is a lazy language, so variable initializers are not evaluated until the variable is used. Evaluation order is very hard to notice. It only becomes relevant when code throws errors, or takes a long time to execute. For more discussion of laziness, see design rationale.

0x11 全局參數配置

有兩種方法可用於 全局參數配置

• 外部變量
• 頂層編碼

外部變量

• 傳入變量:

jsonnet --ext-str prefix="Happy Hour "   --ext-code brunch=true ...

• 代碼中讀取變量:

local lib = import 'library-ext.libsonnet';
{
  [std.extVar('prefix') + 'Pina Colada']: {
    ingredients: [
      { kind: 'Rum', qty: 3 },
      { kind: 'Pineapple Juice', qty: 6 },
      { kind: 'Coconut Cream', qty: 2 },
      { kind: 'Ice', qty: 12 },
    ],
    garnish: 'Pineapple slice',
    served: 'Frozen',
  },

  [if std.extVar('brunch') then
    std.extVar('prefix') + 'Bloody Mary'
  ]: {
    ingredients: [
      { kind: 'Vodka', qty: 1.5 },
      { kind: 'Tomato Juice', qty: 3 },
      { kind: 'Lemon Juice', qty: 1.5 },
      { kind: 'Worcestershire', qty: 0.25 },
      { kind: 'Tobasco Sauce', qty: 0.15 },
    ],
    garnish: 'Celery salt & pepper',
    served: 'Tall',
  },

  [std.extVar('prefix') + 'Mimosa']:
    lib.Mimosa,
}

頂層變量

• 應該能被各個子模塊引用
• 應該提供默認值
• 應該以庫的方式提供出來

庫

{
  // Note that the Mimosa is now
  // parameterized.
  Mimosa(brunch): {
    local fizz = if brunch then
      'Cheap Sparkling Wine'
    else
      'Champagne',
    ingredients: [
      { kind: fizz, qty: 3 },
      { kind: 'Orange Juice', qty: 3 },
    ],
    garnish: 'Orange Slice',
    served: 'Champagne Flute',
  },
}

使用

local lib = import 'library-tla.libsonnet';

// Here is the top-level function, note brunch
// now has a default value.
function(prefix, brunch=false) {

  [prefix + 'Pina Colada']: {
    ingredients: [
      { kind: 'Rum', qty: 3 },
      { kind: 'Pineapple Juice', qty: 6 },
      { kind: 'Coconut Cream', qty: 2 },
      { kind: 'Ice', qty: 12 },
    ],
    garnish: 'Pineapple slice',
    served: 'Frozen',
  },

  [if brunch then prefix + 'Bloody Mary']: {
    ingredients: [
      { kind: 'Vodka', qty: 1.5 },
      { kind: 'Tomato Juice', qty: 3 },
      { kind: 'Lemon Juice', qty: 1.5 },
      { kind: 'Worcestershire', qty: 0.25 },
      { kind: 'Tobasco Sauce', qty: 0.15 },
    ],
    garnish: 'Celery salt & pepper',
    served: 'Tall',
  },

  [prefix + 'Mimosa']: lib.Mimosa(brunch),
}

0x12 面向對象

先說幾個特性:

• 所有對象繼承自 JSON
• 對象都可以用 + 符號合併, 如果有字段衝突, 以右邊的對象字段爲準
• self 指向當前對象

附加一些有趣的特性:

• 隱藏字段使用::作前綴, 就不會輸出到json/yaml了.
• 支持super關鍵字,用於訪問父類字段
• 使用 +: 語法覆蓋原來的對象字段值.

local Base = {
  f: 2,
  g: self.f + 100,
};

local WrapperBase = {
  Base: Base,
};

{
  Derived: Base + {
    f: 5,
    old_f: super.f,
    old_g: super.g,
  },
  WrapperDerived: WrapperBase + {
    Base+: { f: 5 },
  },
}