重构 JSON 或哈希对象

默认情况下，RDI 会将字段添加到目标中的哈希或 JSON 对象 数据库，这些数据库的 ID 与源表的列非常匹配。 下面的示例显示了如何创建全新的对象结构 从现有字段中使用map转型。

映射到新的 JSON 结构

第一个作业文件示例创建一个新的 JSON 对象结构以写入目标。 这source部分选择employee的chinookdatabase（可选的db值对应于sources.<source-name>.connection.databasevalue defined inconfig.yaml).

在transform部分、map转换使用 JMESPath 表达式指定新的 JSON 格式。（请注意，在expressionline 指示应将后面的缩进行解释为单个字符串。 该表达式类似于 JSON 表示法，但数据值由 表字段和 JMESPath 函数。

在这里，我们将employeeidfield 设置为id并为address和contact信息。这namefield 是现有firstname和lastnamefields（字段），其中lastname转换为大写。 在contactsubobject、emailaddress 的replace()隐藏 '@' 符号和点的函数。

在output部分，我们指定要写入 添加到具有自定义键的 JSON 对象。请注意，在output部分，您必须参考 字段在maptransformation，因此我们使用新名称id对于键而不是employeeid.

完整示例如下所示：

source:
  db: chinook
  table: employee
transform:
  - uses: map
    with:
      expression: |
        {
          "id": employeeid,
          "name": concat([firstname, ' ', upper(lastname)]),
          "address": {
            "street": address,
            "city": city,
            "state": state,
            "postalCode": postalcode,
            "country": country
          },
          "contact": {
            "phone": phone,
            "safeEmail": replace(replace(email, '@', '_at_'), '.', '_dot_')
          }
        }
      language: jmespath
output:
  - uses: redis.write
    with:
      connection: target
      data_type: json
      key:
        expression: concat(['emp:', id])
        language: jmespath

If you query one of the new JSON objects, you see output like the following:

> JSON.GET emp:1 $
"[{\"id\":1,\"name\":\"Andrew ADAMS\",\"address\":{\"street\":\"11120 Jasper Ave NW\",\"city\":\"Edmonton\",\"state\":\"AB\",\"postalCode\":\"T5K 2N1\",\"country\":\"Canada\"},\"contact\":{\"phone\":\"+1 (780) 428-9482\",\"safeEmail\":\"andrew_at_chinookcorp_dot_com\"}}]"

Formatted in the usual JSON style, the output looks like the sample below:

{
  "id": 1,
  "name": "Andrew ADAMS",
  "address": {
    "street": "11120 Jasper Ave NW",
    "city": "Edmonton",
    "state": "AB",
    "postalCode": "T5K 2N1",
    "country": "Canada"
  },
  "contact": {
    "phone": "+1 (780) 428-9482",
    "safeEmail": "andrew_at_chinookcorp_dot_com"
  }
}

Map to a hash structure

This example creates a new hash object structure for items from the track table. Here, the map transformation uses SQL for the expression because this is often more suitable for hashes or "flat" JSON objects without subobjects or arrays. The expression renames some of the fields. It also calculates more human-friendly representations for the track duration (originally stored in the milliseconds field) and the storage size (originally stored in the bytes field).

The full example is shown below:

source:
  db: chinook
  table: track
transform:
  - uses: map
    with:
      expression:
          id: trackid
          name: name
          duration: concat(floor(milliseconds / 60000), ':', floor(mod(milliseconds / 1000, 60)))
          storagesize: concat(round(bytes / 1048576.0, 2), 'MB')
      language: sql
output:
  - uses: redis.write
    with:
      connection: target
      data_type: hash
      key:
        expression: concat('track:', id)
        language: sql

If you query the data for one of the track hash objects, you see output like the following:

> hgetall track:16
1) "id"
2) "16"
3) "name"
4) "Dog Eat Dog"
5) "duration"
6) "3:35.0"
7) "storagesize"
8) "6.71MB"
RATE THIS PAGE
Back to top ↑