向键添加新字段

默认情况下，RDI 会将字段添加到目标中的哈希或 JSON 对象 与源表的列匹配的数据库。 以下示例显示了如何使用add_field转型。

添加单个字段

第一个示例向数据添加单个字段。 这source部分选择customer的chinookdatabase（可选的db值对应于sources.<source-name>.connection.databasevalue defined inconfig.yaml).

在transform部分、add_field转换会添加一个名为localphone添加到对象中，该对象是通过从phone具有 JMESPath 函数的字段regex_replace(). 您还可以指定sql作为language如果您更喜欢创建新的 字段。

这outputsection 指定hash作为data_type写入目标，该 覆盖target_data_type定义于config.yaml.此外，output.with.keysection 指定表单的自定义键格式cust:<id>哪里 这idpart 由uuid()功能。

完整示例如下所示：

source:
  db: chinook
  table: customer
transform:
  - uses: add_field
    with:
      expression: regex_replace(phone, '\+[0-9]+ (\([0-9]+\) )?', '')
      field: localphone
      language: jmespath
output:
  - uses: redis.write
    with:
      connection: target
      data_type: hash
      key:
        expression: concat(['cust:', uuid()])
        language: jmespath

If you queried the generated target data from the default transformation using redis-cli, you would see something like the following:

 1) "customerid"
 2) "27"
 3) "firstname"
 4) "Patrick"
 5) "lastname"
 6) "Gray"
.
.
17) "phone"
18) "+1 (520) 622-4200"
.
.

Using the job file above, the data also includes the new localphone field:

 1) "customerid"
 2) "27"
 3) "firstname"
 4) "Patrick"
 5) "lastname"
 6) "Gray"
 .
 .
23) "localphone"
24) "622-4200"

Add multiple fields

The add_field transformation can also add multiple fields at the same time if you specify them under a fields subsection. The example below adds two fields to the track objects. The first new field, seconds, is created using a SQL expression to calculate the duration of the track in seconds from the milliseconds field. The second new field, composerlist, adds a JSON array using the split() function to split the composer string field wherever it contains a comma.

source:
  db: chinook
  table: track
transform:
  - uses: add_field
    with:
      fields:
        - expression: floor(milliseconds / 1000)
          field: seconds
          language: sql
        - expression: split(composer)
          field: composerlist
          language: jmespath
output:
  - uses: redis.write
    with:
      connection: target
      data_type: json
      key:
        expression: concat(['track:', trackid])
        language: jmespath

You can query the target database to see the new fields in the JSON object:

> JSON.GET track:1 $

"[{\"trackid\":1,\"name\":\"For Those About To Rock (We Salute You)\",\"albumid\":1,\"mediatypeid\":1,\"genreid\":1,\"composer\":\"Angus Young, Malcolm Young, Brian Johnson\",\"milliseconds\":343719,\"bytes\":11170334,\"unitprice\":\"0.99\",\"seconds\":343,\"composerlist\":[\"Angus Young\",\" Malcolm Young\",\" Brian Johnson\"]}]"

Using add_field with remove_field

You can use the add_field and remove_field transformations together to completely replace fields from the source. For example, if you add a new fullname field, you might not need the separate firstname and lastname fields. You can remove them with a job file like the following:

source:
  db: chinook
  table: customer
transform:
  - uses: add_field
    with:
      expression: concat(firstname, ' ', lastname)
      field: fullname
      language: sql
  - uses: remove_field
    with:
      fields:
        - field: firstname
        - field: lastname
output:
  - uses: redis.write
    with:
      connection: target
      data_type: hash
      key:
        expression: concat(['cust:', customerid])
        language: jmespath
RATE THIS PAGE
Back to top ↑