Home > Cloud Cruiser 4 > Collecting, transforming, and publishing > CC Record data format

CC Record data format

Most steps in an ETL flow output an intermediate file called a CC Record (Cloud Cruiser Record) file. CC Record files are Cloud Cruiser’s all-purpose data management format. In general, collectors retrieve native source data from any given resource in the cloud and output the collected data in a CC Record format so it can be processed further.

Collectors use the standard CC Record file format to save feed data and workbooks use it throughout data processing as input to and output from various steps in a flow. The format consists of comma-separated fields and line-end separated records.

Structure and content

The standard CC Record structure is:

START_DATE (yyyymmdd),
END_DATE (yyyymmdd),
START_TIME (hh:mm),
END_TIME (hh:mm),
IDENTIFIER_COUNT,
{IDENTIFIER, VALUE, ...},
RESOURCE_COUNT,
{RESOURCE, QUANTITY[:COST[:RATE[:UNIT[:INTERVAL]]]], ...}

The standard CC Record files contain the following key pieces of information:

  • Date and Time: The period during which a resource was consumed.
    START_DATE (yyyymmdd),
    END_DATE (yyyymmdd),
    START_TIME (hh:mm),
    END_TIME (hh:mm),

    The total amount of time (down to the second) can be important if a resource is allocated. This means that the amount of time it is being consumed is directly related to the cost for using it. For example, VM instances that are charged based on how long they are running are considered allocated resources.

    The usage date consists of four fields: START_DATE, END_DATE, START_TIME, and END_TIME. Only END_DATE is required, as not all resources are allocated.

  • Identifiers: Metadata that provides details about the resource and how it was consumed.
    IDENTIFIER_COUNT,
    {IDENTIFIER, VALUE, ...},

    Identifiers are name-value pairs that allow a resource to report consumption details. For example, for a VM instance, a typical identifier might be VMName and its value might be accounting-0001-linux. A second identifier might be userID with a value of 0034101 and can be used for Cloud Cruiser to determine the unique billing code to charge the usage against.

    There is no limit to the number of identifiers collected, although extraneous identifiers are usually dropped in order to minimize the amount of data stored in the database.

  • Resources: Data on the items consumed and their quantity.
    RESOURCE_COUNT,
    {RESOURCE, QUANTITY[:COST[:RATE[:UNIT[:INTERVAL]]]], ...}

    Resources describe what was used and how much of it was used. For example, a VM-related resource might be named large_instance and have a quantity of 1. A storage resource might be named disk.sas.used with a quantity of 424 (total storage in GB).

Formatting special characters

In a CC Record, identifier values containing a comma (,), double quote ("), or leading or trailing spaces must be enclosed by double quotes. Also, a double quote contained in the value must be escaped by another double quote.

  • Collected value:    This begins with three spaces, contains "double quotes," and two commas
  • In a CC Record: ...,"   This begins with three spaces, contains ""double quotes,"" and two commas",...
Last modified

Tags

Classifications

This page has no classifications.
© Copyright 2018 Hewlett Packard Enterprise Development LP