Cloud Cruiser retrieves Amazon Web Services (AWS) consumption data using the AWS Billing API. If you are using Amazon CloudWatch, Cloud Cruiser can collect performance and utilization data through the CloudWatch API. Because Cloud Cruiser collects data nightly, it collects unblended rates, which are more accurate on a day-to-day basis than blended rates.
Every night, Cloud Cruiser collects usage, billing, and performance data from one or more of your Amazon web Service (AWS) accounts. If you have linked accounts, Cloud Cruiser collects usage and billing data from the payer account, and CloudWatch performance data from each of your linked accounts. During this nightly collection Cloud Cruiser collects data for the current and previous months. This ensures that if something in your AWS bill changes, you always have the most current data available in your Cloud Cruiser analytics.
Cloud Cruiser also collects tax information from the AWS bill, where those records appear with the description
Tax of type <type>. In Cloud Cruiser, this description appears in the
Product field. The Resource ID for these records begins with the prefix
Tax-. If the tax is a monthly charge, it appears in Cloud Cruiser with the usage for the last day of the month.
For more information about how Cloud Cruiser maps source provider data to Cloud Cruiser normalized fields, see Field mapping.
The following video shows you how to configure collection from an Amazon Web Services cloud, so you can see your AWS cost and usage data in Cloud Cruiser charts and reports:
Before you begin
Before creating an AWS collection, you must set up your AWS account(s) to work with Cloud Cruiser. As you set up your AWS account(s), note the following credential information, which you will need when creating an AWS collection in Cloud Cruiser:
- Access Key Id: The access key ID for an AWS user who has access to billing information. If you are using consolidated billing, ensure that the user has access to billing information on the payer account.
- Secret Key: The secret key for this user.
- Cost Bucket: The name of the Amazon Simple Storage Service (S3) bucket containing the data you want to collect. Cloud Cruiser needs only the name of the cost bucket, and not the full directory path. Additionally, the cost bucket name must be at least 3 characters long, and can contain only letters, numbers, periods (.), hyphens (-), and underscores (_).
To set up your AWS account
- In AWS, turn on detailed billing reports for the account for which you want to collect data.
For instructions, see Understand Your Usage with Detailed Billing Reports in the AWS documentation. If you are using consolidated billing, perform this step for the payer account, not the linked accounts. For more information about consolidated billing, see Pay Bills for Multiple Accounts with Consolidated Billing in the AWS documentation.
- Note the name of the Amazon S3 bucket you specify.
- In the list of reports to receive, select both Monthly report and Detailed billing report with services and tags.
- Configure an AWS user account for the collection of billing data. Using Amazon's Identity and Access Management (IAM) service, for your payer account create an AWS user with an access key ID, a secret key, and secure access to the specified S3 bucket. You could simply grant this access to the account owner, but Cloud Cruiser recommends creating a user specifically for billing access. Ensure that you apply the
Amazon S3 Read Only policy to this account. For the JSON version of the
Amazon S3 Read Only policy to apply to that user, see Amazon S3 Read Only in the AWS documentation.
- If you want Cloud Cruiser to collect the CloudWatch metrics that enable cost optimization with Insights, or to recommend actions to lower your cost with reserved instances, then for each of your linked accounts create an AWS user with the necessary permissions and an access key ID and secret key.
For permissions requirements, see AWS permissions needed by Cloud Cruiser.
- If you want Cloud Cruiser to collect your AWS tags, ensure that those tags are included in your Cost Allocation Report. For more information, see Setting Up Your Monthly Cost Allocation Report in the AWS documentation.
Creating an Amazon Web Services collection
To create an AWS collection
- On the Data > Collections screen, click the Add icon .
The New Data Collection screen appears.
- Click AWS.
The Credentials screen appears.
- Enter the Access Key ID, Secret Key, and Cost Bucket for your AWS payer account, and then click Next.
Cloud Cruiser verifies your credentials, and then displays all of your linked accounts, including your payer account.
- On the Manage Linked Accounts page, enter credentials for your linked accounts and specify how they should be used by the RI Advisor, and then click Next.
This is optional, but recommended to enable cost optimization with Insights and the RI Advisor. For instructions on this page, see Managing AWS linked accounts.
- On the Options page, enter the following information:
- Name: Enter a unique name for the collection.
- Comments: Optionally, enter additional information about the collection. For example, you might describe the types of data you are collecting, or details about an AWS instance.
- Earliest Date to Collect: Select the first day's data you want to collect. Cloud Cruiser collects data from that date forward. For example, if you had your cloud in a pre-production mode for a short time while you fine-tuned the services you offer, consider setting the date to the first day your cloud was in production.
- Source Currency: The currency in which your cloud provider reports your charges. For AWS, this is always US Dollars, so you cannot change this setting. If the Source Currency and your account's display currency do not match, the following choices appear. These affect your entire account, not just this collection. For information about them, see Currency and locale:
- Click Finish.
The new collection appears in your list of collections. Any errors or warnings related to your collection appear on the Health Checks page. For more information, see Health checks.
CloudWatch rate limits
If you notice a gap in your AWS performance data in Cloud Cruiser charts and reports, you might have reached the CloudWatch API limit. Enter an AWS Support case to have your limit increased.
Amazon implements limits on customer accounts to prevent runaway usage, including a monthly limit of 1 million CloudWatch API requests. (For information about this and other CloudWatch limits, see CloudWatch, CloudWatch Events, and CloudWatch Logs Limits in the AWS documentation.) The more AWS resources you are using, the more CloudWatch API requests are needed to collect metrics data for those resources. Unfortunately, the limit does not scale with the number of resources you have in production, and it is possible to reach the limit with fewer than 300 AWS resources. If you reach the limit, Cloud Cruiser can no longer collect CloudWatch data.
You must have a Developer, Business, or Enterprise support plan with AWS to submit a technical support case. If you have the Basic plan, you cannot submit a technical support case.
To request an AWS rate increase
- Navigate to the AWS Support page (https://console.aws.amazon.com/support).
- Click Create case.
The Create Case form opens.
- In the Regarding field, select Technical Support.
- In the Service field, select CloudWatch.
- In the Category field, select APIs.
- Select a Severity, and enter a Subject.
- In the Description field, explain that you are using a CloudWatch monitoring tool that makes many
GetMetricStatistics requests. If you do not expect a significant increase in the number AWS resources you use, ask Amazon to double your limit. If you think your resource use will continue to grow, ask for a 4x or 5x increase. To speed a resolution with AWS Support, include your AWS region as part of your request.
- Click Submit.