diff --git a/docs/guides/3.0.0.md b/docs/guides/3.0.0.md new file mode 100644 index 0000000..21a12a2 --- /dev/null +++ b/docs/guides/3.0.0.md @@ -0,0 +1,121 @@ +## Upgrading to version 3 + +**Rate limit rules now support non-IP client identifiers** + +To add support for non-IP client identifiers within rate limit rules we had to convert them from `TypeMap` to `TypeSet`. + +This means that in order to continue using your existing configuration you will need to update your configuration and also update existing resources within your state file. + +We have outlined the changes that will need to be made in detail below. + +### High Level Overview + +1. Backup terraform config and state +2. Change all `rate_limit = {}` definitions to `rate_limit {}` in main.tf +3. Add `client_identifiers {}` to `rate_limit {}` section of main.tf +4. Either manually or using the provided script remove all existing rules and re-import + +### Detailed Overview + +#### Backup terraform config and state +1. Make a copy of your terraform configuration and state files + +``` +tar -czvf terraform-backup.tar.gz main.tf terraform.tfstate +``` + +#### Change all `rate_limit = {}` definitions to `rate_limit {}` +2. Any references that have `rate_limit = {}` will need to be converted to `rate_limit {}`. + +Existing: + +``` +rate_limit = { + threshold = 6 + interval = 10 + duration = 300 +} +``` + +New: +``` +rate_limit { + threshold = 6 + interval = 10 + duration = 300 +} +``` + +#### Add `client_identifiers {}` +3. Rate limit rules now require that you specify `client_identifiers`. To continue having your rate limit rules operate as they were configured for "ip" based client identifiers you will need to update your `rate_limit {}` section to also specify `client_identifiers`. Example as follows: + +``` +rate_limit { + threshold = 6 + interval = 10 + duration = 300 + + client_identifiers { + type = "ip" + } +} +``` + +The supported values for `client_identifiers` are as follows: `ip`, `requestHeader`, `requestCookie`, `postParameter`, `signalPayload`. +Each of these (except for `ip`) allow you to specify additional parameters for `name` and `key`. + +You can combine up to 2 of these together (e.g. ip+requestHeader). + +For example: + +``` +rate_limit { + threshold = 6 + interval = 10 + duration = 300 + + client_identifiers { + type = "ip" + } + client_identifiers { + type = "requestHeader" + name = "x-my-header" + } +} +``` + +#### Either manually or using the provided script remove all existing rules and re-import +4. Due to `rate_limit` changing from a `TypeMap` to a `TypeSet` **all** existing site rules need their state removed and re-imported (even those that do not explicitly define `rate_limit`). + +Manually this would be as follows: + +* Run `terraform state rm sigsci_site_rule.` for every rule. +* Re-import each rule using `terraform import sigsci_site_rule. :` + +**IMPORTANT**: You should run `terraform state rm` for every rule before doing the import! + +To aid with this, we have provided the below script that can be used to automate this process. +The script uses `terraform state pull` to gather the existing state and pulls in all of your site rules, their names, and ids. It then removes the existing state and imports it again into the new structure. + +**Note: The below script requires that you have installed `jq`** +``` +#!/bin/bash + +TF_STATE=$(terraform state pull) +IMPORT_CMD="" +for rule_name in $(echo "$TF_STATE"|jq -r '.resources[]| select(.type=="sigsci_site_rule") | .name'); do + site_short_name=$(echo "$TF_STATE"|jq --arg rule_name "$rule_name" -r '.resources[]| select(.type=="sigsci_site_rule") | select(.name==$rule_name) | .instances[].attributes | .site_short_name '); + rule_id=$(echo "$TF_STATE"|jq --arg rule_name "$rule_name" -r '.resources[]| select(.type=="sigsci_site_rule") | select(.name==$rule_name) | .instances[].attributes | .id '); + + printf "Removing state for: site:%s rule_name:%s rule_id:%s\n" "$site_short_name" "$rule_name" "$rule_id"; + terraform state rm sigsci_site_rule."$rule_name" + if [ -z "$IMPORT_CMD" ] ; then + IMPORT_CMD="terraform import sigsci_site_rule.\"$rule_name\" \"$site_short_name\":\"$rule_id\""; + else + IMPORT_CMD="$IMPORT_CMD;terraform import sigsci_site_rule.\"$rule_name\" \"$site_short_name\":\"$rule_id\"" + fi +done; + +echo "Re-importing rules" +eval $IMPORT_CMD; +``` diff --git a/docs/resources/site_rule.md b/docs/resources/site_rule.md index 520a2bb..9f39255 100644 --- a/docs/resources/site_rule.md +++ b/docs/resources/site_rule.md @@ -179,7 +179,7 @@ resource "sigsci_site_rule" "test-signal-exclusion" { ### Optional - `actions` (Block Set, Max: 2) Actions (see [below for nested schema](#nestedblock--actions)) -- `rate_limit` (Map of String) Rate Limit +- `rate_limit` (Block Set, Max: 1) Rate Limit (see [below for nested schema](#nestedblock--rate_limit)) - `requestlogging` (String) Indicates whether to store the logs for requests that match the rule's conditions (sampled) or not store them (none). This field is only available for rules of type `request`. Not valid for `signal` or `rateLimit`. - `signal` (String) The signal id of the signal being excluded @@ -254,6 +254,29 @@ Optional: - `response_code` (Number) HTTP code agent for agent to respond with. range: 301, 302, or 400-599, defaults to '406' if not provided. Only valid with the 'block' action type. - `signal` (String) signal id to tag + + +### Nested Schema for `rate_limit` + +Required: + +- `client_identifiers` (Block Set, Min: 1) Client Identifiers (see [below for nested schema](#nestedblock--rate_limit--client_identifiers)) +- `duration` (Number) duration in seconds (300 < x < 3600) +- `interval` (Number) interval in minutes (1, 5, 10) +- `threshold` (Number) threshold + + +### Nested Schema for `rate_limit.client_identifiers` + +Required: + +- `type` (String) (ip, requestHeader, requestCookie, postParameter, signalPayload) + +Optional: + +- `key` (String) +- `name` (String) + ### Templated Signals We have curated a list of templates for common rules, the full list of available signals is available below. diff --git a/provider/lib.go b/provider/lib.go index 36b2f6f..d7868bb 100644 --- a/provider/lib.go +++ b/provider/lib.go @@ -474,45 +474,65 @@ func expandAttackThresholds(attackThresholdsResource *schema.Set) []sigsci.Attac return attackThresholds } -func expandRuleRateLimit(rateLimitResource map[string]interface{}) *sigsci.RateLimit { - var threshold, interval, duration int - var err error - if val, ok := rateLimitResource["threshold"]; ok { - threshold, err = strconv.Atoi(val.(string)) - if err != nil { - return nil - } - } else { +func expandRuleRateLimit(rateLimitResource *schema.Set) *sigsci.RateLimit { + if len(rateLimitResource.List()) == 0 { return nil } - if val, ok := rateLimitResource["interval"]; ok { - interval, err = strconv.Atoi(val.(string)) - if err != nil { - return nil - } - } - if val, ok := rateLimitResource["duration"]; ok { - duration, err = strconv.Atoi(val.(string)) - if err != nil { - return nil + + genericElement := rateLimitResource.List()[0] + castElement := genericElement.(map[string]interface{}) + + var clientIdentifiers []sigsci.ClientIdentifier + for _, m := range castElement["client_identifiers"].(*schema.Set).List() { + key := m.(map[string]interface{})["key"].(string) + name := m.(map[string]interface{})["name"].(string) + t := m.(map[string]interface{})["type"].(string) + + ci := sigsci.ClientIdentifier{ + Key: key, + Name: name, + Type: t, } + + clientIdentifiers = append(clientIdentifiers, ci) } return &sigsci.RateLimit{ - Threshold: threshold, - Interval: interval, - Duration: duration, + Threshold: castElement["threshold"].(int), + Interval: castElement["interval"].(int), + Duration: castElement["duration"].(int), + ClientIdentifiers: clientIdentifiers, } } -func flattenRuleRateLimit(rateLimit *sigsci.RateLimit) map[string]string { +func flattenRuleRateLimit(rateLimit *sigsci.RateLimit) []interface{} { if rateLimit == nil { return nil } - return map[string]string{ - "threshold": strconv.Itoa(rateLimit.Threshold), - "interval": strconv.Itoa(rateLimit.Interval), - "duration": strconv.Itoa(rateLimit.Duration), + + clientIdentifiers := []map[string]string{} + for _, ci := range rateLimit.ClientIdentifiers { + m := map[string]string{} + m["type"] = ci.Type + + if ci.Name != "" { + m["name"] = ci.Name + } + + if ci.Key != "" { + m["key"] = ci.Key + } + + clientIdentifiers = append(clientIdentifiers, m) + } + + return []interface{}{ + map[string]interface{}{ + "threshold": rateLimit.Threshold, + "interval": rateLimit.Interval, + "duration": rateLimit.Duration, + "client_identifiers": clientIdentifiers, + }, } } diff --git a/provider/resource_site_rule.go b/provider/resource_site_rule.go index 4af443b..f6654c3 100644 --- a/provider/resource_site_rule.go +++ b/provider/resource_site_rule.go @@ -200,8 +200,9 @@ func resourceSiteRule() *schema.Resource { }, }, "rate_limit": { - Type: schema.TypeMap, + Type: schema.TypeSet, Description: "Rate Limit", + MaxItems: 1, Optional: true, Elem: &schema.Resource{ Schema: map[string]*schema.Schema{ @@ -218,9 +219,32 @@ func resourceSiteRule() *schema.Resource { "duration": { Type: schema.TypeInt, Description: "duration in seconds (300 < x < 3600)", - Default: 600, Required: true, }, + "client_identifiers": { + Type: schema.TypeSet, + Description: "Client Identifiers", + Required: true, + Elem: &schema.Resource{ + Schema: map[string]*schema.Schema{ + "type": { + Type: schema.TypeString, + Description: "(ip, requestHeader, requestCookie, postParameter, signalPayload)", + Required: true, + }, + "name": { + Type: schema.TypeString, + Description: "", + Optional: true, + }, + "key": { + Type: schema.TypeString, + Description: "", + Optional: true, + }, + }, + }, + }, }, }, }, @@ -251,7 +275,7 @@ func resourceSiteRuleCreate(d *schema.ResourceData, m interface{}) error { siteRulesBody.Conditions = expandRuleConditions(d.Get("conditions").(*schema.Set)) siteRulesBody.Actions = expandRuleActions(d.Get("actions").(*schema.Set)) - siteRulesBody.RateLimit = expandRuleRateLimit(d.Get("rate_limit").(map[string]interface{})) + siteRulesBody.RateLimit = expandRuleRateLimit(d.Get("rate_limit").(*schema.Set)) rule, err := sc.CreateSiteRule(corp, site, siteRulesBody) if err != nil { @@ -343,7 +367,7 @@ func resourceSiteRuleUpdate(d *schema.ResourceData, m interface{}) error { updateSiteRuleBody.Conditions = expandRuleConditions(d.Get("conditions").(*schema.Set)) updateSiteRuleBody.Actions = expandRuleActions(d.Get("actions").(*schema.Set)) - updateSiteRuleBody.RateLimit = expandRuleRateLimit(d.Get("rate_limit").(map[string]interface{})) + updateSiteRuleBody.RateLimit = expandRuleRateLimit(d.Get("rate_limit").(*schema.Set)) _, err := sc.UpdateSiteRuleByID(corp, site, d.Id(), updateSiteRuleBody) if err != nil { diff --git a/provider/resource_site_rule_test.go b/provider/resource_site_rule_test.go index 96ef447..7e27004 100644 --- a/provider/resource_site_rule_test.go +++ b/provider/resource_site_rule_test.go @@ -151,10 +151,13 @@ func TestACCResourceSiteRuleRateLimit_basic(t *testing.T) { signal=sigsci_site_signal_tag.test_tag.id type="logRequest" } - rate_limit = { + rate_limit { threshold=10 interval=10 duration=600 + client_identifiers { + type = "ip" + } } }`, testSite, testSite), Check: resource.ComposeAggregateTestCheckFunc( @@ -167,9 +170,13 @@ func TestACCResourceSiteRuleRateLimit_basic(t *testing.T) { resource.TestCheckResourceAttr(resourceName, "actions.194535128.type", "logRequest"), resource.TestCheckResourceAttr(resourceName, "actions.194535128.signal", "site.my-rate-limit-tag"), resource.TestCheckResourceAttr(resourceName, "conditions.#", "1"), - resource.TestCheckResourceAttr(resourceName, "rate_limit.threshold", "10"), - resource.TestCheckResourceAttr(resourceName, "rate_limit.interval", "10"), - resource.TestCheckResourceAttr(resourceName, "rate_limit.duration", "600"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.#", "1"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.3378795908.threshold", "10"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.3378795908.interval", "10"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.3378795908.duration", "600"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.3378795908.client_identifiers.#", "1"), + + resource.TestCheckResourceAttr(resourceName, "rate_limit.3378795908.client_identifiers.2548995648.type", "ip"), ), }, { @@ -878,10 +885,13 @@ func TestACCResourceSiteRule_UpdateRateLimit(t *testing.T) { signal=sigsci_site_signal_tag.test_tag.id requestlogging="" expiration= "" - rate_limit = { + rate_limit { threshold = 5 interval = 10 duration = 300 + client_identifiers { + type = "ip" + } } conditions { type="single" @@ -899,10 +909,10 @@ func TestACCResourceSiteRule_UpdateRateLimit(t *testing.T) { resource.TestCheckResourceAttr(resourceName, "type", "rateLimit"), resource.TestCheckResourceAttr(resourceName, "site_short_name", testSite), resource.TestCheckResourceAttr(resourceName, "reason", "Site rule update with rate limit"), - resource.TestCheckResourceAttr(resourceName, "rate_limit.%", "3"), - resource.TestCheckResourceAttr(resourceName, "rate_limit.threshold", "5"), - resource.TestCheckResourceAttr(resourceName, "rate_limit.interval", "10"), - resource.TestCheckResourceAttr(resourceName, "rate_limit.duration", "300"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.#", "1"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.2479056779.threshold", "5"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.2479056779.interval", "10"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.2479056779.duration", "300"), resource.TestCheckResourceAttr(resourceName, "actions.#", "1"), resource.TestCheckResourceAttr(resourceName, "actions.2210203034.type", "logRequest"), resource.TestCheckResourceAttr(resourceName, "actions.2210203034.signal", "WRONG-API-CLIENT"), @@ -931,10 +941,15 @@ func TestACCResourceSiteRule_UpdateRateLimit(t *testing.T) { signal= sigsci_site_signal_tag.test_tag.id requestlogging="" expiration= "" - rate_limit = { + rate_limit { threshold = 6 interval = 10 duration = 300 + client_identifiers { + type = "requestHeader" + name = "X-Request-Info" + key = "key1" + } } conditions { type="single" @@ -951,10 +966,16 @@ func TestACCResourceSiteRule_UpdateRateLimit(t *testing.T) { resource.TestCheckResourceAttr(resourceName, "type", "rateLimit"), resource.TestCheckResourceAttr(resourceName, "site_short_name", testSite), resource.TestCheckResourceAttr(resourceName, "reason", "Site rule update with rate limit"), - resource.TestCheckResourceAttr(resourceName, "rate_limit.%", "3"), - resource.TestCheckResourceAttr(resourceName, "rate_limit.threshold", "6"), - resource.TestCheckResourceAttr(resourceName, "rate_limit.interval", "10"), - resource.TestCheckResourceAttr(resourceName, "rate_limit.duration", "300"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.#", "1"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.2941527592.threshold", "6"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.2941527592.interval", "10"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.2941527592.duration", "300"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.2941527592.client_identifiers.#", "1"), + + resource.TestCheckResourceAttr(resourceName, "rate_limit.2941527592.client_identifiers.1349511376.type", "requestHeader"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.2941527592.client_identifiers.1349511376.name", "X-Request-Info"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.2941527592.client_identifiers.1349511376.key", "key1"), + resource.TestCheckResourceAttr(resourceName, "actions.#", "1"), resource.TestCheckResourceAttr(resourceName, "actions.2210203034.type", "logRequest"), resource.TestCheckResourceAttr(resourceName, "actions.2210203034.signal", "WRONG-API-CLIENT"), @@ -977,3 +998,84 @@ func TestACCResourceSiteRule_UpdateRateLimit(t *testing.T) { }, }) } + +func TestACCResourceSiteRuleRateLimit_client_identifiers(t *testing.T) { + resourceName := "sigsci_site_rule.test" + resource.Test(t, resource.TestCase{ + PreCheck: func() { testAccPreCheck(t) }, + Providers: testAccProviders, + CheckDestroy: testACCCheckSiteRuleDestroy, + Steps: []resource.TestStep{ + { + Config: fmt.Sprintf(` + resource "sigsci_site_signal_tag" "test_tag" { + site_short_name = "%s" + name = "My rate limit tag" + description = "test description" + } + resource "sigsci_site_rule" "test" { + site_short_name="%s" + type= "rateLimit" + group_operator="any" + enabled= true + reason= "Example site rule update" + signal= sigsci_site_signal_tag.test_tag.id + expiration= "" + requestlogging= "" + conditions { + type="single" + field="ip" + operator="equals" + value="1.2.3.4" + } + actions { + signal=sigsci_site_signal_tag.test_tag.id + type="logRequest" + } + rate_limit { + threshold=10 + interval=10 + duration=600 + client_identifiers { + type = "ip" + } + client_identifiers { + type = "requestHeader" + name = "X-Request-Info" + key = "key1" + } + } + }`, testSite, testSite), + Check: resource.ComposeAggregateTestCheckFunc( + //testCheckSiteRuleExists(resourceName), + //testCheckSiteRulesAreEqual(resourceName), + resource.TestCheckResourceAttr(resourceName, "type", "rateLimit"), + resource.TestCheckResourceAttr(resourceName, "site_short_name", testSite), + resource.TestCheckResourceAttr(resourceName, "reason", "Example site rule update"), + resource.TestCheckResourceAttr(resourceName, "actions.#", "1"), + resource.TestCheckResourceAttr(resourceName, "actions.194535128.type", "logRequest"), + resource.TestCheckResourceAttr(resourceName, "actions.194535128.signal", "site.my-rate-limit-tag"), + resource.TestCheckResourceAttr(resourceName, "conditions.#", "1"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.#", "1"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.115245974.client_identifiers.#", "2"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.115245974.threshold", "10"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.115245974.interval", "10"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.115245974.duration", "600"), + + resource.TestCheckResourceAttr(resourceName, "rate_limit.115245974.client_identifiers.1349511376.key", "key1"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.115245974.client_identifiers.1349511376.name", "X-Request-Info"), + resource.TestCheckResourceAttr(resourceName, "rate_limit.115245974.client_identifiers.1349511376.type", "requestHeader"), + + resource.TestCheckResourceAttr(resourceName, "rate_limit.115245974.client_identifiers.2548995648.type", "ip"), + ), + }, + { + ResourceName: resourceName, + ImportStateIdPrefix: fmt.Sprintf("%s:", testSite), + ImportState: true, + ImportStateVerify: true, + ImportStateCheck: testAccImportStateCheckFunction(1), + }, + }, + }) +}