Need help with parsing a file containing the output of Terraform plan command

[Subhajit97]

Hi! I am trying to parse a file that contains the output of the terraform plan command. I was referring to https://docs.kics.io/latest/platforms/#terraform_plan.

Are there other documentation or sample code snippets explaining the above parsing methods?

[cxMiguelSilva]

Hi @Subhajit97,
I believe I could not fully comprehend your question. I will answer what I think you have asked. Bee free to add more questions 🙂!

KICS supports the scan of Terraform Plan files and like other IaC files to scan it, you just need to provide the path to the file.
kics scan -p /path/to/your/tfplan
During the pre-scan KICS identifies the files to scan. The supported files are identified in the analyzer, where through multiple regular expressions KICS is able to identify the type of the file from within the supported ones.
If you want to know a bit more about this process you can see the code here

Then based on the types identified KICS parses the sources with the correct parser, for the tfplan case is used the JSON parser. The JSON TfPlan parser created output to the payload similar to the terraform parser in order to be able to use the same queries.
The tfplan parser is located here

Let me know if this helps 😃

[Subhajit97]

Thanks for the quick reply, @cxMiguelSilva! It really helps.

I can read the Terraform plan using the method Parse. But seems like the data that the method returns only contains the resource information. Is there any way to read the output changes as well if the terraform plan also contains changes in the output block?

[cxMiguelSilva]

Great!

You can see the output of the parsing by exporting the payload. There you can see the object created for every document, that the queries will run over. To export the payload use the flag -d.
More info here.

[Subhajit97]

Let me expand my use case a little bit.
I am using the Parse method to parse the JSON file containing the Terraform plan into the []model.Document.

The TF plan has both resource_changes and output_changes. For example:

"output_changes": {
    "name": {
      "actions": ["update"],
      "before": {
        "name": "..."
      },
      "after": "my-value",
      "after_unknown": false,
      "before_sensitive": false,
      "after_sensitive": false
    }
  },

But, when I pass the TF plan file content to the Parse method, it only returns the document object for the resource, but didn't get anything for the output changes.

Does it only return the resource from the TF plan?
https://github.com/Checkmarx/kics/blob/master/pkg/parser/json/tfplan.go#L12

For example:

map[resource:map[my_test_resource:map[example:map[_kics_lines:map[_kics__default:map[_kics_line:17] _kics_prop1:map[_kics_line:18] _kics_prop2:map[_kics_line:19] _kics_prop3:map[_kics_line:20]] prop1:value1 prop2:value2 prop3:value3]]]]

Code:

package main

import (
	"fmt"
	"os"

	"github.com/Checkmarx/kics/pkg/parser"
	jsonParser "github.com/Checkmarx/kics/pkg/parser/json"
)

func main() {
        p := jsonParser.Parser{}

	path := "/path/to/sample-plan.tfplan.json"
	content, err := os.ReadFile(path)
	if err != nil {
		return
	}

	var str string
	documents, _, err := p.Parse(str, content)
	if err != nil {
		return
	}
	for _, doc := range documents {
		fmt.Printf("%+v\n", doc)
	}
}

[cxMiguelSilva]

Hi @Subhajit97,
The current parser of the tf plan files only takes into account the resource changes.
We hope to be able to take a look to analyze that use case.
We will get to you once we have any info.

[cxMiguelSilva]

Hi @Subhajit97, how are you doing?

After some more specific searches, I was able to evaluate the current implementation of the TF Plan parser and understand that it only takes into account the planned values for each module presented in the file. Any values outside of the planned_values in the JSON file are ignored.

Since the current KICS won't make use of the extra info for the resource changes and output changes, would you be able to suggest where that data could be used? like some specific custom queries, etc ...

Correct me if I understood wrong but do you want us to include the data of the resource changes and output changes in the payload data, correct?

From what I could see the resource changes and output changes contains the information for which resources were created/updated/deleted and what values inside of them changed.

[bigdatasourav]

Hey @cxMiguelSilva, can we use this parser, which supports multiple file formats to parse plan files? If no, then what purpose does it have if it doesn't handle different file types? Issue Reference.

[cxMiguelSilva]

Hi @bigdatasourav, I am not sure I was able to properly understand the issue you are facing.
KICS currently only supports JSON output plan files. Could you specify what other file types are you looking for?

[JoaoAtGit]

hi @Subhajit97 ,
How are you?
Internally we decided not to proceed with this request, because after analysis was possible to conclude that could introduce false positive results.