Attempting to forecast command values from Linux trace

[theoctopusride]

Hi,

I am trying to take a trace from a Linux device and use it to forecast future command values. A trace of a Linux device can take up a lot of space depending on how you are using the device while tracing. I am now using a trace that was taken over the course of 48-72 hours that has 6 million lines of activity. Each of these lines has a timestamp associated with it as well as other information: Latency (ms), size in bytes, and Command. I am interested in training on the trace and then forecasting future commands: Write, Read, Hibern8, Sync Cache, etc.

I've used a Vanilla LSTM from Keras to predict future values, but would like to try something from Flow Forecast if possible.

For this project:
I will need to be able to predict multiple steps at a time.
At each step, I will need to forecast the percent likelihood of each command: Read, Write, Hibern8_Enter, Hibern8_Exit, Sync Cache, etc.

Fake Example Data

Timestamp        Latency (ms)        Size                Command
12:01:00.08530   0.00136             4096                Write
12:01:00.09279   0.00278             8192                Write
12:01:00.09435   0.01097             4096                Read

Please let me know if you think you can point me in the right direction or if additional information is needed.

Thanks

[theoctopusride]

Just a comment after 2 weeks of no response - any help on this topic is greatly appreciated.

[isaacmg]

Sorry for the delayed response. So you are trying to classify what future timestamps commands will be based on prior commands + some other data? I'm a bit confused? What is the format of data you are currently passing into your LSTM and what are you currently getting out of it?

[theoctopusride]

Hi Isaac, I appreciate your help with this.

I am mainly interested in forecasting the future command values, but in the process of using the LSTM, I am getting a forecast or prediction for all variables (including the commands).

I have the commands as one-hot encoded columns and have used a min-max scaler for the other values. Here's an example of 5 timestamps worth of training data or what I'm passing in:

Currently, the algorithm looks at 18 timestamps worth of data and attempts to predict the next 18 timestamps, commands, etc.. Below is an example of what I'm getting out. In the example, all commands are forecasted/predicted to be Read cmds.

[isaacmg]

In that case you probably want to approach it as a forecasting problem and feed in those values in our standard format.