Fixes for zeekctl error loading dhcp-fp.zeek; dhcp_unknown.py updated for python3; standalone kyd.py working;

[fubar2]

Thanks very much for creating this zeke plugin - it's exactly what I need.
This PR fixes things for me, but

• I have not tried this PR any other zeek sites other than my own very new one
• I am very new to zeek and so not in the least confused by facts when it comes to zeek language scripting.

Those disclosures notwithstanding, using ZeekControl Version 2.0.0-25 and zeek version 3.1.0-dev.314-debug compiled from current git repo today, after

zkg install kyd
zkg load kyd

No installation problems but check throws an error:

[ZeekControl] > check
zeek scripts failed.
error in /usr/local/zeek/share/zeek/site/packages/./kyd/./dhcp-fp.zeek, line 22: syntax error, at or near "dbfile"

Of course it doesn't load and no dhcp-fp.log appeared.

After the change in this PR, I see:

[ZeekControl] > check
zeek scripts are ok.

and now I see that the script has been loaded in loaded scripts and I am seeing dhcp-fp.log at last!

I also adjusted kyd.py so it runs standalone - "filename" was not being defined anywhere. Also dhcp-unknown now works (for me) in python3. Goodbye python2. Print statements and use of "unicode" have been updated and if no input file is provided on the CL, it will trawl through every zeek log directory - path is hardcoded :( parsing all the dhcpfp logs compressed or not.
Finally, a cosmetic fix to the metadata.

TODO: fix dhcp-unknown so it checks the local database and updates where new signatures are found.

I hope this works for everyone else. Thanks again!

[fubar2]

Sorry for all the commits but I had to use zkg locally and it insists on the repo being in sync. It took me a while to figure out that the two copies of dhcp-db.txt in your repository are different. The one in the zeek folder had some records with a bogus tab before the newline so loading the file into the table in zeek failed for me. Debugging this stuff is tricky.
Now dhcp-fp.zeke seems to work for me with the full path for dhcp-db.txt but otherwise fails to load that file. I have no idea what the problem with the unadorned file name as the path to the data file is but it will not load for me.

[fatemabw]

Hi Ross,

Thank you for fixing the Python scripts and apologies for the syntax error in zeek script.
I have fixed the dbfile option variable in the zeek script and it should work fine now.

I have to review more the python script changes to be able to merge it into the master branch.

Fatema

[fubar2]

Hi Fatema,
Thanks for taking the time to review this. Ah yes - option not global - thanks! That works here for me now and I've updated my fork. I have a lot to learn about zeek script.

updated: your fix does not work here for me - my current fork has the absolute path and works in my setup as described above. No idea if it's just me and my setup but if I just use the filename, I get a reporter.log with a message about failing to load.

[fatemabw]

Hi Fatema,
Thanks for taking the time to review this. Ah yes - option not global - thanks! That works here for me now and I've updated my fork. I have a lot to learn about zeek script.

updated: your fix does not work here for me - my current fork has the absolute path and works in my setup as described above. No idea if it's just me and my setup but if I just use the filename, I get a reporter.log with a message about failing to load.

The dhcp-db.txt file has to be in the current folder where your dhcp-fp.zeek file is. And if not, you have to provide the absolute path as mentioned in the comment in the script. If you have installed this plugin, both - the zeek script (dhcp-fp.zeek) and the DHCP db input file (dhcp-db.txt) should be residing under ~kyd/zeek/ folder.

[fubar2]

If you have installed this plugin, both - the zeek script (dhcp-fp.zeek) and the DHCP db input file (dhcp-db.txt) should be residing under ~kyd/zeek/ folder.

They are! However, I did install/uninstall it dozens of times with and without zkg so I'll try again with a fresh zeek installation and let you know - I may well have done something silly in the process of testing...

update:
Clean install latest git clone of zeek and zkg install kyd
I'm still seeing the same reporter.log message

1578360458.991022 Reporter::WARNING dhcp-db.txt/Input::READER_ASCII: Init: cannot open dhcp-db.txt (empty)

If I adjust the path in /usr/local/zeek/spool/installed-scripts-do-not-touch/site/packages/kyd/dhcp-fp.zeek to the full path it loads and works fine.

[fubar2]

Hi Fatema,
While trying to figure this out, I noticed that nearly all the example scripts in
https://github.com/michalpurzynski/zeek-scripts/search?q=source&unscoped_q=source use full paths as source, so maybe my fix isn't that odd - as I said I know nothing about zeek script....