UnicodeDecodeError

[c4inju]

I have problem when I run command: python3 ../pwd_guess.py --config-args guess_len8_config.json
Issuse:
Traceback (most recent call last):
File "../pwd_guess.py", line 2585, in
main_entry_point()
File "../pwd_guess.py", line 2582, in main_entry_point
main_bundle()
File "../pwd_guess.py", line 2578, in
main_bundle = lambda: main(args)
File "../pwd_guess.py", line 2486, in main
config, args = read_config_args(args)
File "../pwd_guess.py", line 2478, in read_config_args
config = ModelDefaults(config_args['config'])
File "../pwd_guess.py", line 382, in init
self._intermediate_data = self._read_intermediate_data()
File "../pwd_guess.py", line 393, in _read_intermediate_data
info_as_str = info.read()
File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 98: ordinal not in range(128)

Please help me!
Thanks

[c4inju]

I find some problem when I run: python3 pwd_guess_unit.py

.....................WARNING:root:Unknown config format: .cfg. Defaulting to JSON
...................................ERROR:root:Cannot find intermediate data 'empirical_weighting'. Looking in :memory:
..ERROR:root:Cannot find intermediate data 'empirical_weighting'. Looking in /tmp/tmp1rx0d1oo
.ERROR:root:Cannot find intermediate data 'empirical_weighting'. Looking in /tmp/tmpf6yt7azg
.ERROR:root:Cannot find intermediate data 'empirical_weighting'. Looking in /tmp/tmpuuw6dji7
.ERROR:root:Cannot find intermediate data 'empirical_weighting'. Looking in /tmp/tmp3bseeef7
........./usr/lib/python3.6/unittest/case.py:605: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/tmp93yae1vi' mode='w' encoding='ANSI_X3.4-1968'>
testMethod()
..s/usr/lib/python3.6/unittest/case.py:605: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/tmptwh2llcw' mode='w' encoding='ANSI_X3.4-1968'>
testMethod()
./usr/lib/python3.6/unittest/case.py:605: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/tmpuui60nje' mode='w' encoding='ANSI_X3.4-1968'>
testMethod()

[tianhao199703]

I got the same problems:

File "../pwd_guess.py", line 393, in _read_intermediate_data
info_as_str = info.read()
File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 98: ordinal not in range(128)

i try to add "encoding='latin1'"when open file, however, i got another problem:
Traceback (most recent call last):
File "../pwd_guess.py", line 2585, in
main_entry_point()
File "../pwd_guess.py", line 2582, in main_entry_point
main_bundle()
File "../pwd_guess.py", line 2578, in
main_bundle = lambda: main(args)
File "../pwd_guess.py", line 2486, in main
config, args = read_config_args(args)
File "../pwd_guess.py", line 2478, in read_config_args
config = ModelDefaults(config_args['config'])
File "../pwd_guess.py", line 382, in init
self._intermediate_data = self._read_intermediate_data()
File "../pwd_guess.py", line 397, in _read_intermediate_data
return json.loads(info_as_str)
File "/home/user/anaconda3/lib/python3.7/json/init.py", line 348, in loads
return _default_decoder.decode(s)
File "/home/user/anaconda3/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home/user/anaconda3/lib/python3.7/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I belive that intermediate_data.sqlite is the reason.
have you fixed that?
plz let me know if u solved the problem, thanks

[m33x]

Your problem is:

"UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 98: ordinal not in range(128)"

Aka, you are trying to use non-ASCII passwords, e.g., passwords that include special characters like for example German umlauts (öäüß).

Solution: Remove non-ascii passwords from your training/evaluation dataset and you should be fine.

[tianhao199703]

Your problem is:

"UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 98: ordinal not in range(128)"

Aka, you are trying to use non-ASCII passwords, e.g., passwords that include special characters like for example German umlauts (öäüß).

Solution: Remove non-ascii passwords from your training/evaluation dataset and you should be fine.

Thanks for your reply. it seems that this file "intermediate_data.sqlite" cause the problem, do you know the possible reason?

File "../pwd_guess.py", line 382, in init
self._intermediate_data = self._read_intermediate_data()
File "../pwd_guess.py", line 393, in _read_intermediate_data
info_as_str = info.read()
File "/home/user/anaconda3/lib/python3.7/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe6 in position 98: invalid continuation byte

[m33x]

As said before:

„UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe6 in position 98: invalid continuation byte“

0xe6 is not a valid utf-8 character. you are using passwords that are not ASCII.

[JaronMink]

I have this error as well with the pre-trained network with the preset configs.
This occurs for me while trying to read the provided "intermediate_data.sqlite" under /pre_build_networks/

File "../pwd_guess.py", line 382, in init
self._intermediate_data = self._read_intermediate_data()
File "../pwd_guess.py", line 393, in _read_intermediate_data
info_as_str = info.read()
File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 98: ordinal not in range(128)

[JaronMink]

Looks like commit de2f63b took out sqlite representation for intermediate data but the pre-trained network still come with sqlite files and thus causes issues when attempting to read.