Skip to content

Crashes with "KeyError: 'entries'" #9

Description

@Fusl

axeman pulled down a few TBs of CSVs from various CTLs and then crashed:

[450] Finished, writing CSV...
[450] CSV /tmp/certificates/ct2.digicert-ct.com_log/72408700-72408766.csv written!
[450] Parsing...
[450] Finished, writing CSV...
[450] CSV /tmp/certificates/ct2.digicert-ct.com_log/72407530-72407596.csv written!
[450] Parsing...
[450] Finished, writing CSV...
[450] CSV /tmp/certificates/ct2.digicert-ct.com_log/72410455-72410521.csv written!
[450] Parsing...
[450] Finished, writing CSV...
[450] CSV /tmp/certificates/ct2.digicert-ct.com_log/72410195-72410261.csv written!
[450] Parsing...
[450] Finished, writing CSV...
[450] CSV /tmp/certificates/ct2.digicert-ct.com_log/72412535-72412601.csv written!
[450] Parsing...
[450] Finished, writing CSV...
[450] CSV /tmp/certificates/ct2.digicert-ct.com_log/72414160-72414226.csv written!
[450] Parsing...
[450] Finished, writing CSV...
[450] CSV /tmp/certificates/ct2.digicert-ct.com_log/72415460-72415526.csv written!
[450] Parsing...
[450] Finished, writing CSV...
[450] CSV /tmp/certificates/ct2.digicert-ct.com_log/72415850-72415916.csv written!
[450] Parsing...
[450] Finished, writing CSV...
[450] CSV /tmp/certificates/ct2.digicert-ct.com_log/72417410-72417476.csv written!
[450] Parsing...
[450] Finished, writing CSV...
[450] CSV /tmp/certificates/ct2.digicert-ct.com_log/72419880-72419946.csv written!
[450] Parsing...
[450] Finished, writing CSV...
[450] CSV /tmp/certificates/ct2.digicert-ct.com_log/72419360-72419426.csv written!
[INFO:root] 2019-10-17 06:47:18,526 - Completed DigiCert Log Server 2, stored at /tmp/ct2.digicert-ct.com_log.csv!
[INFO:root] 2019-10-17 06:47:18,526 - Finished downloading and processing ct2.digicert-ct.com/log
[INFO:root] 2019-10-17 06:47:18,526 - Downloading certificates for DigiCert Yeti2019 Log
[INFO:root] 2019-10-17 06:47:30,526 - Starting processing coro and process pool
[1] Making dir...
[INFO:root] 2019-10-17 06:47:31,057 - Getting things to process...
[INFO:root] 2019-10-17 06:47:31,058 - Queue Status: Processing Queue Size:0 Downloaded blocks:50/1130984 (0.0044%)
Traceback (most recent call last):
  File "/usr/local/bin/axeman", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/axeman/core.py", line 295, in main
    loop.run_until_complete(retrieve_certificates(loop, concurrency_count=args.concurrency_count))
  File "uvloop/loop.pyx", line 1417, in uvloop.loop.Loop.run_until_complete
  File "/usr/local/lib/python3.7/site-packages/axeman/core.py", line 112, in retrieve_certificates
    await download_tasks
  File "/usr/local/lib/python3.7/site-packages/axeman/core.py", line 53, in download_worker
    for index, entry in zip(range(start, end + 1), entry_list['entries']):
KeyError: 'entries'

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions