Commits · fed19d55914038fdda740a789123eacbf845480f · a / Erigon · GitLab

good morning!!!!

Snippets Groups Projects

Mar 22, 2022

Refactor PoS downloader (#3717) · 74a7d7c7

Andrew Ashikhmin authored 3 years ago

* Remove blockHeight arg from FeedHeaderPoW

* Revert "Remove blockHeight arg from FeedHeaderPoW"

This reverts commit 9bf92921db11cd4e13386fcee29f30241d070fc6.

* Move PayloadMessage & ForkChoiceMessage to package engineapi

* RequestList instead of newPayloadCh & forkChoiceCh

* Introduce request status

* Send reply only when request status is New

* Move BeaconRequestList to HeaderDownload

* Don't SetFetching when PoS (Fetching means handling newBlockHashes)

* Merge Syncing & Synced into DataWasMissing

* Introduce SyncStatus

* onlyNew arg in WaitForRequest

* Move waitingForBeaconChain into RequestList

* Interrupt instead of skipCycleHack

* Introduce timeout for PoS headers

* Split downloadMissingPoSHeaders

* Move StartPoSDownloader into HeaderDownload

* Move Stopping interrupt to StartPoSDownloader

* Move stopping PayloadStatus back to HeadersPOS

* cleanUpPoSDownload

* Post-merge fix

* TestBogusForkchoice

* TestPoSDownloader

* requestStatus in attemptPoSDownload

* Broadcast in SetStatus

* Cosmetics

* attemptPoSDownload -> schedulePoSDownload

* Demote some logs to Trace

74a7d7c7

Mar 21, 2022
- [erigon2] Fix rebuilding state (#3747) · ce15968d
  ledgerwatch authored 3 years ago
  
  Unverified
  
  ce15968d
- Snapshots: script to auto-commit torrent hashes (#3745) · 586ad297
  Alex Sharov authored 3 years ago
  
  Unverified
  
  586ad297
Mar 20, 2022

[erigon2] forward changesets to optimise access (#3743) · 5e4655cd

ledgerwatch authored 3 years ago


* [erigon2] forward changesets to optimise access

* Update to erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

5e4655cd

Mar 19, 2022

[erigon2] Separate handover timing (#3736) · d5fb8f7d

ledgerwatch authored 3 years ago


* Separate handover timing

* Update

* Corrected handover time calculation

* Not use compression when aggregate

* Update to latest erigon-lib

* Update to erigon-lib main

* Update

* Disable reproducible builds

* Restore github actions

* Restore github actions

* Revert "linter to support go 1.18 (#3739)"

This reverts commit 1fd434d3.

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

d5fb8f7d

Mar 18, 2022

[erigon2] Optimise handover to the background merge (#3735) · ec4b5e6c

ledgerwatch authored 3 years ago


* [erigon2] Optimise handover to the background merge

* Update to erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

ec4b5e6c

[erigon2] Optimise Huffman decoding (#3727) · c47cf1fd

ledgerwatch authored 3 years ago


* Update to erigon-lib

* Fix for block number

* Update

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>

c47cf1fd

Snapshots: grpc event (#3732) · 7bf87e95
Alex Sharov authored 3 years ago

Unverified

7bf87e95

Mar 17, 2022
- Pool cost fix (#3726) · d0db4ed6
  Alex Sharov authored 3 years ago
  
  * save * Update to erigon-lib main Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
  Unverified
  
  d0db4ed6
- Mdbx: WriteMap fallback on open error (#3711) · 956a57f7
  Alex Sharov authored 3 years ago
  
  Unverified
  
  956a57f7
Mar 16, 2022
- Pick up erigon-lib fix for h2048 (#367) (#3713) · 3dbd6bef
  Andrew Ashikhmin authored 3 years ago
  
  Unverified
  
  3dbd6bef
- Open erigon-lib use same version of crypto package with erigon #3710 (#3710) · 1c1190ae
  Alex Sharov authored 3 years ago
  
  Unverified
  
  1c1190ae
- up cobra version (#3709) · e3fb3aef
  Alex Sharov authored 3 years ago
  
  Unverified
  
  e3fb3aef
Mar 13, 2022

[erigon2] history indices based on Elias Fano (#3679) · 8b00636e

ledgerwatch authored 3 years ago


* [erigon2] history indices based on Elias Fano

* Tidy

* Update

* tx lookup to support blocTo in integration

* Update to fix

* Update

* Fix

* Update to erigon-lib main

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

8b00636e

Mar 12, 2022
- Snapshots: retire testing tool (#3684) · f03d08c5
  Alex Sharov authored 3 years ago
  
  Unverified
  
  f03d08c5
Mar 10, 2022

Discovery: split node records to a sepatate DB table (#3581) (#3667) · 04f07a0f

battlmonstr authored 3 years ago

Problem:
QuerySeeds will poke 150 random entries in the whole node DB and ignore hitting "field" entries.
In a bootstrap scenario it might hit hundreds of :lastping :lastpong entries,
and very few true "node record" entries.
After running for 15 minutes I've got totalEntryCount=1508 nodeRecordCount=114 entries.
There's a 1/16 chance of hitting a "node record" entry.
It means finding just about 10 nodes of 114 total on average from 150 attempts.

Solution:
Split "node record" entries to a separate table such that QuerySeeds doesn't do idle cycle hits.

04f07a0f

Discovery: throttle node DB commits (#3581) (#3656) · ca973254

battlmonstr authored 3 years ago

UpdateFindFails/UpdateLastPingReceived/UpdateLastPongReceived events
are causing bursty DB commits (100 per minute).

This optimization throttles the disk writes to happen at most once in a few seconds,
because this info doesn't need to be persisted immediately.

This helps on HDD drives.

ca973254

Add system-txs to begin end of block (#3654) · b003d078

Alex Sharov authored 3 years ago

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* up torrent

* up torrent

* up torrent

* up torrent

* up torrent

* up torrent

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

b003d078

Mar 09, 2022

[erigon2] Allow uncompressed words (#3655) · 3cab0c78

ledgerwatch authored 3 years ago


* Optimise compressor

* Update erigon-lib

* Update

* Update

* Update

* Update to erigon-lib main

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

3cab0c78

Mar 08, 2022

rpcdaemon: limit amount of read transactions (#3637) · 0b2ce6f8

Alex Sharov authored 3 years ago

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* a

* a

* a

* a

* a

* a

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

0b2ce6f8

Mar 05, 2022
- up torrent version (#3647) · 6d9baab5
  Alex Sharov authored 3 years ago
  
  Unverified
  
  6d9baab5
Mar 04, 2022
- kv: forAmount noop on 0 amount (#3644) · 902824a8
  Alex Sharov authored 3 years ago
  
  Unverified
  
  902824a8
- mdbx v0.11.5(#3624) · 230a177d
  Alex Sharov authored 3 years ago
  
  Unverified
  
  230a177d
Feb 28, 2022

Added Engine Authentication [JWT] (#3531) · cb8aacae

Giulio Rebuffo authored 3 years ago


* jwt

* fuuuuture

* added auth

* merge

* merge

* Update jwt to the latest version

* ops

* comments

* cleanup

* bad

* gut

* maybe

* mod sum

Co-authored-by: yperbasis <andrey.ashikhmin@gmail.com>

cb8aacae

Feb 24, 2022
- PrevRandao Renaming (#3616) · 99f688c3
  Giulio Rebuffo authored 3 years ago
  
  * PrevRandao Renaming * lint
  Unverified
  
  99f688c3
- logger version up (#3608) · f685d6bf
  Alex Sharov authored 3 years ago
  
  * snapshots: fix indexing of recently merged range * logger version up
  Unverified
  
  f685d6bf
- snapshots: ensure tmpdir exists (#3602) · 3067cf99
  Alex Sharov authored 3 years ago
  
  Unverified
  
  3067cf99
Feb 21, 2022
- [erigon2] ETL optimisations (#3550) · e784f2b8
  ledgerwatch authored 3 years ago
  
  Unverified
  
  e784f2b8
Feb 19, 2022

[erigon2] fix history creation (txNum confusion) (#3547) · 80d7734e

ledgerwatch authored 3 years ago


* More tracing

* More tracing

* More tracing

* Keep correct txNum

* Keep correct txNum

* Update to tracing

* Off by one

* Print memory info

* Quicker startup

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

80d7734e

Feb 18, 2022
- last version of kanzi-go (#3543) · 952085a8
  Alex Sharov authored 3 years ago
  
  Unverified
  
  952085a8
- more_stable_grpc_err_check (#3542) · 2755af23
  Alex Sharov authored 3 years ago
  
  Unverified
  
  2755af23
- RPCDaemon: to start with disabled txpool (#3539) · 3de83e2d
  Alex Sharov authored 3 years ago
  
  Unverified
  
  3de83e2d
- Use class dir.Rw - to separate Ro and Rw access to snapshotDir (#3534) · 226982d2
  Alex Sharov authored 3 years ago
  
  * save * save * save * save * save * save * save
  Unverified
  
  226982d2
Feb 17, 2022

Embed rpcdaemon fixes (#3535) · e74b9ef9
Alex Sharov authored 3 years ago
```
* save

* save
```
Unverified

e74b9ef9

[erigon2] Thin commitment (2nd attempt) (#3515) · c5b4d3e3

ledgerwatch authored 3 years ago


* Another fix for history files

* Fix NPE

* Another fix

* Proper closing sequence

* Remove first byte insert flag

* Fixes

* Update

* Add traceblock to erigon2 command

* Update

* Update

* Update

* Update

* print panic

* Fix history reads

* Update

* Update locks

* Update

* Fixes

* Set changesets and commitments upfront

* Logging instead of printing

* Fixes in aggregator and recsplit

* Update to erigon-lib

* Remove empty commitments

* Update

* Update

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>

c5b4d3e3

Feb 16, 2022
- rpc daemon embed (step 3) (#3519) · 6d9ea162
  Alex Sharov authored 3 years ago
  
  Unverified
  
  6d9ea162
Feb 13, 2022
- [erigon2] Fix history bug, optimise bitmap search (#3510) · dffcc57e
  ledgerwatch authored 3 years ago
  
  * Fix history bug, optimise bitmap search * Optimise SeekInBitmap64 * Update erigon-lib Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
  Unverified
  
  dffcc57e
- return EOF on direct stream end (#3504) · 7566e1db
  Alex Sharov authored 3 years ago
  
  Unverified
  
  7566e1db
Feb 12, 2022

Tools for gathering data on erigon2 history and bitmaps (#3488) · 2074b985

ledgerwatch authored 3 years ago


* Tools for gathering data on erigon2 history and bitmaps

* Always start with 0

* Always start with 0

* Clean up commitment files, trace block

* traceblock

* Add tracing to checkChangeSets

* Fixes to DeleteAccount

* Update to latest erigon-lib

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

2074b985

path -> filepath (path package is for urls) (#3493) · c9cee7a8
Alex Sharov authored 3 years ago

Unverified

c9cee7a8