|
|
|
|
Changelog for libbitcoinconsensus0_sv-1.0.8b-lp155.1.9.x86_64.rpm :
* Tue May 11 2021 marecAATTdetebe.org- Update to version 1.0.8b: * Updates to release notes. * Children of non-standard transactions must go via the orphan pool. * Prevent spurious double-spend callbacks. * reintroduced cfile_util.cpp to cmake configuration * Handling duplicate and double-spend transactions received to the mempool during the reorg. * Add allowance for cancellation budget timing issues on slow windows machines, part 2. * Improve error reporting * Removed test for too-long-mempool-chain exception * changed protocol-id from \'dust_return\' to \'dust\' * Allow dust return transactions with unconfirmed inputs and add a protocol_id * Fix: nInputs was not logged correctly in BlockConnector * Adjusted default value for ancestor count and orphan pool size * Update release-notes.md. * Rename methods and variables. * Remove maxcollectedoutpoints configuration param. * Add new checkpoint block. * Add entry to release notes. * Flush metrics on shutdown * src/CMakeLists.txt: Add missing headers and fix sorting and indentation * src/CMakeLists.txt: Remove cfile_util.{cpp,h} from rpcclient library * fixed typo in log message * Fix typo * Disallow 0 size for some params. * Correction to release notes. * Release notes for 1.0.8 * Varint encoding/decoding. * Script for parsing and plotting histograms * fixed typo * Add instructions to increase validation time for wall clock validation timing * Add allowance for cancellation budget timing issues on slow windows machines. * Use wait_for_ptv_completion() to increase stability of functional tests. * Add wait_for_ptv_completion utility function. * Add waitforptvcompletion rpc interface. * Replace std::optional > with plain T * * Fix typo in help. * Updates to man pages * Set version number to 1.0.8 * Updates for release notes. * Collect children of the already collected orphans to the next batch. * target as header now using 160 single hex string with each entry reversed * Refactor method of calculating ancestors count * Fix Windows errors in double-spend functional tests * added missing sync_blocks in bsv-soft-rejected-blocks.py * Catch file-system errors in destructors. * Updates to manual pages. * Bump version numbers to 1.0.7.1 * removed unused parameter nBlockHeight * CTxMemPool::RemoveForBlock now correctly removes descendants of the transactions in CONFLICT * Fix unit test that assumes the UA string has a version number of the form X.Y.Z * Update definition of a prod-tag * Fix: we should detach a CoinsDB view before re-locking cs_main * Move only: split BlockConnector::Connect function * Move only: move ConnectBlock content to a separate class * Fix: mark CBlockIndex instance as dirty on construction * fixed sprintf format specifier ld instead of l for long signed int * Decrease PTV sleep from 100ms to 10ms. * Test that the part of the chain containing a slow transaction will get processed by a low priority thread. * fix bsv-broadcast_delay.py * Allow chained transactions to use remaining time of ancestor validation * use thread CPU clock for transaction validation cancellation * Use std::deque for mStdTxns as collectTxns() erases from the start. * Don\'t split long chains over validation timeout. * Propagate validation error along the chain * Improve detection of chains * Check outputs only to distinguish known transactions from orphans * reduce wait time for simultaneous coin requests * ratelimit COrphanTxns trimming by adding a 10% hysteresis * Fix COrphanTxns to properly handle unlimited standard transaction size * Add a test how rpc sendrawtransactions handles chains with bad transactions in the middle of the chain * Add timing metrics to validation * A simple threadsafe histogram collection and logging * added json result field \'targetType\' * Extended interface of getmerkleproof2 with params \'includeFullTx\', \'targetType\' and \'format\' * std::chrono compatible thread_clock and CancellationSource * Add RPC getorphaninfo * Implement __hash__ and __eq__ for COutPoint to support maps and sets * added rpc function getmerkleproof2 in line with TSC * New option for sendrawtransactions to list unconfirmed ancestors * added new parameter dustlimitfactor * Add buildconfig parameter to be able to set directory that contains build on Windows * Add some additional functional test scenarios. * disable IPv6 test if IPv6 loopback not enabled on platform * add scenario when callback service has no x-bsv-dsnt in header * add new tests to SOLO_TESTS * add scenario with multiple callback services * add scenario with callback service with IPv6 address * update service callback with configuration for response time * add test for double spend when callback service behaves unexpectedly * add status property to callback service to enable mocking client and server error * add functional test for double spend reporting * add mock callback service for double spend reporting and functional test * Invalid transactions callback that can be registered in publisher * Link bitcoincli lib to bitcoind. * Double spend processing unit tests. * Double spend callback message extraction. * Remove redundant DSA HTTP client code. * Update HTTP client classes for communicating with dsattempt endpoints. * Instantiate a DSAttemptHandler. * Add new DSAttemptHandler class. * A time limited blacklist. * Add helper class for serialising double-spend transactions to disk. * Add convenience constructor for leaky bucket to create part filled. * Add DSCallbackMsg class. * Split out CheckInputs(). * Add contains() to the limitedmap interface. * Add fScriptsChecked transaction validation flag. * Add a dedicated logging category for the double-spend processing. * Add config parameters for double-spend processing. * Bump openssl Gitian version to 1.1.1i * Bump zeromq Gitian version to 4.3.4 * Bump libevent Gitian version to 2.1.12 * Update Gitian boost version to 1.75 * Remove not needed package descriptions. * add verifyscript RPC method * core dumps documentation added * added test for donation transactions * Fix a possible false negative result during querying the processing queue. * Ignore consolidation factor for donations * Store -streamsendratelimit in config. * Store -blockdownloadmaxparallelfetch in config. * Store -blockdownloadslowfetchtimeout in config. * Store -blockdownloadwindow in config. * Store -blockstallingtimeout in config. * Store -blockstallingmindownloadspeed in config. * add processing_block_index.h to Makefile.am * Config interface is divided into Config which contains all getters and thread safe setters and into ConfigInit which contains all setters. * Making GlobalConfig class thread safe for values that can be changed after the startup: maxGeneratedBlockSize, maxBlockSize and maxSendQueuesBytes. At creating new block in journaling_block_assembler.cpp max block size was computed for every transaction in block. To prevent repeatedly read lock of configMtx mutex, calculation is moved outside while loop. * Introducing with...() member functions in BlockValidationOptions class. * maxsendbuffer value set to previous default in functional test * correction of functional test error message * remove unnecessary const_cast * passing DirtyBlockIndexStore& to methods that need it instead of storing it in every CBlockIndex object * add BlockIndexStore documentation * make DirtyBlockIndexStore\'s methods private and add general class documentation * move ForceClear and ForceLoad to new class BlockIndexStoreLoader * remove redundant LOCK from blockchain RPC * move implementation for creating CBlockIndex objects in container to BlockIndexStorage * removed method CBlockIndex::LoadFromPersistentData * Always return BlockIndexStore::mBestHeader value * Move setDirtyBlockIndex into a separate class * Remove or limit the scope of cs_main locks * Move LoadBlockIndexGuts from block tree to BlockIndexStore * Move pindexBestHeader to BlockIndexStore * NotifyHeaderTip pindexHeaderOld should have its own mutex * Minimal encapsulation of mapBlockIndex into a BlockIndexStore class * revert DiskBlockMetaData to be struct * delete assignment and copy constructor in CBlockIndex * minor improvements (usage of initializer lists, const correctness) * document CBlockIndex * Improved CBlockIndex GetPrev/GetSkip const correctness * Const correctnes of CBlockIndex usage * add global array of mutexes for locking CBlockIndex * move implementation of GetAncestor into const version of the method * make some methods in CBlockIndex const * Minor AcceptBlockHeader cleanup * Make CBlockIndex nTx private and protected by blockIndexMutex * Make CBlockIndex nTimeMax private and thus immutable * Make CBlockIndex nHeight private and thus immutable * Make CBlockIndex pprev and pskip private thereby immutable by api * Make CBlockIndex nChainWork private and protected by a lock * Protect nChainTx and nSequenceId in CBlockIndex by a mutex * Make permanently immutable block index members private * Encapsulate CBlockIndex construction * Move part of index population from LoadBlockIndexDB to internal function * make nFile CBlockIndex variable private * Hide some BlockStatus functions from outside world * Making disk block data and meta data access more atomic * make CDiskBlockMetaData\'s members diskDataHash and diskDataSize protected * Better encapsulate SetDiskBlockMetaData * make SetDiskBlockMetaData private * make nStatus member of CBlockIndex private and lock methods * move implementation of static methods to member methods * move GetBlockPos() usages to CBlockIndex * Remove unneeded SetNull function from CBlockIndex * lock soft rejection methods with blockIndexMutex * introduce blockIndexMutex on CBlockIndex and use it in new member methods * make CBlockIndex\'s GetUndoPos method private * remove redundant overload of IsUAHFEnabled and IsDAAEnabled methods * move CDiskBlockIndex to a separate file * refactor CDiskBlockIndex class so that it does not inherit from CBlockIndex * move implementation of static methods into member methods * move methods that use activeChain\'s non-final data to CBlockIndex * move block index related classes to new file block_index.h and block_index.cpp * comment out soft block orphaning test * add test that rejects headers message with missing previous block * functional tests for soft rejected blocks * soft rejected blocks * helper class BlockIndexWithDescendants * Rollback changes to libatomic linking requirements. * Initalize secp256k1 library before main() to avoid data races. * Initialize static variable on construction in ParseScript. * prevent collision between donotcheck tx with same relayed tx also in sendrawtransactions (vs sendrawtransaction) * prevent collision between donotcheck tx with same relayed tx * Replace sorted associative containers by unordered counterparts to optimise internal storage and lookup. * remove explanation comments from added header lines * Intermediate commit for review, header files with commented simplest explanation what includes the added header * Fix file sorting * Unify whitespace in the lists * config.cpp is compiled twice * utilstrencodings.cpp is compiled twice * BlockFileAccess FlushBlockFile should silently ignore missing empty files * TCP_MAXSEG is not supported on Windows versions < 10 * Move CMerkleTree out of the consensus library. * Update UTs. * Remove protection, which prevents all found orphan txs (from the same parent) to be reprocessed in parallel. The protection is no longer required, due to the fact that the mempool doesn\'t track properties (size/count) for descendant transactions. * Add a leaky bucket algorithm class. * make DateTimeFormatter thread_local * Tidy logging categories. * Use new net logging categories. * Split NET logging category. * Fix: CBlockFileInfoStore::FlushBlockFile can dereference end of vector * perform cleanup in blockfile_reading_tests * correct bsv-getdata functional test so that assumes automatically received blocks * Reduce wasted TCP segments. * Reset release notes ready for next release. * added convenience class for using random generator * Access the thread-local thread name through static local in a function to make construction order predictable. * DateTimeFormatter * Fix functional tests * Remove all tor related usages, references * Remove torcontrol files (this currently does not build because some functions are still used in other places and will be removed in next commit) * Remove tor related unit tests * add missing test to makefile * suppress MSVC warning C4834 regarding nodiscard attribute on basic_string.at() * fix MSVC specific warnings * suppress warning due to incompatible function types * resolve -Wsign-compare warning * replace deprecated method RAND_screen with RAND_poll * suppress warning due to string truncation * suppress warning in prevector due to maybe uninitialized variable * create macros for disabling warnings * remove unused method in miner_tests * remove unused variables * update CMakeLists.txt with warnings settings for different platforms * Wait to get proper connection count instead of just checking it once. * Post review rename of GetBlockFile to OpenBlockFile * post review CAutoFile API cleanup * LoadExternalBlockFile should accept UniqueCFile instead of raw FILE pointer * more extensive post review CDiskBlockPos and CDiskTxPos api cleanup * minor post review CDiskBlockPos and CDiskTxPos api cleanup * post review CBlockFileInfo api cleanup * Introduction of internal serialization locking for block file access * Introduce stricter file opening rules in BlockFileAccess * Replace FILE pointers with CFileHandle in BlockFileStore * Making OpenBlockFile and OpenUndoFile implementation detail * Encapsulation of OpenBlockFile use into block file store * Move block file related part of GetTransaction to block file store * Move CDiskTxPos to a separate file * Move block file pre-allocation code to block file store * Move FlushBlockFile implementation to block file store class * Move CBlockFileInfo into a separate file * Move UndoReadFromDisk to block file store * Move block reader functions to block file store * Move write block related functions to block file store * Initial move of block file related code to a separate file * Minor C FILE use cleanup * Move CDiskBlockPos to a separate file * added missing parenthesis * Fixed two compilation problems for visual studio and clang * use wait_for_getdata correctly in functional tests * replaced std::rand with std::mt1937 * removed \'defaulted Snapshot default ctor implicitly deleted\' warning * removed superfluous semicola * minor issues with signed versus unsigned * fixed minor type mismatches * illegal anonymous struct in class * generated code adds obsolete semicolon after namespace * leveldb uses C++17 syntax but cmake declares the code as C++11 * silencing zero length array warning * Increase transaction processing timeout. * Remove old MaybeSetPeerAsAnnouncingHeaderAndIDs function. * Copy MaybeSetPeerAsAnnouncingHeaderAndIDs and data to BlockDownloadTracker. * Fix failing functional test. * Functional test for parallel block downloads. * Create functional test to check blocks propagate with higher priority. * Send block INV messages over high priority stream. * Apply a sending bandwidth rate limit. * Trigger parallel block download when stall detected. * Add check for inflight block download stalling. * Refactor net_processing to use the BlockDownloadTracker. * Add a BlockDownloadTracker class. * Expose some functions from net_processing. * Move CNodeState out into its own file. * Add a new FT to test long chains of cpfp txs. * When connecting nodes in functional tests, wait for association to be fully established if multistreams are used * removed redundant atomicity JournalingBlockAssember memeber mLastBlockStats * Add maxstackmemoryusageconsensus to rpc getsettings * remove bitcoin_test_fuzzy test * Show proper parameter name in the error message. * removed initialisation via Init function * Thu Apr 22 2021 marecAATTdetebe.org- Update to version 1.0.7.1: * Release notes for 1.0.7.1 * Updates to manual pages. * Bump version numbers to 1.0.7.1 * Fix cs_main CoinsDB lock ordering + perf cleanup * Fix unit test that assumes the UA string has a version number of the form X.Y.Z * Update definition of a prod-tag * Wed Feb 10 2021 marecAATTdetebe.org- Update to version 1.0.7: * Final release notes updates for Dynastic release. * Prevent that mempoolminfee (rolling fee) becomes greater than blockminfee while we still have secondary mempool transactions * Improve error reporting for version messages. * Add version processing functional test. * Preserve the id type in RPC calls in getblock, getblockbyheight, getblocktemplate * Functional test to check persistence of transactions in the transaction database * Fix shutdown ordering race. * Avoid clearing the mempool at startup before mempool.dat is loaded. * Removed obsolete switch --journal-broken-tests * Prevent overflow when calculating new fee-rate after eviction * Handling negative values in the limitacnestorcount and limitcpfpgroupmemberscount * coinbaseValue added to the help message of getminingcandidate * Modify test util function to strip new line * Log parameters at startup set by switches and config file. * Add rpc method to list parameters set by switches and config file. * Avoid dereferencing nullptr in help rpc command * move CInv type to CInv class in mininode * add functional test for GETDATA messages * Fix to ubsan warnings * Fix protoconf parsing in the testing framework. * correct undefined behaviour when obtaining unknown block * Diagnostic script and readme added * Fix: coins cache size should be default limited and minimum limited * Check for null association ID. * Extra error handling for CreateStream and StreamAck. * Add whitespace at the end of strings concatenated by the compiler * Change local variable name. * Bring back forwarding Add and Remove to the mampool txdb. * Get rid of unnecessary const_cast in tests. * Move the salted TxID hasher classes from libconsensus to libcommon to remove link-time dependency. * CORE-1207: format default parameter for -maxcoinsdbfiles * Change the async mempool txdb API to add or remove only one transaction at a time. Add/remove coalescing in the background thread ensures that actual writes to disk are combined into larger batches. * Release notes for 1.0.7 beta. * Updates to manual pages for 1.0.7 * Update version and (C) year. * cmake check target should also build secp256k1 benchmarks, like autotools * CORE-1211: Fix compilation of bench_internal that includes source files in different order than the library * Fixup: eviction score for the groups * Functional test for mempool evictions * Printouts in the check_mempool_equals * Deletion of no longer used variables. * Fix: TxMempoolInfo::GetTx returning reference to temporary * Add test for new streampolicies field in getnetworkinfo() * Add list of available stream policies to output of getnetworkinfo() * Fix: missing include * Fix: replacing unique_lock with scoped_lock in CJournalChangeSet * tsan: fix for mChangeSet access without a lock * Fix: remove missleading std::move * tsan: fix TxMempoolInfo tx atomic access * Check negative values for consolidation parameters * Updated getmempoolinfo help string * Set -maxmempoolsizedisk it be 0 by default and disable help message. * Fixup of the CPFP Group ID * make nSequence member variable from CZMQAbstractPublishNotifier atomic to avoid possible race condition * bsv-journal-mempool-reorg-ordering test should wait for all submitted transactions. * Raised absolute time until the test node is banned * CORE-1188: Add parameter maxcoinsdbfiles * Call reset() instead of release() on the pcoinsTip * Add missing curl examples for rpc calls * Fix examples for bitcoin-cli to work consistently on Windows and Linux * Fix some curl examples for rpc methods * Add a new test for easy setup of associations within the Python test framework. * Classes to support associations. * Provide run_node_with_associations method. * Added unit tests for predicted CTransactionWrapper behaviour. * Save one mutex lock in the mempool txdb worker thread. Add and clarify comments. * Make the transaction wrapper always return the same transaction pointer as long as a copy of the transaction it controls remains in memory, even if the transaction has been stored to disk in the meantime. * CORE-1181: add a unit test for JBA/wrapper transaction aliasing issue * Allow asan to ignore assembly functions * Fix CORE-1180: add missing header * Fix bsv-protoconf test to enable setting INV size to send separately from maxprotocolrecvpayloadlength * Initialize instanceId to prevend uninitialized memory access when opening version 1 mempool.dat * Add documentation constant instead of empty constructor * Introduce CTransactionConflict to describe a conflicting transaction * Introduce CTransactionConflict to describe a conflicting transaction * \"-disablebip30checks\" command line param * Use uniquely named or in-memory mempool transaction databases in tests. * Rework checkJournalNL() so that it never reads transactions from the mempool txdb. * Fix typo. * Adjust expectations in the blockencodings unit tests. * Change CJournalEntry to contain a pointer to the transaction wrapper, not to the transaction itself, so that we don\'t keep unaccounted transactions in memory. * Parallelize RemoveForBlock * Logging time spent in the RemoveForBlock * Using static salt for the hashers. * Handle exceptions in the mempool TxDB worker thread. * ThreadSafeQueueRefactor and OnBlockedCallback implementation * Fix init list in CThreadSafeQueue constructor. * Add logging of blocked methods to CThreadSafeQueue, use it to log in CAsyncMempoolTxDB. * Limit the size of the mempool txdb work queue. * Add a benchmark test for CThreadSafeQueue to evaluate relative performance of batch and single push/pop. * Use CThreadSafeQueue in the async mempool database so that we can limit the queue size. * remove JOURNAL_BROKEN_TESTS * Fix for test that are assuming that zero fee transactions will not be rejected * Delete high priority transactions tests * Remove -blockprioritypercentage * Remove CTransaction::CalculateModifiedSize * Remove CTransaction::ComputePriority * Remove CCoinsViewCache::GetPriority * Remove total chain input for mempool entry * Remove GetPriority from mempool entry * \"startingpriority\" and \"currentpriority\" removed from getmempoolancestors, getmempooldescendants, getmempoolentry and getrawmempool RPC calls * Remove -relaypriority * Enabling -minrelaytxfee to be set to 0 * Fix for Block sync to height 638 timed out * Remove -limitfreerelay * Removed priority form the mempool. * Clean up broken tests hack from the test_runner * Fixup: modified fee for the eviction tracker * Fixup: rawtransactions.py, mempool is filled with transactions which pay enough for mining * txmempool: convert all locks to {} intitialization * Add missing description to DEFAULT_MEMPOOL_MAX_PERCENT_CPFP * Convert all open-coded mempool limit caliculations to the new abstraction * Make maxmempoolsizedisk default be 10 times mempool RAM size * Rework limit unit tests for secondary mempool. * Catch flaky unit tests that add duplicate transactions to mempool * Enforce secondary mempool limit * Add secondary mempool limit as percentage * Convert mempool limits to a class * txinvs access must be protected by mininode_lock on all paths * reindex test duration should be limited with a timeout * prevent calculation of mempool memory usage to underflow * mempool limit unit tests * Track secondary mempool by count and bytes * Fix documentation for -maxmempool * Remove leftover from SigOps counting in the block * Remove and update obsolete comments. * Restore size accounting for cached parents and children in the mempool. * Preventing unnecessary reading from database in the AddToMempoolForReorg * Preventing unnecessary reading from database in the GetTxSnapshot * Preventing unnecessary reading from database in the NotifyEntryAdded and NotifyEntryRemoved * Preventing reading from database when transactions are removed from the mempool * switch from std::map to boost::multi_index_container for mapNextTx * Restore the SaltedOutpointHasher. * removing need to use std::map::lower_bound to be able to use hash map in the mapNextTx * switch to store txiter rather than CTransactionWrapper * in the mapNextTx * Change mapLinks from std::set to std::unordered_set * Fix after review: - Rename the TxDB worker thread. - Avoid race in CTxMemPool::SaveTxsToDisk. - Tweak one more auto variable initialization. * Document thread safety constraints of CMempoolTxDB. * Update externally visible disk usage and transaction count only when the modification of the mempool txdb succeeds. * Update initializations etc. as per coding style, no functional change. * Add unit tests for asynchronous coalescing adds and removes. * Implement add/remove coalescing in the mempool transaction database background thread. * Add coalescing batch add/remove interface to CMempoolTxDB. * Change CMempoolTxDB API in preparation for coalescing removes in the background thread. * Remove an on-disk transaction from the txdb during validation if it\'s cached in the non-final mempool. * Avoid updating the mempool index multiple times during transaction insertion. * Cross-check mempool.dat and the mempool txdb at startup. * Bug fix: lock the mempool index in CTxMemPool::SaveTxsToDisk. * Update mempool.dat to retain references to the mempool txdb during Dump/LoadMempool. * Manage UUID cross-reference keys in the mempool transaction database. * Use transaction wrappers in TxMempoolInfo to propagate the transaction in-memory knowledge. * Remove depth-and-score sorting. The mempool\'s QueryHashes() and InfoAll() methods now return their results in the insertion order. * Update CTxInputData to remember if a transaction was stored in the mempool transaction database, so that this info can be propagated during restore from mempool.dat. * Add accounting for the number of stored transactions in the mempool txdb. * Fixed a bunch of compiler warnings, most of them about unnecessary object copying or unused variables. * Add a method to the mempool to check the integrity transaction database against mempool contents. * Retreive all transaction IDs from the mempool transaction database. Moves the SaltedTxidHasher to primitives/transaction.h. * Use a background thread to write transactions to and delete them from the mempool transaction database. * Mempool entries should only read from the transaction database, the mempool is responsible for writing and deleting. * Drop the last of // MARK comments from legacy block assembler removal * Clear the mempool transaction database when the mempool is cleared, except during reorg. Added low-level mempoolTxDB unit tests. * During reorg we may be adding entries to the mempool index that already have their transactions stored on disk, so make sure they\'re in the right mempool database and that we don\'t clear said database. * Remove transactions from disk (in batches) when they\'re removed from the mempool. * Make the mempooltxdb_tests check preconditions correctly, and don\'t use the global mempool. * Set the reference to the mempool transaction database when a new mempool entry is inserted into the mempool, not when the entry is created. * Remove unused parts of the mampool/database API, fixing an inifnite loop along the way. * SelectCoins adjusted to the new limits * Limiting cpfp chain tx count * Reimplementing limiting ancestor count * increase timeouts in bsv-block-stalling-test.py * Fixed after review: - GetMostWorthles -> GetMostWorthless - brace initialization - typo fixes * fixed waiting for the end of block chain activity * increased RPC timeout in abc-p2p-compactblock.py * add nullptr check in GetDiskBlockStreamReader * Use groups and take most worthless tx when calculating rolling fee * Add functional test of fee-based priority for eviction from the mempool. * Try to fix the Windows build, it seems that namespace lookup in MSVC is lacking a few edge cases. * Enable the mempool_limit.py functional test. * Fix CTxMemPool::Size(), it should count the secondary mempool, too. * Enable mempool trim-to-size and its unit test. * Add a unit test to verify correct stats in the secondary mempool withcomplex chains. * Add default initialization for the secondary mempool statistics. * Make TrimToSize use the eviction tracker (but still keep it disabled). * Integrate the eviction candidate tracker with the mempool, but don\'t actually instantiate it yet. * Make the MempoolSizeLimitTest compile with the current code base, but leave it disabled. * Merge common code in salted hashers and make them copyable. * Journal using groups * bsv-journal-mempool-reorg-ordering test should wait for all submitted transactions. * Use endomorphism optimization by default * Fix error and help message for recvinvqueuefactor * Remove mempool_packages.py test * Update failing test list * CTxMemPool::RebuildJournal reimplementation * Prioritize transaction can create/disband groups * Refactoring CTxMemPool::RemoveForBlock * CTxMemPool::RemoveStaged accepting CJournalChangeSet instead of CJournalChangeSetPtr * CTxMemPool::removeStagedNL now can disband groups * Refactor CTxMemPool::removeStagedNL as there is no need to update descedants * Updating affected transaction after tx insertion * Adding a single group Also fixing some tests which are affected by the primary/secondary mempool size. * Fix functional test: name for zmq notification removedfrommempool is now discardedfrommempool * Change CTxMemPool::setEntries form std::set to std::unordered_set * Minimal block fee-rate set in the mempool * CheckBlock now checks primary mempool size * Fix check journal content * Fix lifetime bug for mempool_tests lambdas * Fix lifetime bug for CConnman::getTxnValidator() * Fix auto reference lifetime bug * Use a pointer to the transaction wrapepr in mapNextTx, its lifetime is the same as the map\'s. * Fix functional test: generate takes to long for hard to validate block * Refactor: rename ip_address to address in zmq tests since it not only contains ip * Add method for distinct zmq ports when running zmq tests in parallel * Remove redundant using declarations. * Copy keys and values into the mempool\'s mapNextTx, because the assumption that all transactions are in memory is no longer valid. * Remove the unused AncestorCounts and AncestorCountsPtr types. * Use public accessors of CTxMemPoolEntry where available. * Add unit tests for secondary mempool stats. * Track secondary mempool statistics when adding entries to the mempool. * Make CPFP group info private to the mempool entry and use the available public accessor methods in the implementation and unit tests. * Add atomic initialization and disk usage of the mempool database. * Added saving txs to disk in LimitMempoolSize * add maxmempoolsizedisk param * Added unit and timing tests, added disk usage checking * Added saveTxsToDisk() * Added moveTxToDisk() * Unused PBV mNodeAsyncTaskLimit variable deletion * Reset release notes after 1.0.6 release. * Added txDB to CTransactionRefWrapper * Added db to txmempool * Added CTransactionWrapper class * Added disk usage param * Added mempooltxdb class * Replace pattern entry->GetTx().GetId() with entry->GetTxId() and remove CTxMemPoolEntry::GetTx() which will become unsafe when mempool transactions may move to disk. * Stopping updating ancestors in the mempool * Removed sigop count statistics from the mempool * removed \"...WithAncestors()\" functions from the mempool entry * Removing ancestor count and sig-op count from journal * Added sequenced index to the CTxMemPool::mapTx to preserve insertion order * CTxMemPool::LimitDisconnectPoolSize call mempool methods directly * CTxMemPool::removeUncheckedNL call mempool methods directly * CTxMemPool::AddUncheckedNL call mempool methods directly * Convert LoadMempool and DumpMempool to public CTxMempool methods * Text only move of LoadMempool and DumpMempool to txmempool.cpp,h * AddUncheckedNL: Move adding to changeset to the end. * Removing the need for smart pointers * Adding mempool\'s eviction candidates tracker. Class that tracks which transactions can be evicted (transactions that does not have children), and chooses most worthless among them. * removed default parameter from virtual function * Basic structures for CPFP groups * Update comments after changing the mempool API. * Make some more of the mempool API private. What remains now is used by CCoinView, and fixed elsewhere. * Make CTxMemPool::RemoveRecursive private * Convert more unit tests to use CTxMempoolTestAccess * Access data for DumpMempool() through a public API and make the details private. * Fix: revert of commit that added GetCoin to AddCoin to improve performance * Using DB::Get() to read records from LevelDB database This means that contents of the whole record will be read into memory even if it is not needed, but performance is better than using LevelDB iterators (at least for small records). * Make some more parts of the mempool private except for tests. * Replace all explicit uses of the mempool\'s const_iterator with txiter, since these are now equivalent. * Remove CTxMemPool::vTxHashes and all external references to the mempool iterator. * Introduce CTxMemPoolTestAccess * Remove the \"coin age priority\" index that was only used by the legacy block assembler. * Remove ancestor and descendant stats from verbose mempool RPC responses and functional tests. * Reduce the CTxMemPool public API by moving ancestor/descendant calculation to private methods. * Fix functional test merkle_proof.py generate takes to long for 1500 blocks * Revert \"Remove SaltedOutpointHasher\" * Modify rpc sendrawtransactions to use json streaming response. * Deletion of bitcoin-seeder * CTxMemPool::updateAncestorsOfNL no loger uses the set of ancestors parameter. * Remove CTxMemPool::CalculateDescendants. Make the ...NL (unlocked) variant private to the mempool. * Unit test for group aware journalling block assembler * Add whole group of transactions or none. * Add group info to CJournal * Create JournalingBlockAssembler::BlockAssemblyState for easy rollback * Move manipulation of mRecentlyUpdated out of the addTransaction * Move BlockAssembler::mBlockFees to JournallingBlockAssembler * Fix JouirnalingBlockAssembler.addTransaction API to be able to add groups. * preventing copy to loop item in range for * Add an extra config params to the bsv-ptv-rpc-sendrawtransactions.py FT. * Rename AncestorDescendantCounts to AncestorCounts everywhere. * Remove cumulative descendant state from the mempool entry. * Remove descendant score index from the mempool. * Annotated and skipped functional tests that are going to start failing due to intermediate code changes. * Removed ancestor-score index from the mempool transaction multi-index. * add rpc function to return block and transaction policy settings * Parametrized bsv-protoconf.py to test maxprotocolrecvpayloadlength parameter * removed CMessageHeader::IsValidWithoutConfig, since it was used only in unit tests * Add configurable factor parameter for number of recieved full size inventory messages to be queued at once * Max protocol payload length as command line parameter (-maxprotocolrecvpayloadlength) instead of a constant * Remove legacy block assembler * Stop including legacy.h in header files. * Move nLastBlockTx and nLastBlockSize from legacy to the only user in rpc/mining.cpp * Move IncrementExtraNonce from legacy to rpc/mining.cpp+h * Move MAX_COINBASE_SCRIPTSIG_SIZE to consensus.h. Fixup CheckCoinbase. * Move UpdateTime from legacy to mining/assembler.cpp+h * Markup legacy.cpp dependencies * Markup legacy.h dependencies * Allow use of deprecated -minconsolidationinputmaturity * Consolidation tx cleanup. - Rename option -minconsolidationinputmaturity to -minconfconsolidationinput - Fix IsConsolidationTxn() so callers pass in chainTip height as intended. * replace modulus with FastMod * Disable currently broken test bsv-cpfp.py * HACK: Temporarily add --journal-broken-tests to filter out failing tests * Revert \"checkTxnExists function added to journal\" * Revert \"Journal change set provides information about added and removed transactions\" * Revert \"Minimal block-fee and CPFP in the JBA\" * Revert \"Optimization for removing for block and adding for reorg\" * Revert \"Help message for the \"-blockprioritypercentage\".\" * Improve interface to GetNextMessage() * do not load whole block into memory where not needed * DisconnectTip: Encapsulate last user of DisconnectedBlockTransactions innards * Clean up implementation of the new methods, trim down the public API. * Make RemoveFrom- and AddToMempoolForReorg mempool methods. * Cleanup RemoveFromMempoolForReorg. vHashUpdate is always empty, remove dead code. * Cleanup RemoveFromMempoolForReorg. vTxInputData is always empty, remove dead code. * Inline constant fAddToMempool and remove obviously dead code. * Move fAddToMempool to the only usage site * Duplicate UpdateMempoolForReorg into AddTo and RemoveFrom variants * Add block statistics tracking to JournalingBlockAssembler * remove \'bitcoin\' prefix from thread names * Refactor miner_tests/CreateNewBlock_validity * Add option --lists-tests to check if test filtering works as expected * Add missing dependency for check-functional * cmake: fix check-functional for XCode * Fix gcc check for atomic library, test on 128bit object * Updated to address review comments: - Class CTxMemPool::Slice renamed to CTxMemPool::Snapshot. - Simplfied snapshot validity check, removed fake container and iterator. - As a consequence, simplified container abstraction (size(), empty(), begin(), end()). - Removed use of SaltedTxidHasher. - No functional changes. * Remove dependency on mempool internals from some RPC functions. * Added unit tests for mempool snapshots. * Added methods for creating read-only snapshots of the mempool. * Make CTxMemPool::CalculateDescendants a \'const\' method as it always should have been. * Use DepthAndScoreComparator in CTxMemPool::CompareDepthAndScoreNL to avoid code duplication. * Clean up implementation of CheckTopoSort * copied implementation from utilstrencondins over to uint256 * removed nonsensical casts that make SonarCube believe it\'s slicing * txn_clone functional test should wait for transaction propagator before re-connect * Rename functional test and drop unnecessary bitcound parameters * Remove std::stable_sort from JournalChangeSet. * Add CheckMempoolNL variant * Fix InvalidateBlock by correctly keeping the journal and mempool in sync during a reorg. * bsv-ptv-p2p: drop racy assert * Update P2P to use the -p2phandshaketimeout config value. * Add a new -p2phandshaketimeout config parameter. * Fix bsv-genesis-journal-reorg functional test Test failed on a busy environment if generate RPC could not create all blocks. * Set default block size parameters in DeserializeAndCheckBlockTest * Fix for maxConsolidationInputScriptSize parameter to use GetArgAsBytes() function in init.cpp * Locking of CS_main changed for getblock/getblockheader/getblockbyheight RPC methods to prevent holding the lock during JSON serialization Added a new exception that REST and RPC methods can catch and return their onw error messages * CORE-968: Find correct OpenSSL for functional tests on macOS. * initialized member in wallet * Fix p2p-fullblocktest.py by adding additional timeout paramter to sync_blocks() * Use testConfig reference to change config settings when it is needed. * Use a local CTxMemPool object instead of the global instance. * replaced boost/bind.hpp with boost/bind/bind.hpp * fixes for the windows build * removed unused function declaration AddressCurrenltyConnected * fixed an include file issue * removed further unused functions * removed unused class * unused includes * rpcfix: return call ids as integer * Mine previous transactions before testing new batch * Add HEADERS & GETHEADERS to high priority stream for BlockPriority policy. * Annotate GETDATA requests for blocks with the appropriate payload type. * Use payload type to help prioritise block messages. * Add payload type metadata to CSerializedNetMsg * Replacing pcoinsTip raw with unique ptr * AddCoin should load coin if it is not in cache * Remove SaltedOutpointHasher * Fix: rest_getutxos did not correctly implement db only querying * Move of CCoinsProvider virtual functions that are only needed for DB to where they belong * Removal of HaveCoin in favour of always fetching the basic coin info. * Coins provider/cache/span conversion from pointer to reference * Limited loading of coins data through db cursor depending on the needs. * Documentation only: Description of coin provider/view/span classes * added method to only load s script if it is not larger than specified * avoid creating unneeded copies of data when reading from leveldb and allow custom stream class for unserialization * Introduction of maxcoinsprovidercachesize command line parameter for limiting coins tip database provider cache size. * Unification of Get and Access coin functions * Split between Coin, CoinWithScript and UTXOCoin * Internal coin view hierarchy locking and split coin views and spans * Write preferring upgradable shared mutex * Merger of CCoinsViewDB and CoinsDB. * Split of multi and single threaded coins views * Merge CCoinsViewErrorCatcher into CCoinsViewDB * Removal of legacy block undo without height info from UndoCoinSpend * Hold CCoinsViewCache lock only while changing cache * Split CoinsStore from CCoinsViewCache * Make mempool locking part of CCoinsViewMemPool * Some CCoinsViewCache functions were locking unnecessarily * Removal of swapping backends for coin views * Minor coin views API cleanup. * Fix: test token from check queue pool should also contain the input tokens * Fix heap-use-after-free reported by AddressSanitizer. * add functional test for getblocktemplate RPC call * added missing special members * Fix for wait_until : timeout exceeded 20Test sometime failed, because transactions included in inv message weren\'t in the same order in which they were sent to node. * fixed virtual function call in destructor * also deleted move members of ZMQPublisher * Add unpaused flag to peerinfo stats. * Add receive queue size to getpeerinfo RPC stats. * Check for being paused for sending and receiving. * Allow us to query whether an association is paused for receiving. * added special default members for class ZMQPublisher * add autoescaping when rendering HTML in combine_logs.py * added redundant null check for clarity * added python imports for decimal module * make implicit height castings explicit * make unsigned height variables of type int32_t * implement streaming in getblocktemplate RPC call * make int height variables of type int32_t * Remove unused vFetch variable from ProcessInvMessage. * SetAskFor overflow fix. * add pushKNoComma method to avoid using httpWriter when formatting result * use streaming in getrawnonfinalmempool RPC call * functional tests for RPC calls in a batch * use streaming in getmempoolentry RPC call * use streaming in getmempoolancestors and getmempooldescendants RPC calls * use streaming in getrawmempool RPC call and in rest call mempool/contents * Write a comment about pruning.py test failure in debug * Add tests for configuring available stream policies. * Enable querying what stream message categories are carried over. * Split out stream policy common functionality. * Allow the available stream policies to be configurable. * Use an enumeration for stream types in the functional tests. * deleted special member defaults that may not be used in checkqueue * Fixups from code review comments. * Functional test for setting up an association. * Include stream policy names in protoconf message. * Stream policies. * Increase the default maximum send/receive buffer sizes. * Rename BLOCK and TRANSACTION streams. * Allow the caller to specify which stream type to use for a message. * Implement CreateStream message. * Allow streams to move between associations. * Add association ID to the end of version message. * Add -multistreams flag to control whether we support the new functionality. * Make LimitedString usable for limited byte vectors as well. * deleted special members to prevent unintended copies * Fixed build for CJSONWriter usage * Update windows build instructions * Changed default value for -rejectmempoolrequest to true. * Fix option -acceptnonstdconsolidationinput to use bool internally. * Increase sync_blocks() timeout in mining_journal.py * Fix bsv-genesis-spendable-op-return by replacing sleep() with wait_until() * Add hash od tx, block and blockheader logs (mininode) * Add max tx validation duration to TC1/TC3 in bsv-pvq-timeouts * Added a list for tetst that cant run in parallel * Implicit loop conversions fixed * Increase graceful bitcoind shutdown for functional tests * Increase P2P_ACCEPT_TIMEOUT timeout * Fix bsv-genesis-activation-transactions-beforeafter.py by replacing sleep() with wait_until() * fixed possible missing announcement of highest block * Fix: scriptCheckQueuePool shutdown should occur after p2p network shutdown * RPC: sendrawtransaction - add a new functional test case to check block\'s validity. * Fix timout in height_based_test_framework * Fix wait-until() time report on assert * Optimized CJSONWriter with logic when to use commas after elements and reduced the complexity of the implementation * Increase timeouts in bsv-ptv-p2p.py * Get rid of sleep() in bsv-genesis-activation-transactions.py * Fix timeout in sync_transaction() * An additional unit test cases for CTxPrioritizer class. * RPC: sendrawtransaction - fix PrioritiseTransaction/ClearPrioritisation usage to be exception safe. * RPC: sendrawtransaction - check if txn is already known before txn validation starts. * Fix bsv-coinbase_P2SH.py functional test * add explicit castings in prevector.h * Tue Nov 17 2020 marecAATTdetebe.org- Update to version 1.0.6: * Final release notes updates for 1.0.6. * Stop the node if initialization error occurred when specifying invalidtx parameters. * Fixed return value from the \"clearinvalidtransactions\" RPC call * Don\'t read block from disk when using getmerkleproof * Rename notification and topic of -zmqpubremovedfrommempool to -zmqpubdiscardedfrommempool * Update release notes for STN reset. * Add blockheight 4 as checkpoint in STN * Skip publish invalid transaction for transactions with reject code REJECT_ALREADY_KNOWN. * Man page updates for 1.0.6-final * Release notes for 1.0.6-final * Using directive is needed to bring boost bind placeholders in required scope. Header should be used instead of . * added missing include * zmq: push information (blockhash) in which block transaction we collided with arrived * remove logging from consensus * Removal of isCorruptionPossible from invalid txn publisher response * Split between DoS and CorruptionOrDoS validation status * Unconditionally log P2P stall messages if debugp2pthreadstalls is specified * Fix gettxouts parameters CLI type mismatch * Remove assert and fix shutdown after unsuccessful startup * Fix: CheckTxInputExists should not be used * Fix of the data race * Rename CValidationInterface::InvalidTxMessage to InvalidTxMessageZMQ * Added lock when accessing private members of the ThreadSafeQueue in UnitTestAccess * Do not use an iterator after its content was erased. Issue was found by ThreadSanitiser * Release notes for 1.0.6-beta * Manual page updates for 1.0.6 * Bump version to 1.0.6 * Add missing include for Gitian build. * Rebuilding merkle tree index from data files Handles issues with the database * Caching Merkle Trees * RPC command getmerkleproof and verifymerkleproof Can be used to retrieve list of merkle tree nodes to calculate merkle root and verify transaction in a block * Move the RPC code that tries to find and read a block to helper function. This will be used in other RPC calls. * Sync with the DB * Store Merkle Trees to data files on disk. Data files are pruned when they reach configured disk size limit. * Fast calculation of Merkle Tree Transactions can be added incrementally without the need to wait for all transactions. Smaller trees are calculated in parallel and merged together to create the Merkle Tree. * Make method for zmq test requirements * Add zmq notifications when transaction get removed from mempool. * Modify sendrawtransactions to return transactions we collided with. * Invalid transaction publisher unit test * Move invalid txn sinks to separate files * Make mInvalidTxnPublisher in CConnman a value variable * IvalidTxnPublisher now also contains collision transaction info * IvalidTxnPublisher changed from singleton to the member of the CConnman class * Functional tests for publishing invalid transactions to the ZMQ * Implementation of the ZMQ sink for publishing invalid transactions * Functional tests for publishing invalid transactions to the files * Rpc command for deleting stored invalid transactions * Publishing invalid transactions from p2p rpc etc... * Registering block origin used when publishing invalid transactions * Publishing invalid transactions originated from block. * Getting failed checks * Publishing invalid transactions * Command line switches for configuring publishing invalid transactions * Adding CFileTextWriter, text writer which writes to file * Publishing ZMQ messages in thread safe way with queue and dedicated thread * Thread safe queue * collidedWith transaction for transactions spent in mempool * Functional tests for gettxouts RPC function * Additional checking if coin is spent * gettxouts RPC function added * Point backup download path to download.bitcoinsv.io * Remove help for config options not functional yet. * Make CallRPC a wrapper to the new HTTP request interface. * Dedicated classes for RPC client communications. * Pull out HTTP status codes into their own header file. * Pull out RPC client configuration into a special class. * Rename rpc/client to client_utils. * fixed use after move * Fix: symbols were clashing with init.cpp even though tests are not using them * Add rpc to list active ZMQ notifications * Post-release release notes updates. * update sprintf to snprintf * Wed Sep 16 2020 marecAATTdetebe.org- Update to version 1.0.5: * Update license for Bitcoin SV Node Implementation. * Add release notes entry for txn relaying optimisation. * Detect and process resubmitted txn during synchronous validation call. * Consolidation man page updates. * reduced complexity when removing transactions from CNode::mInvList and CTxnPropagator::mNewTxns * Fix python package dependency locations. * Misspelt -minconsolidationinputnaturity * Fix help message for the sendrawtransactions rpc interface. * moved consolidatin transaction params after relay param * incorporated review comments * Add rpc sendrawtransactions to the convert table, as it uses a non-string argument. * added consolidation transaction config param \'acceptnonstdconsolidationinput\' and changed default for \'minconsolidationfactor\' * Changed logic for consolidatino tx, now testing against utxo-scriptPubKey and limitix utxo-scriptSig from the top * bypass minimum fee test for consolidation txns * fixed rounding problem arising from using modified fee rate instead of modified fee * miner_tests split to miner_tests_journal and miner_tests_legacy * TestingSetup can accept BlockAssemblerType as param * Destruction of the g_miningFactory moved from BasicTestingSetup to TestingSetup * fee filter against modified fee * Updates to man pages * Version number bump to 1.0.5 * Release notes updates for 1.0.5 * stopping NetworkThread after functional test ends * Increase/fix timeout in comptool sync_blocks and sync_transactions * Tests for CPFP with journalig block assembler * Help message for the \"-blockprioritypercentage\". * Convert CTxMemPool::mJournalBuilder from pointer to variable * prioritise transaction tests for JBA * Setting Journaling Block Assembler as default block assembler * Optimization for removing for block and adding for reorg * Minimal block-fee and CPFP in the JBA * Minimal block fee-rate set in the mempool * Journal change set provides information about added and removed transactions * checkTxnExists function added to journal * Fix race condition in net association unit tests. * fix race in NetworkThread when adding new NodeConn object * Fix for sync_all() \"Mempool sync failed\" assertion error. Test sometimes failed because transaction timelock_tx was received and rejected by node1 before block at height 206 was validated. Consequently contents of mempool was not the same as on node0 and self.sync_all() failed in mempool_reorg.py * Fix bsv-genesis-activation-gracefull-period.py by using wait_until() instead of sleep() * Add missing include guard. * Test insufficient fee and mempool eviction using new sendrawtransactions rpc interface. * Add a new rpc interface, sendrawtransactions, which allows a bulk submit of transactions. * Introduce CTxPrioritizer class to support txn prioritisation and cleanup for an input set of transactions: - clearPrioritisation -> ClearPrioritisation - extend PrioritiseTransaction/ClearPrioritisation interface, to accept a vector of TxIds - add a new test case * Update unit tests to check a result returned by processValidation. * Return a set of rejected txns that couldn\'t be processed by processValidation synch batch call (known, invalid and removed transactions). * Extend an interface to get TxIds of known orphans. * Track TxId and validation state of invalid txns processed through validation thread pool. * Fix disconnect_ban.py by increasing ban time * Fix bsv-genesis-general.py by increasing timeout for tx to be rejected in height_based_framework * Add benchmark stage to CI * rename IsPayToScriptHash to IsP2SH * IsPayToScriptHash decoupled from CScript * Exclude some tests from Jenkins CI * fixed inconsistency in example_test.py * Replace sleep() with wait_until() in bsv-genesis-mempool-scriptcache.py * CTxInputData: Extend encapsulation by adding getters, setter and data hiding. * Optimise txn lookup. * Add pTxIdTracker pointer to the CTxInputData class (and order members by decreasing alignment). * Resubmit an existing txn for revalidation by changing it\'s validation state flag. * Introduce CIntermediateResult struct. * Introduce CTxIdTracker class. * disconnect_close() should close only the outbound socket Fixing the issue where splitting the network failed because inbound socket was just closed before disconnectnode RPC. * must wait for all inv messages to arrive * Remove redundant code CScript::IsWitnessProgram * Fixed values for extra_args: -maxmempool and -maxtxsizepolicy * Introduced new unit constants and config helper function to simplify the code. SECONDS_PER_HOUR to replace (60 *60) and constants for KiB, MiB and GiB units. * Moved performance critical parameters to Config. * Allow size values in bitcoin.conf to be set with human readable values * Prevent blocking the socket handling thread. * Fix potential deadlock in rpc walletpassphrase * Fix executable status on tests. * Fix incorrect and misleading comment. * optimize op[lr]shift for shift larger than value array * After sending invalid block we should wait for it to be processed by waiting for reject. * Allow overriding of node shutdown timeout. * Collect spendable outputs from block 0 in the right place. * Fix refactoring error in dbcrash.py * Fix whitespace in RPC help output * Fix spelling in error message * Add new jenkinsfile for Jenkinc CI * Fix functional test net.py fail on ping pong * move opcodes to separate files * corrections to notes * add some short notes on the docker image * Post Swift release documentation updates. * Shutdown JBA while unit test deletes blocks. * Reduce number of lines logger stores in memory * Update dust threshold amount in test * Add ECC_IsStarted method * fix casting alignments in bitcoin-miner.cpp * Added P2P_ACCEPT_TIMEOUT constant default to 10 and added timeout parameter to p2p_accept methods to allow the test scripts to define the timeout parameter * CORE-809 - Increase timeout for p2p accept to 60 * Replace chain tip copy in journaling_block_assembler.cpp with atomic chainActive.Tip() * Replace chain tip copy in CTimeLockedMempool::periodicChecks() with atomic chainActive * Replace chain tip copy CChainActiveSharedData with atomic chainActive.Tip() * Replace chain tip copy in blockchain.cpp with global chainActive.Tip() * Remove RPC waitforblock. It is not needed and not thread safe * Added timeout param to check_mempool * Moved wait_for_log() to util and renamed to check_for_log_msg * Moved check rejected to util -- used in some more tests that are not merged yet * Remove unnecessary setup_network overloads * Removed overrides setup_nodes() that didnt change base function * Extended prepare_init_chain function to cover more cases * Moved spendable output collecting into a function * Moved check_mempool to check_mempool_equals in utils * Removed redundant PreviousSpendableOutput from tests * Modified tests to reuse next_block and update_block from ChainManager() * Moved create_and_sign_transaction to blocktools * Moved create_tx to blocktools * Removed unused get_unspent() * Moved sign_tx to blocktools * Moved hashToHex function to util * Add missing #include * Fix inconsistency between policy and consensus limits for maxopsperscript * Build fix: add missing include for optional * Remove use of deprecated zeromq api functions * Functional test conversions. * GetSigOpCount implemented in terms of instruction_iterator * Revert \"CORE-795 GetSigOpCount implemented in terms of instruction_iterator (pull request #607)\" * Update docs for verifybinaries * Fix verifybinaries script to work with bitcoinsv.io * Update verifybinaries folder to latest version from BTC * Fix potential race condition in bsv-mempool-ancestorsizelimit.py * Unit test. * Allow CreateNewBlock to fetch more txns from the journal. * Increase rate at which JBA processes new transactions. * Add patches to zmq depends package for building v4.3.2 on Windows * Update zmq depends package to 4.3.2 and remove obsolete patch. * Update configure script to require version of libzmq >= 4.3.2 * GetSigOpCount implemented in terms of instruction_iterator * Revert \"CORE-795 GetSigOpCount implemented in terms of instruction_iterator (pull request #607)\" * GetSigOpCount implemented in terms of instruction_iterator * More detailed timing of invalid signature caching * Fix: Just test that it runs faster after invalid signature is cached. * Improve test runner to print currently running jobs * Increase size limit for GETBLOCKTXN messages. * instruction_iterator fix for OP_INVALIDOPCODE traversal. * Include some txns from block2 and 3 in block5. * Functional tests for PBV reorg corner cases. * Handle exception in ActivateBestChainStep. * Add CountOp to replace CScript::Find * Enable -maxmempool config setting, to accept zero value. * Fixup code review comments. * Add some basic unit tests. * Move remaining network related files to under the new net directory. * Add new per stream stats to the output of getpeerinfo. * Add CStream class. * Remove unnecessary thread checks before deleting CNode. * Refactor CNode. * wait until processing completes before sending mempool p2p message * added missing blocks synchronization step * CScript::Find removed * Add functional test for dustrelayfee and dust output handling. * syncing defaults for minRelayTxFee and dustRelayFee * change EvalScripts interface to allow script\'s debugabilities * make SOCKET of type int on UNIX * Clean up comments and dead code from when BIP 125 was removed * include response body in exception when non-JSON response is received * Backport of Core PR#18806 to clarify CVE fix * fixed running two NetworkThread\'s simultaneously * Increase timeout for _new_block_check_accept in height_based_test_framework.py * fixed searching for OpenSLL library in Python 3.8 on Windows * fixed warning in CMake 3.17 when searching for shlwapi library * Remove support for upgrading database from old format. Old format has not been used for more than two years. All users were already required to upgrade to stay up to date with consensus changes. * catch exceptions in NetworkThread * handle errors while reading .cookie file * handle ConnectionAbortedError by retrying * delete .cookie file before starting a node * kill started processes before functional test finishes * Check if can write to log file * Rename logFileOpenFail to logFileOpen * Fix memory buildup if log file doesnt have write permissions * Wed Jul 01 2020 marecAATTdetebe.org- Update to version 1.0.4: * PTV: Change underlying data type in the chain detection algorithm. * Do not relay transactions that were not accepted to the mempool * Fix: pindex became nullptr before dereferenced for logging * Skip checking if block is part of existing safe mode fork for headers. If we received header than we are adding it to block index and it cannot be part of existing fork yet. * Updates to release notes. * CORE-769 - Fixing mining_journal.py functional script * FT updates. * Documentation updates for 1.0.4. * correct location of instruction.h and instruction_iterator.h * code review feedback updates * Detecting a violation of policy limit for max-script-num-length. * Fixes minimum mempool size that should be at least 30% of default mempool size. If the -maxmempool size is less than the minimum, then the node will not start. Update mempool_limit functional test due to new minimal mempool limitation. Removed -maxmempool parameter from bsv-genesis-journal-reorg.py because this parameter has no effect on the test results. It was redundant before (max used mempool capacity is 132144B, which is less than 1MB). In rawtransactions.py bigger transactions are now used to fill up memory pool. * update bsv-stay-in-safe-mode.py functional test with various sending orders * Adding a missing lock required by invalidate_chain test case. * Protecting the wallet from a failure during load & verify process. * Removing a check related to pre-upgrade state. * Added functional tests, that checks if node remains in safe mode. * Fix: safeModeForks should be cleared when mapBlockIndex is cleared * Fixed entering/exiting safe mode: - check if block is part of existing safe mode fork. If that is the case then we should check if the whole fork (or any other existing fork) causes safe mode and not only if fork from fork base to the existing block that received data (pindexNew) causes safe mode. In some node used to exit safe mode when it received block data for block from existing safe mide fork. - when one fork is no longer triggering safe mode check other forks * CScript::GetSigOpCount bench test added * Remove unnecessary memory allocation/copying in GetSigOpCount * Unit test for caching invalid signatures * Cache invalid signatures * DEFAULT_MAX_SCRIPT_CACHE_SIZE increased to 64 MB * Fix for thread pool test. * Post Fortis release documentation updates. * Added Python zmq and rpc modules, removed lcov, corrected bitbucket status notify * Update the expiry date on the securityAATTbitcoinsv.io key. * Fix P2P data xfer deadlock. * Fix formatting of float in log messages. * Add test that stale addresses are considered IsTerrible() * Return newer addresses with less failures in response to getaddr msg. * Added --enable-debug option to CMake * Test reducing false-positive orphans during ptv. * Put detected orphan txns directly into the main processing queue. * Reducing false-possitive orphans, during ptv, by detecting a continuous transaction chain. * remove excess physical dependencies on transaction.h * remove excess physical dependencies on script.h * Fix: Race condition in test with a long new chain * Prevent default build from running unit and functional tests automatically * Tue Apr 28 2020 marecAATTdetebe.org- Update to version 1.0.3: * Added cs_main lock to CheckSafeModeParametersForAllForksOnStartup method. * Final release notes for Fortis release. * Update STN genesis activation height to 100 for network reset (April 2020) * Update security key in rdp. * Add CORRECT blockheight 2 as checkpoint in STN * Add blockheight 2 as checkpoint in STN * Update the expiry date on the securityAATTbitcoinsv.io key. * Add build instructions for Ubuntu16.04 and Centos7 * Add reference to BSV Wiki site to documentation. * Fixed hash comparison. Previous implementation did not correctly matched hashesh with leading zeros. * Fixed getblock with verbositiy=3 for blocks without CDiskBlockMetaData * Fix: Race condition in test with a long new chain * Fix: use iterator content before erasing it * Fixed test. Nodes should be in sync. * Fixed method CheckSafeModeParameters to correctly store fork tip that is causing safe mode. * Man page updates * Release notes for 1.0.3 * Update version number to 1.0.3 * Add regression test for submitblock with malformed block data. * Refactor some calls to HasP2SHOutput() into processBlock(). * correct undeterministic test scenario in threadpool_tests.cpp * Fixed walletbackup.py functional test so that it waits for all nodes to be fully synced after restart and before any getbalance rpc calls. * Added missing dontcheckfee parameter in help message. * Added functional test to check that at startup node updates status of fork tips that have status headers-only but contain invalid ancestors and should have status invalid. It uses chain generated with previous version of node that did not always mark this kind of fork tips as invalid. * Added functional tests for entering and exiting safe mode. * Added new methods for checking if we should enter or exit safe mode. Added third safe mode option for header only forks. * Functional test for preventing submitminingsolution, submitblock, generatetoaddress to put P2SH in coinbase transaction * Prevent node from mining a block with coinbase transaction that creates P2SH output(with submitminingsolution, submitblock), but still treat such blocks coming from other peers as valid * Increase size of some variables(int->size_t) in ConnectBlock and VerifyDB * Modify getminingcandidate to return transaction count and block size * Fix duration profiling and add obtain lock time to ConnectBlock() * add unit test for TTOR with more complex cases * Test feature for topological transaction ordering rule violation detection * Unit tests for TTOR functions * Fixed indentation. * Added TTOR validation when we receive block transactions. Validation is performed for blocks that are 100 or more blocks from current tip. * Fix: PTV race condition in bsv-highsigopsdensitymempool functional test * Reduce txn re-request interval. * Increase size of setAskFor. * Perform early clearing out of mapAskFor. * avoid race condition in notifications functional test by writing to different files * fixed build and unit test alertnotify_test on windows * upgrading broken libevent 2.1.7 to libevent 2.1.11-stable * Additional interruption points between validation levels This will increase granularity of interuption during validation/initialization * Ctrl-C should be able to interrupt block validation during initialization faster. Most time is consumed by ApplyBlockUndo * Join shutdown token with ActivateBestChain input token parameter. * Apply CCancellationToken to handle SIGTERM * test_runner.py will not return error if test is skipped * Skip test if ZMQ is not enabled else throw error if zmq package is not installed * Fix CVE-2017-18350. When connecting to a malicious proxy server, Variable char pchRet3[0] could be implicitly cast from signed char to signed int and then unsigned size_t, causing stack to be overwriten * Remove racy block stalling test. * Remove RPC: estimatefee * Updated functional tests and added new comments to explain changes * New implementation of estimatefee and related functions * Removed all usages of CBlock_Policy_Estimator and code related to it * Fixed path to avoid rebuilding sec256k1 * Fix benchmark CheckBlock to validate block containing P2SH outputs. * Add alertnotify test to test_bitcoin. * Remove alertnotify test for unknown version blocks in notifications.py * [test] Add restart_node to BitcoinTestFramework * Backport of PR #10941 https://github.com/bitcoin/bitcoin/pull/10941/commits * Backport of PR #10941 https://github.com/bitcoin/bitcoin/pull/10941/commits * Backport of PR #10941 https://github.com/bitcoin/bitcoin/pull/10941/commits * Fix sendrawtransaction argument dontcheckfee not accepted as bool * Removed -excessutxocharge parameter and related methods * Fix compiler error for MSVC trigger introduced in DetectStaling * Remove usage of optional in places that is not needed. std::optional::empty() does not affect the execution. ProcessMessage only logs a failure when false is returned * Make p2p timeouts configurable. * Updated instructions for Windows build * Added include * Renamed SendMessage function in zmqpublishnotifier due to conflict with a define in some header file on Windows * New version of ZeroMQ added * Fix --large-block-tests operation. * Fixup unit tests that we broke. * Orphan tracking should hold weak_ptr to node. * A functional test for the stalling timeout. * Add a few more details to getpeerinfo. * Take into account peer speed before disconnecting for stalling. * Increase default stalling timeout window. * Add config parameter -blockstallingmindownloadspeed. * Peer average bandwidth measurement. * Add libdl.so back to list of allowed needed libraries. * Backport of Bitcoin PR to fix symbol-check and security-check * Update OpenSSL depends package to 1.1.1d * CTxMemPoolEntry: remove excess padding * Fix Gcc-9 warnings. * Allow configuration of BLOCK_DOWNLOAD_WINDOW. * Allow adding a new node to test. * entrypoint.py edited online with Bitbucket. --enable-zmq, remove -j from build. CORE-543 * pipe-functionaltests.py edited online with Bitbucket. Return literal result from the test runner. CORE-640 * Jenkinsfile edited online with Bitbucket Added daily build with cron. CORE-640 * Run large block tests during CI nightly build. * Test a large reorg across the Genesis boundary. * Utility function to search the logs. * Clear active changeset when we cross the Genesis boundary. * Post release updates to release notes. * Remove obsolete warning message \"Unknown block versions being mined\" * Add log message when an inbound connection is established * Test that \'num_tx\' is not returned by getblockheader RPC for headers-only block * Update functional test for \'num_tx\' field in getblock RPC calls. * Add field \'num_tx\' to RPC getblock, getblockbyheight and getblockheader. * Update test to use getblockstats and getblockstatsbyheight * Refactor RPC getblockstats to add new RPC getblockstatsbyheight * New RPC getblockstats. Partial backport with modifications. * New RPC getblockstats. Partial backport of files with no conflicts. * Man page updates for 1.0.2 * Bump version to 1.0.2 * Remove Makefile rules for p2sh test data * Fix typo in help message * Log RPC call to stop() even when debug=0 * Functional tests for -banclientua switch * Added switch -banclientua to specify which peers based on UA will be banned * preventing assert in the ProcessHeaderMessages * Fixed JSON formatting when another output had same address and same amount as the last output Same fix also applied to inputs and addresses * Use 64 bit fseek ( _fseeki64) on windows with large offsets. * Fixed tests by adding maxtxnvalidatorasynctasksrunduration to node parameters. * Update release notes. * Remove P2SH test data no longer needed for bitcoin-tx * Remove P2SH support from bitcoin-tx * Update help message to clarify units to be passed in. * Add runtime parameter to test nodes. * Add new -blockstallingtimeout arg. * Remove unused variables from test * Release notes * Update system-requirements. * Man pages updates. * Update version numbers * Release notes. * MacOS CMake build fixes and build instructions * MacOS Autotools build fixes * Adding command line switch -invalidateblock * Catch any exception thrown during txn processing. * add static_assert for 64 bits * Also bump to 1.59 in CMakeLists.txt * Don\'t enforce coin view limits during reorg. * Limit accepted transactions input coins view cache size * remove unused variables * Make scripts executable. * Functional test for no space in non-final pool. * Bail out early if we\'ve got no chance of storing non-final txn. * Functional test for large blocks txindex flag enabled * Added support for skipping functional tests that take too much time to execute and should be run manual. Added functional tests for blockfile io operations and reindex. * Support for 64 bit block size value in block and undo files. If 32 bit size value is uint32_t max (0xFFFFFFFF) then 64 bit size value is following. Added check that we don\'t send P2P messages larger than uint32_t max (0xFFFFFFFF) becasue size field in CMessageHeader is 32 bit. * CDiskTxPos support for transactions at positions greater than 4GB in block * Added new 64 bit fields nSize and nUndoSize to CBlockFileInfo and renamed previous 32 bit nSize and nUndoSize to nSizeLegacy and nUndoSizeLegacy. Also updated usages of this two fields to 64 bit. * replace magic number in static_assert * Default value for maxtxnvalidatorasynctasksrunduration changed from 60 to 10 seconds * Fix for JSON formating because of failing functional test * remove #pragma from prevector * 1) Remove beta reference from README.md 2) Add release signing key to keys.md * Update gitian descriptors. * Enqueue orphan txn if it is not already queued. * Change warning message to include beta. * Check if we are on a production tag before enabling message. * Fixed warnings * Added functional test for (get/decode)rawtransaction methods * Modified methods ScriptToAsmStr, EncodeHexTx, HexStr to write data to CTextWriter class * Implemented method TxToJSON that uses new CJSONWriter to push JSON text directly to HTTP stream * Implemented class CJSONWriter to push JSON string directly to HTTP stream * Update build docs for Windows. * Fixed disconnect pool limit overflow by setting it to the maxmempool size, MAX_DISCONNECTED_TX_POOL_SIZE_FACTOR removed * Prevent blocks of height 1 from skipping checkpoints check * Unit test fix. Adding values to max signed int results in negative values. Asigning negative values to unsigned int results in huge numbers. * Definition of MAX_TX_SIZE_CONSENSUS_AFTER_GENESIS Used for manual execution of additional functional tests. Not practical for automatic execution since creating and validating maximum sized transaction can take lot of time. * Do not reject orphan transactions if their size is same as the limit * Handle help and httpRequest in CRPCTable methods RPCcommand is updated to support both help and httpRequest inside rpc methods * Post release update to the release notes. * Manual page updates. * Remove beta from gitian descriptors. * Release notes. * Replace ABC with SV logo in Doxygen. * Update minimum required version of Boost to 1.59.0 * Fix typo: isGenesisEnsbled -> isGenesisEnabled * Add .vscode to .gitignore * Fix typos in RDP * CBaseChainParams * pindex related checks to prevent dereference of a null pointer. * copyright added to limitedstack.cpp * Fix checksum frequency duration check. * Fix block header announcement checks. * Don\'t log networking message unconditionally. * Fix duplicate misbehaving message for sendheaders. * Update manual pages. * Update (C) year and beta version. * Update release notes for v1.0.0.beta3 * Fix for max_script_size_test - remove op codes from unlock scripts * Extend SCRIPT_VERIFY_SIGPUSHONLY mandatory after genesis to transactions * add SCRIPT_VERIFY_SIGPUSHONLY flag as mandatory after genesis * Fix compile error in op tests. * multisig test fix * Revert \"SIGHASH functional test\" * Revert \"Interpreter enabling old sighash algorithm for utxos before genesis\" * Revert \"Limiting transaction size when signing with old sighash algorithm\" * Added maxstackmemoryusagepolicy and maxstackmemoryusageconsensus to getinfo RPC method. * 5. Added getblockbyheight call * Added option to submit raw transaction without checking fee * -maxstackmemoryusageconsensus and -excessiveblocksize are required parameters We don\'t allow node to be started without specifying those parameters * Added new verbosity for getblock to return block header with coinbase * Added block height to getrawtrasaction. * Fundrawtransaction RPC changePosition parameter causing out of bounds access * Add basic --enable-tcmalloc option to the build. * add functional test for high sigops density for policy and consensus * We now construct blocks without counting sigops after Genesis. Removed -maxblocksigopspermbpolicy CLI parameter * MAX_TX_SIGOPS_COUNT consensus limit is no longer enforced after Genesis. MAX_TX_SIGOPS_COUNT policy is still configurable. Max value is UINT32_MAX * Wallet passing information about is genesis enabled * Remove acceptp2sh from config and command line * Rejecting scripts with P2SH output after Genesis * oplshift_far tests added * op_rshift further than int32_max * CORE-473 - more logging * use valtype::size_type instead of size_t, simplify unit tests * Functional tests for non-standard transactions subset on main net support * removed slow sigops test that generated tx with 20M sigops which is not really any edge case or important max tx sigops count boundary * fixed logic in py method next_block() so that it really generates as many sigops as requested in extra_sigops parameter and throws error otherwise * Update to the RDP in preparation for Genesis * Fix: CTxnDoubleSpendDetector deletion past end of vector * Fixed max_script_size_test * fix lshift and rshift on data larger than 2GB * Fix failing functional test. * Accept a subset of non-standard transactions even on main net * int_serialization redundant code removed * fix undefined behaviour in bsv::deserialize * during graceful tx validation period original reject message and ban score is used * Increase default maximum mempool memory usage to 1000 MB * Rename acceptp2sh to acceptp2sh_ashashpuzzle * Fix failing tests as a result of default fee changes. * Change fee constants. * Remove excessive block size from coinbase message * Remove excessive block size from user agent subversion * Update FTs * Introducing pre-defined CTask::Status levels. * Set maximum validation duration for async tasks in a single run * use utxo_after_genesis instead of SCRIPT_UTXO_AFTER_GENESIS flag in EvalScript * use resize instead of reserve in LimitedStack::padRight (time optimization) * remove ndebug from flags for bitcoind release/relwithdebinfo to work * add specification on LimitedStack::erase method * LimitedStack::insert can only work on child limited vector * make LimitedVector constructor private * Create test for limited stack/vector * fix setting maxStackMemoryUsage in config * add unit test for limiting bloom filters * limit matching of data elements to MAX_SCRIPT_ELEMENT_SIZE_BEFORE_GENESIS * add unit test for maxSendQueuesBytes config * if -excessiveblocksize is not set, use default for max sending bytes in all queues * Fix getblock help not showing * Set the executable bit on the missing scripts. * fix building issue when boost is not installed in the standard location * Fix txn processing if we\'re not in Genesis grace period. * fix memory leak of ASN1_INTEGER * Refactor and fix parallel block validation tests * Fix autotools-configure to fail if boost base is not found or right version * Create a reject message (and update the reject cache) only if txn was detected as fully processed. * Fixed and uncommented tests relying on CORE-165 and CORE-278 * Add hexhdr.py to sources so that it is dist\'d. * Change logging message to be more precise. * more unit tests added for op_checksig and op_checkmultisig * add note about beta release to README * Optimization. Adding progressively larger chunks to vector when deserializing to prevent large number of reallocations. * Genesis release notes. * Update manual pages for Genesis release. * Juggled one log line * Added couple more logging for rejected tx and txs in mempool * Expanding Test framework with possibility to modify created tx * addressing comments from PR#266 * Tue Feb 18 2020 marecAATTdetebe.org- Update to version 1.0.2: * Add log message when an inbound connection is established * Man page updates for 1.0.2 * Bump version to 1.0.2 * Functional tests for -banclientua switch * Added switch -banclientua to specify which peers based on UA will be banned * preventing assert in the ProcessHeaderMessages * Fixed JSON formatting when another output had same address and same amount as the last output Same fix also applied to inputs and addresses * Use 64 bit fseek ( _fseeki64) on windows with large offsets. * Fixed tests by adding maxtxnvalidatorasynctasksrunduration to node parameters. * Update release notes. * Add new -blockstallingtimeout arg. * Catch any exception thrown during txn processing. * remove #pragma from prevector * Enqueue orphan txn if it is not already queued. * Wed Jan 29 2020 marecAATTdetebe.org- Update to version 1.0.1: * Add runtime parameter to test nodes. * Remove unused variables from test * Add missing import to test * Release notes. * Update system-requirements. * Man pages updates. * Update version numbers * Release notes. * MacOS CMake build fixes and build instructions * MacOS Autotools build fixes * Adding command line switch -invalidateblock * Don\'t enforce coin view limits during reorg. * Limit accepted transactions input coins view cache size * Fixed disconnect pool limit overflow by setting it to the maxmempool size, MAX_DISCONNECTED_TX_POOL_SIZE_FACTOR removed * Functional test for large blocks txindex flag enabled * Added support for skipping functional tests that take too much time to execute and should be run manual. Added functional tests for blockfile io operations and reindex. * Support for 64 bit block size value in block and undo files. If 32 bit size value is uint32_t max (0xFFFFFFFF) then 64 bit size value is following. Added check that we don\'t send P2P messages larger than uint32_t max (0xFFFFFFFF) becasue size field in CMessageHeader is 32 bit. * CDiskTxPos support for transactions at positions greater than 4GB in block * Added new 64 bit fields nSize and nUndoSize to CBlockFileInfo and renamed previous 32 bit nSize and nUndoSize to nSizeLegacy and nUndoSizeLegacy. Also updated usages of this two fields to 64 bit. * Make scripts executable. * Functional test for no space in non-final pool. * Bail out early if we\'ve got no chance of storing non-final txn. * Fix for JSON formating because of failing functional test * Default value for maxtxnvalidatorasynctasksrunduration changed from 60 to 10 seconds * 1) Remove beta reference from README.md 2) Add release signing key to keys.md * Fixed warnings * Added functional test for (get/decode)rawtransaction methods * Modified methods ScriptToAsmStr, EncodeHexTx, HexStr to write data to CTextWriter class * Implemented method TxToJSON that uses new CJSONWriter to push JSON text directly to HTTP stream * Implemented class CJSONWriter to push JSON string directly to HTTP stream * Handle help and httpRequest in CRPCTable methods RPCcommand is updated to support both help and httpRequest inside rpc methods * Prevent blocks of height 1 from skipping checkpoints check * Unit test fix. Adding values to max signed int results in negative values. Asigning negative values to unsigned int results in huge numbers. * Do not reject orphan transactions if their size is same as the limit * Post release update to the release notes. * copyright added to limitedstack.cpp * Wed Jan 15 2020 marecAATTdetebe.org- Update to version 1.0.0: * Fix Genesis post-merge compile error. * Manual page updates. * Remove beta from gitian descriptors. * Release notes. * Update manual pages. * Update (C) year and beta version. * Update release notes for v1.0.0.beta3 * Fix for max_script_size_test - remove op codes from unlock scripts * Extend SCRIPT_VERIFY_SIGPUSHONLY mandatory after genesis to transactions * add SCRIPT_VERIFY_SIGPUSHONLY flag as mandatory after genesis * Fix compile error in op tests. * multisig test fix * Revert \"SIGHASH functional test\" * Revert \"Interpreter enabling old sighash algorithm for utxos before genesis\" * Revert \"Limiting transaction size when signing with old sighash algorithm\" * Added maxstackmemoryusagepolicy and maxstackmemoryusageconsensus to getinfo RPC method. * 5. Added getblockbyheight call * Added option to submit raw transaction without checking fee * -maxstackmemoryusageconsensus and -excessiveblocksize are required parameters We don\'t allow node to be started without specifying those parameters * Added new verbosity for getblock to return block header with coinbase * Added block height to getrawtrasaction. * Add basic --enable-tcmalloc option to the build. * Fundrawtransaction RPC changePosition parameter causing out of bounds access * add functional test for high sigops density for policy and consensus * We now construct blocks without counting sigops after Genesis. Removed -maxblocksigopspermbpolicy CLI parameter * MAX_TX_SIGOPS_COUNT consensus limit is no longer enforced after Genesis. MAX_TX_SIGOPS_COUNT policy is still configurable. Max value is UINT32_MAX * Wallet passing information about is genesis enabled * Remove acceptp2sh from config and command line * Rejecting scripts with P2SH output after Genesis * oplshift_far tests added * op_rshift further than int32_max * use valtype::size_type instead of size_t, simplify unit tests * removed slow sigops test that generated tx with 20M sigops which is not really any edge case or important max tx sigops count boundary * fixed logic in py method next_block() so that it really generates as many sigops as requested in extra_sigops parameter and throws error otherwise * Fixed max_script_size_test * Functional tests for non-standard transactions subset on main net support * Fix: CTxnDoubleSpendDetector deletion past end of vector * Update to the RDP in preparation for Genesis * Update to the RDP in preparation for Genesis * fix lshift and rshift on data larger than 2GB * Fix failing functional test. * more unit tests added for op_checksig and op_checkmultisig * Accept a subset of non-standard transactions even on main net * int_serialization redundant code removed * fix undefined behaviour in bsv::deserialize * during graceful tx validation period original reject message and ban score is used * Increase default maximum mempool memory usage to 1000 MB * Rename acceptp2sh to acceptp2sh_ashashpuzzle * Remove excessive block size from coinbase message * Remove excessive block size from user agent subversion * Fix failing tests as a result of default fee changes. * Change fee constants. * Update FTs * Introducing pre-defined CTask::Status levels. * Set maximum validation duration for async tasks in a single run * use utxo_after_genesis instead of SCRIPT_UTXO_AFTER_GENESIS flag in EvalScript * use resize instead of reserve in LimitedStack::padRight (time optimization) * remove ndebug from flags for bitcoind release/relwithdebinfo to work * add specification on LimitedStack::erase method * LimitedStack::insert can only work on child limited vector * make LimitedVector constructor private * Create test for limited stack/vector * fix setting maxStackMemoryUsage in config * add unit test for limiting bloom filters * limit matching of data elements to MAX_SCRIPT_ELEMENT_SIZE_BEFORE_GENESIS * add unit test for maxSendQueuesBytes config * if -excessiveblocksize is not set, use default for max sending bytes in all queues * Fix getblock help not showing * Fix autotools-configure to fail if boost base is not found or right version * Optimization. Adding progressively larger chunks to vector when deserializing to prevent large number of reallocations. * Fixed and uncommented tests relying on CORE-165 and CORE-278 * Refactor and fix parallel block validation tests * Set the executable bit on the missing scripts. * Fix txn processing if we\'re not in Genesis grace period. * fix memory leak of ASN1_INTEGER * block_info tests added for Median Time Past * Create a reject message (and update the reject cache) only if txn was detected as fully processed. * ... * clang-tidy MedianTimePast * clang-tidy fix std::optional * Add hexhdr.py to sources so that it is dist\'d. * Change logging message to be more precise. * add note about beta release to README * Genesis release notes. * Update manual pages for Genesis release. * Fix badly formated printf. * Bump version numbers for Genesis release. * default values for limits set to max int * Removed flags from STANDARD_SCRIPT_VERIFY_FLAGS that are present in MANDATORY_SCRIPT_VERIFY_FLAGS * Functional test added for Genesis graceful activation period * CheckInputs does not add ban score to nodes, for transactions that will become invalid in Genesis and transactions that were invalid before Genesis, during Genesis graceful period * The config params for maximum txn validation duration were renamed. * Put all incoming txns into the \'high\' priority queue. * implement functional test for checking script cache and mempool clearing * Update Genesis block height * remove incorrect move expression * clear script cache when genesis is activated * delete mempool when crossing genesis in ConnectTip * A new FT which tests a behaviour of PVQ (including timeouts configuration). * Rename config params for maximum txn validation duration to start with \'max\' prefix. * remove unused variable * Fixed unit tests to use bsv::bint::deserialize instead of bsv::deserialize * Added functional test for max script number length limits. * Added consensus and policy parameter for maximum length of numbers used in scripts. * Added functional tests * big_int error handling review * Changed MAX_SCRIPT_SIZE limits * bint constructor ranges * Unit test for txn validation queue size limits. * Apply maximum memory usage limit for transaction queues. * Add configuration parameter to control max memory usage in txn queues. * Track memory usage for transactions in the standard & non-standard queues. * bint constructor fix for windows * Work around windows compiler bug. * move constants to consensus.h and policy.h * add functional test for max stack memory usage * add unit tests for stack size * implement and use LimitedStack instead of basic vector for stack in interpreter * introduce DEFAULT_STACK_MEMORY_USAGE and make it configurable * define pre-genesis behaviour with MAX_SCRIPT_ELEMENT_SIZE_BEFORE_GENESIS * use MAX_STACK_ELEMENTS_BEFORE_GENESIS constant instead of 1000 in EvalScript * Added some extra logging to the new height based test framework * op_rshift big ints * op_lshift big int support * fix bug when fetching source in gitian build * Set timeout value for high (standard) and low (non-standard) priority transactions. If validation timeout is exceeded for the high priority txn then move it to the low priority queue. * Replace TxType by TxValidationPriority which is more accurate due to low & high priority validation tasks. * Add IsStandardTx flag into CValidationState class. * Non-Standard txn validation timeout config. * Standard txn validation timeout config. * Set validation timeout flag in the validation state object. * Add validation timeout flag into CValidationState class. * CORE-321 unsigned cast fix on linux and mac * op_split big number support * Added functional tests for block sigops limit * Increased maximum number of block sigops for Genesis * Rename mIdleQueuesLock to mQueuesLock * Limit parallel sibling validation to block validator queue count * Fix: use best chain tip candidate work for determining premature block check termination * Scheduler should wait for worker threads to exit before exiting * Correctly handle shutdown of queue checkers and their worker threads * Removed unused threadGroup variable in checkqueue unit tests * Wrap all threadGroup threads with TraceThread for more logging * Fix for missing end() iterator check * Check whether pointer to error string is provided before assigning to it * Prevent rejected block race condition on slow computers * cs_main lock was not guarding AreOlderOrEqualUnvalidatedBlockIndexCandidates * Fix: checkqueue test was accessing non thread safe instance on two branches * Functional tests * Reject transactions with P2SH outputs after Genesis is activated. Updated CORE-206 and reverted CORE-236. Transactions with P2SH outputs are no longer treated as non-standard and are rejected by default after Genesis. * op_pick unit tests corrected * Added flag -maxorphantxsize, new function to check size instead of number of orphan transactions, updated unit tests * no failures * op_roll test * test added * Fix null iterator dereference in FindPreviousBlockIndex. * op_size big number support * OP_DEPTH big number support. * Late configuration for time locked mempool. * Add a bloom filter to the time locked mempool. * op_within big number support * Read activeChainHeight from chainActiveSharedData. * Split AlreadyHave into two methods IsTxnKnown and IsBlockKnown. * Remove EXCLUSIVE_LOCKS_REQUIRED from AlreadyHave * Read activeChainHeight and chainActiveTipBlockHash from chainActiveSharedData. * Use chainActiveSharedData. * Introducing CChainActiveSharedData class. * Fixes after integration. * A missing entry during a check for descendant chain size limit. * implemented consensus tx limit for P2P tx messages * fixed CLI validation of negative values for switch -genesisactivationheight * added functional tests for max consensus tx size * added unit tests for configuration of max tx consensus size * unlimited max consensus tx size after Genesis * Use a set data type to control unique elements to be returned by collectDependentTxnsForRetry. * fix finding openssl * fix windres call for openssl 1.1.1 cross compiling with mingw * fix some error in bash variable expansion * Pump openssl version to 1.1.1 for gitian build * big int support for op_1add, op_1sub, op_negate, op_abs, op_not and op_not_equal * added functional tests for max policy transaction size * added unit tests for max policy transaction size * implemented configuration of max policy tx size after Genesis * Changed CScript::GetSigOpCount to correctly count sigops in OP_CHECKMULTISIG after genesis * EvalScript reads MAX_PUBKEYS_PER_MULTISIG from config now * bytespersigop option is no longer used Option was added with commit 45b8e278, updated with ab942c15 and later removed with b569a713 where weight was replaced with sigops count we use today. * Functional tests * Unit tests * Make CBlockIndex a required element in QueuedBlock MarkBlockAsInFlight that fills QueuedBlock list never gets a null pointer. This gives us the opportunity to always set needed block height into FillBlock. * Use proper values for MAX_TX_SIGOPS_COUNT depending on Genesis and policy * Fix compilation failure for MSVC caused by narrowing conversion in initialization * Functional test for maximum operations per script * Increased maximum number of non-push operations per script for Genesis. * Fixed failing functional tests * Fee calculation for utxo * Increased prevector size types to support larger amount of data * Added consensus parameter and Config parameter in EvalScript and dependant methods When height of the script is not available, we use consensus limit (consensus=true) * Added logic to solver to check scripts with pubKeys greater than 1 byte (256) * Make transaction relaying aware of the non-final mempool. * Move TxMempoolInfo out into its own file. * Dump & restore the non-final mempool. * Add flag to ContextualCheckTransaction to indicate if txn is from a block. * Replace STANDARD_LOCKTIME_VERIFY_FLAGS with a function. * Add a new transaction reject code for a mempool being full. * Fix some missing includes. * Add a new finalised source for transaction validation. * Add transaction validator resubmitTransaction method. * Validation for genesis time-locked transactions. * Make the txn double spend detector aware of non-final txns. * Add an extra non-final validation flag. * Extend mempool checking method to check for non-final txns. * Hook time locked mempool into main mempool. * Add check for non-final transactions to AlreadyHave() * Add new getrawnonfinalmempool RPC command. * Config parameters and periodic checks. * A new class to track time-locked transactions. * Functional tests for restoring nLockTime & nSequence. * Key\'s secret byte incorrectly initialized. * Orphan txns scheduled to reprocess: Add txn to the result set if it\'s not there yet. * op_depth big integer support * Fix race in miner_tests at shutdown. * Add threadpriority.cpp to CMakeLists.txt * Max transaction validation duration should be configurable from command line * Cancel transaction validation if it exceeds max duration threshold * Timed cancellation token that cancels task after N time has elapsed * Configuration for standard and non-standard txns in the FT * (re)name interpreter genesis flags * Include prioritised validation queues config params in the test: -numstdtxvalidationthreads, -numnonstdtxvalidationthreads * Due to asynchronous validation some transactions can be processed faster than others. * Get total number of queued transactions by taking mtxs in a read mode. * Include a check for non-standard queue in the UT method waitForEmptyQueue. * Use openssl BN_bn2mpi and BN_mpi2bn to serialize big numbers * Join tokens that are used by functions called from TxnValidation * Check for cancellation request on every step of multisig check * Connection of CScriptCheck cancellation token with ActivateBestChain token * Forward check queue task cancellation token inside EvalScript function * Forward check queue task cancellation token inside checker * Include txn\'s type in the log message. * Enable an independent processing for standard and non-standard transactions. * Support for a default number of Low and High priority threads used by the Validation thread pool. * Basic support for nonstandard transactions. * Change batch transactions into a single transaction per task. * Dual queue support. It allows to handle standard and non-standard tasks at the same time. * SCRIPT_ENABLE_BIG_INTS replaced with SCRIPT_UTXO_AFTER_GENESIS * Python test framework optimization use mutable arrays when sending data. This increases mininode send performance for almost 50% * Increased timeout for bsv-block-size-activation-genesis.py * Temporary workaround because ASN1_INTEGER_get_int64 is not available in older versions of OpenSSL * bn_op_num2bin -ve length unit tests added * Set executable flags on python functional tests * Fixed functional tests * Removed 2GB block size limit for genesis (set to 4GB - 1MB due to P2P protocol limitations). Removed previous block size increment code. * fix leveldb visual studio 17 build * Sunset operators OP_CHECKLOCKTIMEVERIY and OP_CHECKSEQUENCEVERIFY * Process double spend transactions in parallel * OP_BIN2NUM and OP_NUM2BIN big integer support * SIGHASH functional test * Limiting transaction size when signing with old sighash algorithm * Interpreter enabling old sighash algorithm for utxos before genesis * Genesis activation based tests setup * Height based framework * temporary change callback function inside the NodeConnCB using context manager * wait_until configurable error message an check interval * Monitoring file (e.g. bitcoind.log) when the test is running * reverting split SolverNoData & SolverWithData -> Solver * p2sh python tests * Tests sunset P2SH: fix for misc tests * Tests sunset P2SH: validation and cacheing scripts * Tests sunset P2SH: P2SH scripts and transactions * Tests sunset P2SH: multisig tests * Tests sunset P2SH: SigOp Test * Tests sunset P2SH: script tests * Tests sunset P2SH: transaction tests * Minor cleanups when converting tx to json * SigOpCount depending on genesis * ExtractDestinations correctly reacting to p2sh * IsGenesisEnabled on CWalletTx * Solver returning nonstandard if P2SH * Delivering correct flags to VerifyScript * Move node\'s punishment procedure from the Validator into p2p hadling methods. * Remove dependency on cs_main from P2P CNodeState. * Fix: MacOS clang compiler warnings * Bump max scriptcheck threads from 16 to 64 * Fix: g_connman and transaction validation race condition in unit tests * Improve ActivateBestChain state parameter description visibility * test pbv functionality with actual long to validate transaction * Fix: coin selection benchmark configuration was not properly set * Extend parallel block validation test to early announcements testing * Fix: va_list was populated but never cleared * Fix: OptBool use static analizer warning * CNode bloom filter cleanup * Block out of order processing test * add functional tests for parallel block validation * Block validation command line configuration * Check pool instances should be terminatable * Long running task cancellation mechanism * Release cs_main lock during script validation * Synchronize p2p block request without calling ActivateBestChain * add rpc helper methods and class for block validation status * Prefer faster validation to first accepted block * Use CCheckQueuePool instead of a single CCheckQueue global variable * Network requests that validate blocks should run async * Added RPC for tracking of blocks/transactions being processed * Fix: CChain was at places being accessed without cs_main lock * VerifyScript using UTXO_AFTER_GENESIS * Adding flag SCRIPT_FLAG_LAST * A simple locked reference helper. * New txn handler: config improvement. * Enable parallel txn validation when reorg occurs. * Adding synchronous batch validation support. * treat disabled opcodes the same as invalid opcodes * disallow multiple ELSEs in one IF statement * Preventing that IsGenesisEnabled is with MEMPOOL_HEIGHT * non top level OP_RETURN should be followed by valid script * clear mempool when disconnecting last pre-genesis block to remove mempool transactions that were validated according to Genesis rules * delete IsGenesisEnabled overload and correct functional tests accordingly * update bsv-datacarriersize.py to always run before genesis * Update genesis test with reorg scenario. * When checking if genesis enabled, discard coins with MEMPOOL_HEIGHT. * add functional test for genesis activation * Implementation for UTXO based activation of OP_RETURN. ExecutionSctiptCache related issues will be solved as part of CORE-204. * ExtractDestinations() now correctly identifies data by calling SolverWithData. This is used when converting a transaction to JSON format. * Solver now correctly handles pre and post genesis OP_RETURN. It has been broken into two separate functions: - one that requires UTXO height and is able to identify TX_NULL_DATA (used by IsStandard) - the other that does not care about TX_NULL_DATA and is used in almost all other places. * CScript::IsUnspendable now handles genesis interpretation OP_RETURN. * Add support for original OP_RETURN functionality when SCRIPT_UTXO_AFTER_GENESIS is present * Updates to big number doc * big_numbers.rst -> big_numbers.md * Invalid transactions handling tests * Core-316 big_num undefined behaviour fixed * Update release notes post Ark release. * Make the notes regarding beta release clearer. * Version is 0.2.2 (beta) * Adding possibility to specify ip adress in the run_node_with_connections * Yielding tuple of connections instead if the list to be able to use tuple expansion * libssl-dev added to all gitian descriptors * Make the notes regarding beta release clearer. * Update the release notes to indicate beta status * Version is 0.2.2 (beta) * Update the release notes to indicate beta status * Updated Release notes for STN * Updated Release notes for STN * update release notes to include the STN reset * Set checkpoint in STN for reset of the chain. The chain is large, over 2TB, its time to reset and start from scratch. * Remove capability to produce min difficulty blocks on STN. This capability results in regular block competitions and the requirement for stability of the network given unpredictable hash power is already covered by the new DAA. * libssl-dev added * rename big_int.hpp to big_int.h for consistency and add to makefile.am * mining_score index and associated test removed * Functional tests for inv. msg. time order added * Propagate transactions in order they are validated * Fixed big num test data by adding cast to 64 bit integer before performing +/- operations. * Reset the max orphan lenght to 5000. * Reset the max orphan lenght to 5000. * Do not allow the current outpoint to be used again when collecting dependent txns for retry. * Process all detected double spend txns instead of discarding them. * Do not allow the current outpoint to be used again when collecting dependent txns for retry. * Process all detected double spend txns instead of discarding them. * CORE-277 rename frame to arg * Added command line parameter for genesis activation height * Updated DisconnectTip with Genesis check * Updated CheckInputs methods with Genesis check * Fix possible deadlock identified for Clang thread sanitiser. * Added SCRIPT_GENESIS flag * Adding Genesis Upgrade activation * CORE-306 nMockTime made atomic * allow only whitelisted peers to request current tip even if queue is full * make AddUncheckedNL private * make DynamicMemoryUsageNL a private function - quick win, prevent future use outside of class * update description of CTxMemPool and add warnings about deprecation of functions and typedefs * Fix possible deadlock identified for Clang thread sanitiser. * Check if fee estimation can be applied before txn is being validated. * Fix bug in GetHeightFromCoinbase for very low heights. * Move GetHeightFromCoinbase() into the block.cpp file. * update release notes to include the STN reset * CORE-277 big number attack mitigation * Check if fee estimation can be applied before txn is being validated. * CORE-299 Big Number Documentation added * Fix bug in GetHeightFromCoinbase for very low heights. * Set checkpoint in STN for reset of the chain. The chain is large, over 2TB, its time to reset and start from scratch. * link the systems requirements document into the readme * Lock cs_main before calling StreamBlockFromDisk in unit tests. * add enable-debug flag for configure * Lock cs_main before calling StreamBlockFromDisk in unit tests. * Remove capability to produce min difficulty blocks on STN. This capability results in regular block competitions and the requirement for stability of the network given unpredictable hash power is already covered by the new DAA. * Add missing mutex lock to orphan txn processing. * Fix CConman shutdown error. * Fix CConman shutdown error. * Add missing mutex lock to orphan txn processing. * CScriptNum big ints and little ints never mix * CScriptNum overload commented * Incorrect mempool lock in transaction propagator * Removed explicitly default constructor that is implicitly invalid * fix unused op warning * fix: initialize value or it might have diffrent value then expected * use separate fd set for exception fds * fix intended fall through warning * add script to check seed nodes version * add stn network magic and network ports constant * fix warning comparison between unsigned and signed integer * JournalingBlockAssember unprotected read fix * OP_CODESEPARATOR tests * change command line switches to lower case * Fix bsv-broadcast_delay.py by adding -txnvalidationasynchrunfreq * reorder setting diskBlockMetadata to guarantee backward compatibility * Fix mempool mutex and logging * Add some extra logging for journal error cases. * Pass CJournalChangeSetPtrs as const& * Apply basic journal change sets as early as possible. * Rename getSimple to getTailAppendOnly. * Include state message in the log data message for an invalid orphan txn. * add missing multiple make & job executions in dev jenkinsfile * Update timeouts for test cases. * remove the extended tests, for this set of tests we\'re interested in speed of execution * CScript print functionality * bn_op_tests added * big int activation logic * CScriptNum using std::variant * Bug fix in bint::operator<< * big_int.cpp corrected in Makefile * CScriptNum Arithmetic operators refactored * CScriptNum relational operators refactored * CScriptNum equality operators refactored * CScriptNum::MinimallyEncode moved to int_serialization.h * CScript::IsMinimallyEncoded moved to int_serialization.h * CScriptNum::set_vch replaced with bsv::deserialize * CScriptNum::serialize replaced with bsv::serialize * CScriptNum moved to separate file to reduce physical coupling and compile times * bsv::swap(bint&, bint&) added * bint remove to_[u]int * methods * minor comment correction * big_int new files added to cmakelists.txt * big_int replace magic_numbers in static_asserts * Remove redundant log message for an orphan txn (all valid cases are alredy covered). * Solving issues caused by the order of locks. * lower_bound error fixed for in TxConfirmStats * Correct log message when an orphan txn is detected * Remove inputs from the double spend detector only by transaction that added them. * add random block hash to avoid blockfile_reading_tests failing * remove code coverage from dev tests Jenkins file * copyright notices added to new files * big integers class/functions added * Log info update: Read mempool size & dynamic memory usage when txn is commited (in the same call). * Change default for P2SH acceptance. * Remove references to P2SH changes from documentation for next release. * Fix \"use before set\" error in bsv-protoconf-versions-compatibility.py * Fix ITF in bip68-sequence.py * Use a larger timeout when syncing the largest block. * Allow the accepted() timeout to be overridden. * Update manual pages for ARK release. * Update release number to 0.2.2 * Update release notes for ARK release candidate. * FT update: Wait until txn is accepted by the mempool. * Changing nTransactionsUpdated type to std::atomic_uint * An interface update for PrioritiseTransaction - passing strHash by a const std::string reference * Decoupling NotifyEntryAdded signal from smtx. Code formatting. * Adding a NL suffix for trackPackageRemoved (as it is a non-locking private method). * Replacing CCriticalSection cs (a recursive mutex) by std::shared_mutex smtx (a shared mutex). * Changing nCheckFrequency type from uint32_t to std::atomic_uint32_t * Change return type from bool to void for AddUnchecked & AddUncheckedNL. * Move AddUncheckedNL to the public interface. * Introducing a non-locking version of DynamicMemoryUsage * An interface update: Passing a hash value via const reference. * Putting back an explicit LOCK(mempool.cs) into blockchain.cpp methods. - At the moment I can not optimize it as CTxMemPoolEntry is not a smart ptr inside mapTx member. * Moving implementation from .h into .cpp * Renaming entryToJSON to entryToJSONNL as it requires to take the mempool.cs lock. * Renaming public methods: Start a method name with a upper-case character. * Renaming private methods: Start a method name with a lower-case character. * Renaming private and a non-locking method UpdateForDescendants. * Introducing a non-locking version of exists which takes Outpoint object as an argument. * CCoinsViewMemPool: Replacing get & exists by a non-locking versions. A usage of the view is always protected by an explicit LOCK(mempool.cs) call. * Using CalculateMemPoolAncestorsNL in getmempoolancestors as it is alredy protected by LOCK(mempool.cs) * Introducing a non-locking version of exists & get. * Introducing a non-locking version of isSpent * Replacing a non-locking method CompareDepthAndScoreUnlocked by CompareDepthAndScoreNL * Replacing a non-locking _clear by clearNL * - Replacing ApplyDeltas by a non-locking version in AddToBlock, addPriorityTxs methods where LOCK(mempool.cs) is already used. - Replacing CalculateMemPoolAncestors by a non-locking version in addPackageTxs method where LOCK(mempool.cs) is already used. - Changing removeConflicts by a private non-locking version * Removing an explicit LOCK(mempool.cs) from getmempooldescendants. * Introducing a private and a non-locking version of: - UpdateForRemoveFromMempoolNL - removeUncheckedNL - RemoveStagedNL - addUncheckedNL - removeRecursiveNL - CalculateDescendantsNL * Introducing a private and non-locking version of addUnchecked. * Introducing a private and non-locking version of CalculateMemPoolAncestors * Interface clean up. Use NL suffix to show which methods are non-locking. * Submit a valid txn to the mempool in validation thread - Due to existing limitations (related to cs_main lock) TransactionAddedToMempool & Misbehaving calls need to be moved to the Validator thread * A dedicated FT to verify a processing of P2P txns depending on node\'s configuration. - It allows to configure test cases with and without double spend money issue * Necessary update for functional tests - Due to an aynchronous processing of txn validation * A failure during UTs execution in debug mode. The cs_main lock is held by the Validator during txn validation. Removing AssertLogHeld from: - CheckSequenceLock - GetBlockScriptFlags - IsCurrentForFeeEstimation * Removing an explicit reference to the global object chainActive from ContextualCheckTransactionForCurrentBlock function. * Protect an access to pfilter during node\'s destruction. * Introducing cs_invQueries lock to properly protect askedFor structures in mt env. * Introducing a dedicated lock for scriptExecutionCache - removing dependency between scriptExecutionCache and cs_main lock * Removing ATMP and related functions * Activate synchronous validation for txns loaded from a file (LoadMempool) - Enable it using txn Validator * Activate synchronous validation for reorg txns - Enable it using txn Validator * Activate synchronous validation for wallet txns - Enable it using txn Validator * Activate synchronous validation for rpc txns - Enable it using txn Validator * COrphanTxns: Moving constexpr variables into class. * Removing old implementation for orphan & compact block extra txns. - UT update * P2P: Introducing a new support for orphan & compact block extra txns. - Replacing previous implementation * Replacing ATMP by a new functionality in ProcessTxMessage method * Adding new methods for CConnman to support P2P txns. * This commit introduces initial txn validation support for p2p txns. - Introducing asynchronous interface into CTxnValidator - Extending validation support for p2p txns - Create an instance of the Validator - Shutdown validator before propagator - UT update for asynch interface * Update TrimToSize method to return removed TxIds. * Removing GetSpendHeight call from CTxMemPool::check. * Moving orphan txn\'s config from net_processing.h into orphan_txns.h - #include \"orphan_txns.h\" will be removed when legacy code is removed from the file. * Introducing CTxnValidator class - It supports synchronous validation interface - UTs for the validator * Moving static FlushStateToDisk function to the header file. * HasNoInputsOf was not protected by the mempool lock. * Introducing TxnValidation & ProcessValidatedTxn methods - These two methods will replace ATMP - A basic, initial UT test update to verify correctness - Introducing a new logging tag - P2P txns are not fully supported by this commit * Support a vector of outpoints to uncache. * Mt support for CCoinsViewCache. * CheckSequenceLocks: Take the mempool explicitly by an argument * Moving check for conflicts with in-mempool transactions into CTxMemPool class. * class CValidationState: Adding support for a missing inputs. * Introducing a basic class support for orphan txns - class COrphanTxns * Introducing a basic class support for recently rejected txns - class CTxnRecentRejects. - Currently, it will be used by p2p txns. * Introducing a basic class support for double spend detection used during txn validation. - class CTxnDoubleSpendDetector * Introducing basic classes to support parallel validation - CTxInputData - an instance of this class will be used to represent a txn and all related data required to trigger validation - CTxnValResult - an instance of this class will be used to return a result of validation - A validation config file * Use GetCurrentThreadId on cross-compile build for Windows. * Boost.Test: Solving an ambiguous overload for nullptr. * Missing files in Makefile configuration. * include thread name only at the beginning of new lines * Prevent JournalingBlockAssembler from locking same CJournal twice in a single thread * Fix #define clash with enumeration on some Windows builds. * Remove std move from return statement * Move common test functions outside anonymous namespace * Deletion of dead code * Replacement of braces around scalar initializer * add missing using statement requried by MSVC compiler * fix: Error C2361 initialization of \'journalingAssembler\' is skipped by \'default\' label * Race condition in p2p node messages access * process batch of RPC requests in chunks * remove blockToJSON method * send blocks in chunks for RPC (for all verbosity options) * send blocks in chunks for REST (bin, hex, json) * add functional test for -acceptP2SH * Discourage usage of P2SH in output script. * document P2SH policy change * add support for sending replies in chunks with eventlib * add HAVE_DECL_DAEMON define to cmake build * add bitcoin-miner to CMakeLists * pipe-functionaltests.py edited online with Bitbucket * Update functional test for journaling assembler. * Improve the performance of the journal tester. * Add new RPC commands for controlling the journal. * Reduce ITFs within bsv-factorMaxSendQueuesBytes.py * Tests for the new journaling block assembler. * Ensure the journal is invalidated whenever the tip changes. * Fix up the factory to create the appropriate block assembler type. * The new journaling based block assembler. * Add check for candidate builder being updated. * Update locking for the journal. * Refactor common code. * Add fee and sigops counts to the journal. * Allow iterating over the current journal. * Allow block assembler type to be configured. * Change sets don\'t need to be shared. * Jenkinsfile edited online with Bitbucket. Removed code coverage on request. * vNodes in ParallelForEachNode was no longer thread safe * Additional logging in bsv-factorMaxSendQueuesBytes.py test * Debug log duration for p2p request processing that takes too long * Make SendMessage a CNode function * Wrap net_processing most recent block caching * Async file reader support for streams * Use disk block streaming for gettxoutproof rpc request * Use streams for wallet transaction scanning * Convert merkle block sending to read block data from a stream * Stream CBlockHeaderAndShortTxIDs creation from disk block * Test that calling UnlinkPrunedFiles doesn\'t terminate opened streams * Move disk block file size and hash info into CBlockIndex class * Enable block from disk streaming * Handle block file pruning for files with open file handles * Added CBlock stream deserialization into CBlockHeader and CTransaction entries * Extend network streams support to CSerializedNetMsg * Conversion of message buffer to chunked streaming capable containers * Added System requirements to manual. * removig limit to only one OP_RETURN * Fix failure in mining_journal.py. * add missing using statements required by MSVC compiler * update tests data for transactions with data; use os.path.join for joining paths * Fix the block timestamps so they are not now too far in the future. * Now build only develop branch, not all; added slack notifications. * Fix race in mining_api.py * Add a test method for checking the journal. * Unit tests for journaling. * Move ActivateBestChain call inside InvalidateBlock. * Add hooks for journal change set. * Allow querying of the current journal size over RPC. * Classes to support journal tracking of changes to the mempool. * Add new logging catagory for the journal. * Allow mempool ancestor count details to be shared. * Add enum_cast. * Functional test for transaction ordering and journaling. * slackSend notifications added * Jenkinsfile edited online with Bitbucket * CRYPTO_USE_ASM should be set in CMake list before first use. * Fix for cmake build when wallet is disabled * Fix for autotools build when wallet is disabled * add blockmaxsize switch to avoid all transactions being included in one block * set bitcoind time to be before activation time * rename some error messages * use OP_FALSE OP_RETURN when generating transactions using data through rpc createrawtransaction and bitcoin-tx * Jenkinsfile edited online with Bitbucket * Jenkinsfile edited online with Bitbucket * Jenkinsfile edited online with Bitbucket * Dockerfile edited online with Bitbucket * Dockerfile edited online with Bitbucket * Dockerfile edited online with Bitbucket * Dockerfile edited online with Bitbucket * Dockerfile edited online with Bitbucket * Dockerfile edited online with Bitbucket * Dockerfile edited online with Bitbucket * Jenkinsfile edited online with Bitbucket * cpanm term::readline * relocated * relocated under contrib directorY * devops files relocated here from the top level; appropriate change to Dockerfile * Update release notes after Orinoco release. * add thread name in debug log * Move run_connections to test_framework.py * new files for the CI pipeline and to execute tests when SV is built * Add functional tests for protoconf message * Scale number of transactions we request in single GETDATA message with effective allowed protocol message length of the remote peer (instead of limiting it to 1000) Remove MAX_INV_SZ - it is not used anymore. * Scale number of entries in mapAskFor and setAskFor with maximum protocol message length. * Scale the limits for number of INV elements with protocol message size: * Define protoconf message, send and process protoconf message, define LEGACY_MAX_PROTOCOL_PAYLOAD_LENGTH and MAX_PROTOCOL_SEND_PAYLOAD_LENGTH * Implement CInv::estimateMaxInvElements * Rename MAX_PROTOCOL_MESSAGE_LENGTH to MAX_PROTOCOL_RECV_PAYLOAD_LENGTH, rename nMessageSize to nPayloadLength * Use relative activation time for functional tests. * add test for logging time difference between block timestamp and received headers timestamp * Fixed the comment about picking the more honestly mined block * Merged logging block diff times from Bitcoin ABC * add test for rest/block for bigger blocks * remove redundant static object * Move error messages to Config class to consolidate checks for invalid parameters at once location * Adding rpc and command line setblockmaxsize * Removed check in the GlobalConfig::SetMaxGeneratedBlockSize * add tests for verifyblockcandidate RPC call * verifyblockcandidate implementation * Disable USE_ASM when building with autotools * Documentation for use of sanitizers * Use of LegacyBlockAssembler as BlockAssembler without virtual destructor * Fix for CRollingBloomFilter use race condition * mTemplates variable population should not cause a thread race condition * Let g_connman terminate propperly * Deleted unused variable that was experiencing undefined behaviour * Support for CMake build with clang static analyzer * Added undefined behaviour sanitizer for gcc and clang development support * Division by 0 - detected by undefined behaviour sanitizer * Addition of thread sanitizer * Race condition detected by thread sanitizer * Add support for enabling address sanitizer for gcc and clang builds * sha256_sse4.cpp should not be built if CRYPTO_USE_ASM is disabled * Remove additional versionbits code, add comments * Tests for csv (bip9) modified * Activate CSV using block height (not bip9) * Remove the rest of BIP9 code * API change: Remove BIP9 related result and arguments from getblocktemplate * Remove BIP9 from setting block version * Remove BIP9 csv activation * Remove BIP9 version bits warning * Remove BIP9 associated tests * Cleanup after backport * Remove bip9 from getblockchaininfo RPC call * Remove the -bip9params flag * Fri Jul 12 2019 marecAATTdetebe.org- Update to version 0.2.1: * Update release notes * Update release notes * Update release notes * Update Release information * Fix deep JSON mitigation problem. * Introduce REJECT_TOOBUSY reject code and timeout when node is overloaded * Limit total size of currently sending block messages * add possibility for verbosity argument to be string in getblock call (RAW_BLOCK, DECODE_HEADER, DECODE_TRANSACTIONS) * add enum for argument verbosity of getblock rpc call * Move -preload help message from RPC to general section * add functional tests for getblock * add checks for incorrect verbosity parameter to RPC getblock * add comment on implementation of obtaining verbosity value * refactor TxToJSON() and ScriptPubKeyToJSON() * Use a verbosity instead of two verbose parameters * RPC: Allow multiple names for parameters * Include libgen.h on Linux and Mac. * Remove duplicate calculation of pindexPrev. * Replace MAC_OSX define with __APPLE__ * excluding VMTouch code from build on windows * Rename EncodeLegacyAddr and DecodeLegacyAddr with EncodeBase58Addr and DecodeBase58Addr * Remove the Bitcoin ABC cashaddr address format * Tidy up GetMerkleProofBranches() * Remove unnecessary call to MkBlockTemplateVersionBits(). * Add bounds checking within CBlokc::GetHeightFromCoinbase(). * Update test for -blockvaliditytest flag. * Make block validity checking from new mining candidate optional. * Add config option to control whether we run TestBlockValidity(). * Update the OpenBSV license to specifically license test networks. * array.tostring deprecated alias for tobytes * OP_FALSE,OP_RETURN behaves in the same way as OP_RETURN * Removed CScript::IsCommitment because it is not used * Return info about % of chainstate that is currently loaded in RAM as part of getmmemoryinfo RPC call * Preload UTXO dataset at startup to speed up transaction validation. * Tue May 21 2019 marecAATTdetebe.org- Update to version 0.2.0: * Adding the test for mining from an old block candidate. * Update unit tests for stricter MiningCandidate interface. * Record block header fields with mining candidate. * Update license to belong to Bitcoin Association * Fix crash with new mining API. * Minor documentation updates. Add a document for the public keys. * Optimizing test to reduce memory consumption. * Improve stability of pruning functional test. * Update of the release notes for version 0.2.0 and of the Responsible Disclosure Policy. * Update man pages by contrib/devtools/gen-manpages.sh * Link with bcrypt.lib when building on Windows. This is required by the new mining API (CORE-13/SV-342) which uses boost uuid library * Update the copyright and license information in source files. * Change default for -datacarriersize to 100KB which enables us to relay larger OP_RETURNS by default. * Move dataCarrierSize, limitDescendantSize and limitAncestorSize to config. * Set the timeout for block sync during tests to 300 seconds, to support large blocks. * Performance optimization of mininode serialization code - use \"join\" instead of \".\" when concatenating arrays * Removed remaining usage of global constants DEFAULT_MAX_BLOCK_SIZE and DEFAULT_MAX_GENERATED_BLOCK_SIZE, since defaults are now network specific. Updated HelpMessage() to reflect new meaning of command line switches -blockmaxsize and -excessiveblocksize * Added unit test for block size related configuration * Unit test cleanup * Removed workaround introduced in 81dc539b. It made sense when configuraiton values were stored in global variables. They were later moved to instance variables making this code noop. * Make sure that GlobalConfig::SetDefaultBlockSizeParams has been called before using GLobalConfig * added added -blocksizeactivationtime parameter, added GetMax[Generated]BlockSize overloads that take time and use them when mining/validating blocks * -blockmaxsize moved to config * Added DefaultBlockSizeParam to config, which are initialized per-chain. Activation is not implemented in this commit. * Rename Block::GetHeight() to GetHeightFromCoinbase() to be more specific. * Use static hasher when checking transactions. * Only call CheckBlock() once when validating a new block template. * Add an entry for bitcoin-cli to convert coinbase arg to getminingcandidate. * Fixup example CPU miner to work with the new API. * Remove old mining test. * A test for the new mining API. * Complete submitminingsolution. * Use a UUID for the mining candidate IDs. * Add coinbaseValue to result of getminingcandidate. * Make provision of coinbase txn by getminingcandidate optional. * Allow user to specify timeout for sync_all() * Allow overriding of test binary from within test. * Add for secp256k1 & univalue tests as well * Update release notes for v0.2.0 * Insert original license holder. * Update the license to the Open BSV License. * Add missing \'make\' optimization for code coverage step * Replace more references to BCH with BSV * Fix MSVC build by adding a static cast in ReadVarInt to handle template instantiations that still incorrectly use signed int. * Refactor CMiningCandidate into its own full class with its own manager. * Refactor CBlockTemplate. Convert the block member object to a shared pointer. * Refactor BlockAssembler. Create interface BlockAssembler & rename class to LegacyBlockAssembler. Add CMiningFactory. * re-organise mining into sub-directory * Added initial help text * Incorrect comments syntax. * Fixes as a result of code-review. * Changes in response to pull request review and issue found by Brad with block chain height in coinbase in getminingcandidate * New BU-spec mining interface (getminingcandidate) * First tested version of BU getminingcandidate iterface - includes CPU miner (bitcoin-miner) * Initial implementation of getminingcandidate and submitminingsolution. * Add new checkpoint (block 2951) to STN. * change name of log file to bitcoind.log from debug.log (part #3) - documentation and comments * change name of log file to bitcoind.log from debug.log (part #2) - source code * Fix ITF in example_test.py * Test sending blocks larger than default preferredBlockfileSize * parametrize excessive_block_size in test * make blockFileSize configurable, rename MAX_BLOCKFILE_SIZE global variable - it now reflects preferred file size * Add missing blockfileinfostore.h for gitian build. * Add command line option -rejectmempoolrequest. When present, P2P mempool request form non-whitelisted peers are rejected. Updated p2p-mempool test * Bump ZeroMQ version to 4.3.1 * Set DEFAULT_RUN_FREQUENCY_MILLIS to 250ms. * Set default broadcastDelay to 150ms, add tests for broadcastDelay * remove max ecessive size relation to max block file size * Remove mallopt setting for 32-bit systems. * Remove dead code related to ABC\'s forced obsolescence * Remove tests related to ABC\'s forced obsolescence and regenerate sighash.json * Use C++11 default initialisation for CNode. * Add test for unsolicited addr messages. * Add addresscount field to getnetworkinfo RPC command. * Don\'t relay additional addresses on outbound connections. * Ignore unsolicited ADDR messages. * Removed GetBlockFileInfo from validation.h * Moved CBlokFileStore related functions to a separate file * Fix log spamming issue. * Add comments * Back to parallel jobs in functional tests * Add leveldb tests * Add Jenkinsfile * Moved Open *File family of functions to a helper class. Moved part of reindexing code into ReindexAllBlockFiles (validation.cpp) to make it possible to remove OpenBlockFile from validation.h * Moved global setDirtyFileInfo to CBlockFileInfoStore * Removed globals cs_LastBlockFile, vinfoBlockFile and nLastBlockFile - they have been moved to CBlockFileInfoStore * Added CBlockFileInfoStore::GetLock() and used that instead of single remaining usage of global cs_LastBlockFile. Removing call to LOCK() FlushStateToDisk (line 2418) is not the option despite the fact, that some of the CBlockFileInfoStore functions already acquire the same lock and we could add locking to the remaining few. This would change locking semantic from a lock that is held for whole period from line to 2418 to line 2556 to multiple non-continuous locking of the same lock. In periods when lock is not held, another thread could perform modifications to pBlockFileInfoStore resulting in possible inconsistent state. * Added accessor CBlockFileInfoStore::GetnLastBlockFile() * Added accessor CBlockFileInfoStore::GetBlockFileInfo * Extracted code to CBlockFileInfoStore::Clear * Extracted code into CBlockFileInfoStore::ClearFileInfo() * Extracted method CBlockFileInfoStore::GetAndClearDirtyFileInfo() * Extracted loading code into CBlockFileInfoStore::LoadBlockFileInfo method. * Moved FlushBlockFile into CBlockFileInfoStore * Moved FindUndoPos into CBlockFileInfoStore, changed signature to remove direct usage of global variable fCheckForPruning * Moved FindFilesToPruneManual into CBlockFileInfoStore * Moved FindFilesToPrune into CBlockFileInfoStore * Moved FindBlockPos into CBlockFileInfoStore, added explicit parameter fCheckForPruning * Moved CalculateCurrentUsage int class. This (and further commits) are WIP. The code compiles, but it will not work until refactoring is finished. * Added comment about BLOCKFILE_BLOCK_HEADER_SIZE * Remove special branch for reconfiguring pipeline * Fixed executable flags on python files * Added Windows build documentation * Added `--buildconfig` parameter to specify which Windows build configuration is used by functional test runner. On windows, copy test runner (and associated files) to build directory instead of creating symbolic links to them. CMake\'s make_link does not work on Windows and creating symbolic links through Window\'s mklink required elevated privileges. * Skip symlink wallet test on windows, where users usually do not have SeCreateSymbolicLinkPrivilege activated. * Fix versionbits functional tests. When running \"echo \'x y\' >somefile\", Windows will store single quotes as part of the file content, but Linux will not. To handle both cases, we we now search for substring instead of exact match * use cross platform filenames in wallet_dump functional test * Do not globally import fcntl in functional tests. It is not present (nor used) on Windows * Remove --forced from test_runner.py. Functional tests are not disabled on Windows anymore * qa: Prepare functional tests for Windows * MSVC compatibility: fix signed/unsigned mismatch error * bench: prefer a steady clock if the resolution is no worse * bench: switch to std::chrono for time measurements * Remove countMaskInv caching in bench framework * Removed bitcoin seeder from MSVC build since it is not cross platform * MSVC compatibility: exclude secp2561k benchmarks from MSVC build, since they use non standard time functions * MSVC compatibility: Removed dependency on BDB C API- we still used BDB_CXX. Use versioned library name with MSVC * MSVC compatibility: linq targets that use zmq with ip helper * MSVC compatibility: test_bitcon now uses cross platform way of finding python. Bumped dependency to python3 (that is also used by functional tests) * MSVC compatibility: link with crypt32.lib - required for openssl * MSVC compatibility: fix compiler options - use static runtime (/MT), undefine NDEBUG * MSVC compatibility: fix zmq library name * MSVC compatibility: use supported scalar and field implementations that do not required inline assembly or int128 support (but is probably slower) * Fixed check_builtin_exist() CMAKE macro that failed to correctly set the value of cahced variables resulting in incorrect #defines * Resolved initialization order warning * Generate bench block-header-include in platform independent way * Squelch non-virtual destructor warning * Convert C-style casts to reinterpret_cast * MSVC and clang compatibility: random_shuffle has been removed from C++17. Use std:suffle instead * MSVC compatibility: do not use designated initializers (c++20) * MSVC compatibility: include cstdint, define ssize_t, do not use ssize_t for NUM_OS_RANDOM_BYTES * MSVC compatibility: levedb fixes: - define ssize_t, - undefine snprintf macro - do not include unistd.h * MSVC compatibility: Use .data() instead of std::begin() to get pointer to the first byte of std::array, since std::begin() returns an iterator which is not implicitly convertible to void * * Remove magnetic activation parameters and check function. * Remove check for re-org across the magnetic upgrade point. * Remove the SCRIPT_ENABLE_MAGNETIC_OPCODES flag. * Remove the magnetic upgrade 128MB block size activation. * Remove MAGNETIC_MAX_OPS_PER_SCRIPT. * Increase max no of opcodes per script to 500, independently of the magnetic upgrade. * Remove magnetic activation for the re-enabled opcodes. * By default only build for amd_64. * Code fixes for C++17. * Add windows Gitian build. * Riscv64 fixups. * Replace gitain-build.sh with gitian-build.py * Update symbol-check.py to latest version from ABC. * Wrap log2f and __divmoddi4 functions. * Update gitian-linux.yml. * Fixup ITF in function test. * Fix race in ActivateBestChain() * Return after processing reject. * Tidy up Misbehaving locks. * Refactor P2P message processing. * C++17 for cmake * fix typo - error message after return * bitbucket pipeline for C++17 * add testnet seeder * typo in seed name * Update build docs for newer GCC version. * Update m4 build macros. * Update the build to mandate C++17. * add txn_propagator to CMakeLists.txt * Updates after another review * Updates following code reviews * fixed test for 32-bit build and changed the json parse depth default value to be constexpr rather than a magic number in the constructor * Removed the conditional build of the JSON parse checking * Updates after comments from the previous pull requests * Updates for comments after the previous pull request * Commit updated tests for bloom filters * Fixed rebase/merge conflict * Updated tests for bloom filters, blockcheck sizes and serializing data * Fixed rebase/merge conflict * Updated both BloomFilter constructors to throw an exception in the event of invalid parameters * Fix to mitigate a possible stack exhaustion during JSON deallocation * Fix for undefined behaviour if an incorrect or invalid argument is passed to the Bloom filter constructor * Allow user to set magicByte separating testnet * Tue Feb 12 2019 marecAATTdetebe.org- Update to version 0.1.1: * updated release notes * change name of log file to bitcoind.log from debug.log * Add txn src logging. * Allow logging to multiple catagories. * correct nPruneAfterHeight for STN * update test to be independent of version * Updates for release 0.1.1 * Fixup lcov make target. * use DEFAULT_MAX_BLOCK_SIZE * Bump protocol message length * add unit tests for CMessageHeader from protocol.[h|cpp] * Sat Jan 05 2019 marecAATTdetebe.org- initial package build
|
|
|