I don't entirely understand the calculation for starting_gas_height and finishing_gas_height, what's the point of that above just computing startgas*len(tx_set) for each tx_set in rangeset R?

interesting. Can you give an example of when this would be best taken advantage of?

Wondering why this is at transaction level, not contract level. If a contract could specify what other contracts it can communicate with, then all transactions sent to it would be computed in parallel, and as a bonus the blockchain itself could eventually be.

Given addresses are randomly distributed throughout the address space, I'm not sure how useful a prefix-based scheme is likely to be - how often will contracts you wish to interact with be conveniently clustered under a common prefix?

@MaiaVictor What if the contract can communicate with other contracts specified at runtime?

Once we have #98 clients can also implement optimistic concurrent processing of txs. The assumption here would be, that most txs in a block don't share any state, except for block.coinbase. So we'd need a different mechanism to collect fees for the miner, but otherwise this can be done without any further protocol updates (a hint on parallelizable group of txs in the transaction list, might be helpful though).

@Arachnid From my understanding, you could maybe have 10 contracts for a given Token (e.g. SNT) where each SNT contract only has access to 1/10 of the address space, with the ability to do cross-token-contract swaps as well. If this is possible, then you could make SNT transactions parrelizable to some extent.

That's the best example I could come up with, but I am sure there are more sensible ones.

I would rather that DELEGATECALL (and for the sake of symmetry, CALLCODE) worked out-of-range. Blindly deploying a contract with libraries is unlikely to group them in the same range. Technically, one could brute-force them into doing so by grinding away at addresses and nonces, but this would be incredibly user-unfriendly. It is also backwards-incompatible.

I do realize that a DELEGATECALL could fail if the targeted contract selfdestructed. One possible solution would be that a DELEGATECALL uses the code of a contract at the beginning of a block, and ignore selfdestructs for this one purpose. That adds a wart, but it also improves the life of the average user.

All that said, I agree with @Arachnid. In a way, this penalizes non-address-grinded multi-contract dapps, For that matter, if address grinding ever became a significant factor in dapp design, users will inevitably want to cluster themselves around popular ranges, leading to pseudo-shards. Elegant and emergent in its own bizarre way, but I don't think it would be desirable compared to an actual first-class sharding system.

@heikoheiko: I'm curious why sharing block.coinbase is a problem. Unless that one address is involved with the transaction, there should be no difference whenever it receives rewards.

Once we have #98 clients can also implement optimistic concurrent processing of txs. The assumption here would be, that most txs in a block don't share any state, except for block.coinbase. So we'd need a different mechanism to collect fees for the miner, but otherwise this can be done without any further protocol updates (a hint on parallelizable group of txs in the transaction list, might be helpful though).

Agree. However, there is currently no gas incentive to make nicely parallelizable txs, and furthermore the worst case still has zero parallelization, and client devs would still have to design for the worst case. This EIP increases the gas limit for "normal usage" by up to 8x but does NOT worsen the worst case at all, and it also introduces an up to 8x subsidy for parallelizable txs.

Given addresses are randomly distributed throughout the address space, I'm not sure how useful a prefix-based scheme is likely to be - how often will contracts you wish to interact with be conveniently clustered under a common prefix?

Most transactions only need to interact with one contract; with EIP86 maybe two. To give one trivial example, any transactions that use two distinct ERC20 tokens will usually have independent read/write ranges.

If a contract could specify what other contracts it can communicate with, then all transactions sent to it would be computed in parallel, and as a bonus the blockchain itself could eventually be.

Consider things this way:

1. (a) There exist contracts that only need to interact with a few other contracts. (b) There also exist contracts that could theoretically interact with anything.
2. (a) There exist transactions that only need to interact with a few known contracts. (b) There also exist transactions that could theoretically interact with any address.

Ideally, we want to retain support for 1b and 2b, but recognize and subsidize 1a and 2a. If we recognize 1a in protocol, then we don't recognize all of 2a, because there are cases where a contract could theoretically interact with anything (quick example: decentralized exchange), but the transaction can be more static (one particular order in the DEX with a known counterparty). However, any instance of 1a is also an instance of 2a, as if a given contract C only affects D1, D2 and D3 then any transaction sending to C can only affect C, D1, 2 and D3. Hence, 1a is a subset of 2a, and so 2a is the more general thing to recognize and subsidize in protocol.

interesting. Can you give an example of when this would be best taken advantage of?

Some quick examples:

1. Transactions going to an ICO can happen in parallel to almost everything else, so the ICO would not interfere with normal traffic the same way. In fact, Status-style network congestion would not be possible unless there were 8 ICOs happening at the same time.
2. ERC20 tokens can be parallelized (ie. transactions sending MKR and transactions sending REP can be totally separated)

I would even go so far as to say that nearly all activity on the current ETH chain would be parallelizable.

I would rather that DELEGATECALL (and for the sake of symmetry, CALLCODE) worked out-of-range.

This is what specification v2 is for. Libraries are generally read-only, and v2 allows for intersecting read ranges, so it should cover all of these use cases.

how often will contracts you wish to interact with be conveniently clustered under a common prefix?

I think you might have misunderstood. My scheme does not require you to choose a single prefix. You can set the read or write range for a transaction as the union of multiple prefixes. So address grinding should not be necessary or particularly helpful.

I don't entirely understand the calculation for starting_gas_height and finishing_gas_height, what's the point of that above just computing startgas*len(tx_set) for each tx_set in rangeset R?

Not sure I understand your proposal. The intuition behind my proposal is that the finishing_gas_height of a transaction can be thought of as an upper bound on the computational "depth" of the transcation - that is, the number of clock cycles of time needed to execute that transaction and all of its dependencies that can't be parallelized.

I'll give an example. Suppose you have four transactions, and suppose that the full address range is the letters ABCDEFGHIJ for simplicity. The first tx has 25000 gas and read/write ranges ABCD. The second tx has 50000 gas and read ranges EFGHIJ and write ranges EFG. The third tx has 65000 gas and read ranges EFGHIJ and write ranges HIJ. All three of those txs can be executed in parallel. Now, suppose the fourth tx has 20000 gas and has read/write ranges CDEF. Executing this tx requires having already executed the first and the second, though not the third (as the third does not write anything that the fourth reads or writes); hence its finishing_gas_height is max(25000, 50000) + 20000 = 70000. A computer with 8 threads would be able to finish processing all four transactions in 70000 cycles (assuming 1 cycle = 1 gas); hence these 4 transactions could fit into a block with a 70000 gas limit.

Hope that makes some sense.

I like this proposal a lot. Given that developers usually opt towards "pull" rather than "push" transactions, most transactions should be able to take advantage of this.

If the subsidy is removed, could this be implemented as a soft fork? Client devs may struggle with multi-threaded implementations and I don't see a pressing need for non-miners to implement this.

@vbuterin I have a question on finishing_gas_height which you described as the computational depth of the transaction.

In your example 4 transactions you calculate: finishing_gas_height(T3) = max(25000, 50000) + 20000 = 70000

The number 70000 is the maximum gas needed to execute that transaction and all of its dependencies that can't be parallelized. Correct?

But then you mention that:

these 4 transactions could fit into a block with a 70000 gas limit

Unless I am missing something, the block's max gas limit required shouldn't change due to the way you process the transactions .Shouldn't it be 160000, basically the sum of all transactions of the block?

Some additional comments on the actual spec:

I really like the idea and believe it will really help relieve the network pressure in Status-like ICO situations.

Even though I suppose it would be a bit more work, I think that spec V2 where you specifically specify both read and write ranges of the transaction is far superior since it will allow for more efficient parallelization.

Unless I am missing something, the block's max gas limit required shouldn't change due to the way you process the transactions .Shouldn't it be 160000, basically the sum of all transactions of the block?

This EIP will change the role of the block gas limit. So a 4.7m gas limit would be not just 4.7m gas worth of computation, but rather 4.7m gas worth of time on an 8-core machine. Perhaps renaming it to gas_height_limit would make that clearer?

Even though I suppose it would be a bit more work, I think that spec V2 where you specifically specify both read and write ranges of the transaction is far superior since it will allow for more efficient parallelization.

Agree. I definitely believe v2 is superior.

As discussed at the sharding workshop last week, above definitions of starting and finishing gas heights can lead to suboptimal results if transactions are not ordered properly (or I have misunderstood the proposal). Example:

• MAX_THREADS = 2
• gas_limit = 150k
• tx1.gas_used = 50k
• tx2.gas_used = 50k
• tx3.gas_used = 100k
• tx4.gas_used = 100k
• All access lists are non-overlapping

If the transactions are ordered [tx1, tx2, tx3, tx4] the "gas ranges" (starting_gas_height and finishing_gas_height are as follows:

• tx1: 0 - 50k
• tx2: 0 - 50k
• tx3: 50k - 150k
• tx4: 50k - 150k

However, if the order is [tx3, tx1, tx2, tx4]:

• tx3: 0 - 100k
• tx1: 0 - 50k
• tx2: 100k - 150k
• tx4: 100k - 200k

So in this order the global gas limit would be exceeded, although a sensible miner strategy can execute the transactions in the same amount of time ("schedule the next transaction if there's a free thread and it doesn't overlap with any non-executed earlier transaction").

I think this could be fixed by keeping track of MAX_THREAD separate gas heights and, for each transaction, choosing the lowest possible one under the constraint of no conflicting parallel transactions.

I wanted to point this out, but in my opinion it is not worth implementing this more complicated algorithm. Both methods appear be equivalent if the miner (or, more generally, the proposer) chooses an optimal transaction order in the first place. This needs to be done anyway in order to maximize gas usage if transactions do have overlapping access lists.

Whatever happened to this? Long ago it was slated for metropolis

@joeykrug, cross-posting my reply from reddit:

It was discussed on an AllCoreDevs call around the time (it was never "slated for Metropolis", as in it was never officially Accepted for inclusion in the next HF on an All Core Devs call). As I recall, the feedback was that increasing the number of CPU threads doesn't address the actual bottleneck in scaling tx throughput, which is disk I/O (not CPU processing). You can see this feedback in the github issue itself and in vbuterin's reply:

This assumes that disk/memory operations are parallelizable, when in fact we usually find these to be some of the largest bottlenecks while optimizing.

I'd definitely be very interested in getting more data on parallelization and SSD reads.

Once it became widely realized that disk I/O is the bottleneck, research began on optimizing the state trie's data structure, and how it might be possible to parallelize SSD reads: Roadmap for Turbo-Geth.

After EIP 648, the next research thrust was on stateless clients: "this is in many ways a supercharged version of EIP 648." Stateless clients would enable a sort of 'parallelized state i/o' because all accessed state is pre-specified in the witness list (i.e. all accessed state is pre-loaded from disk into memory before the transaction is processed; in theory disk I/O of the validating node is no longer the bottleneck).

In subsequent months, discussions around sharding and stateless clients mainly revolve around the question of "stateless clients vs storage rent". And that brings us to the present day, ime.

Would this not allow the DAO-style soft forks, and with it, censorship? http://hackingdistributed.com/2016/07/05/eth-is-more-resilient-to-censorship/

If the ranges are tight, then miners can cheaply figure out what are the effects of the transaction, and censor them. If the range are broad, then there is less effect of the parallelisation. They way I see it - it is trading off censorship resistance for more parallel execution. Not sure this tradeoff is worth doing

There has been no activity on this issue for two months. It will be closed in a week if no further activity occurs. If you would like to move this EIP forward, please respond to any outstanding feedback or add a comment indicating that you have addressed all required feedback and are ready for a review.

This issue was closed due to inactivity. If you are still pursuing it, feel free to reopen it and respond to any feedback or request a review in a comment.

Why not use a similar approach to BlockSTM? I think Polygon has already implemented it for EVM or at least an MVP.

Why not use a similar approach to BlockSTM? I think Polygon has already implemented it for EVM or at least an MVP.

This belongs to pessimistic concurrency control, but BlockSTM is based on optimistic concurrency control. They are not very similar.