CIP-0132? | New Plutus Builtin DropList

cip-builtin-drop/README.md

@@ -0,0 +1,140 @@
+---
+CIP: ?
+Title: New Plutus Builtin dropList
+Status: Proposed
+Category: Plutus
+Authors:
+    - Philip DiSarro <info@anastasialabs.com>
+Implementors: []
+Discussions: 
+    - https://github.com/cardano-foundation/CIPs/pull/418/
+Created: 2024-02-25
+License: CC-BY-4.0
+---
+
+<!-- Existing categories:
+
+- Meta     | For meta-CIPs which typically serves another category or group of categories.
+- Wallets  | For standardisation across wallets (hardware, full-node or light).
+- Tokens   | About tokens (fungible or non-fungible) and minting policies in general.
+- Metadata | For proposals around metadata (on-chain or off-chain).
+- Tools    | A broad category for ecosystem tools not falling into any other category.
+- Plutus   | Changes or additions to Plutus
+- Ledger   | For proposals regarding the Cardano ledger (including Reward Sharing Schemes)
+- Catalyst | For proposals affecting Project Catalyst / the Jörmungandr project
+
+-->
+
+## Abstract
+<!-- A short (\~200 word) description of the proposed solution and the technical issue being addressed. -->
+This document describes the addition of a new Plutus builtin `dropList` with the signature `Integer -> List a -> List a` that drops a given number of elements the list. This drastically increases the efficiency of `elemAt` which is currently a huge throughput bottleneck for many DApps. 
+
+## Motivation: why is this CIP necessary?
+<!-- A clear explanation that introduces the reason for a proposal, its use cases and stakeholders. If the CIP changes an established design then it must outline design issues that motivate a rework. For complex proposals, authors must write a Cardano Problem Statement (CPS) as defined in CIP-9999 and link to it as the `Motivation`. -->
+
+The deterministic script evaluation property of the ledger (also stated as "script interpreter arguments are fixed") is a unique characteristic of the Cardano ledger that allows us to perform powerful optimizations that are not possible in systems with indeterminstic script evaluation. For instance, searching for elements in a data structure can 
+be done entirely off-chain, and then we simply provide the onchain code with the index (via redeemer) to where the element we want to find is supposed to be, and then check (onchain) that it is indeed the element we were expecting. This design pattern of passing the indices of elements required for validation logic in the redeemer is commonly referred to as redeemer-indexing. 
+Even though it is still a very powerful optimization in its current state, it is currently bottlenecked by the lack of a builtin that applies tail a given number of times to a list. Currently, any implementation of `elemAt :: Integer -> List a -> a` or `drop` requires the use of the fixed point combinator (Y combinator) which has a significant cost in onchain code.
+
+Consider the naive approach:
+```haskell
+{- | Fixpoint recursion. Used to encode recursive functions.
+     Hopefully this illustrates the overhead that this incurs. 
+-}
+pfix :: Term s (((a :--> b) :--> a :--> b) :--> a :--> b)
+pfix = phoistAcyclic $
+  punsafeCoerce $
+    plam' $ \f ->
+      plam' (\(x :: Term s POpaque) -> f # plam' (\(v :: Term s POpaque) -> punsafeCoerce x # x # v))
+        # punsafeCoerce (plam' $ \(x :: Term s POpaque) -> f # plam' (\(v :: Term s POpaque) -> punsafeCoerce x # x # v))
+
+-- | Lazy if-then-else
+-- Two forces + two delays + builtinIfThenElse
+pif :: Term s PBool -> Term s a -> Term s a -> Term s a
+pif b case_true case_false = pforce $ (pforce $ punsafeBuiltin PLC.IfThenElse) # b # pdelay (f PTrue) # pdelay (f PFalse) 
+ where
+   f = \case
+     PTrue -> case_true
+     PFalse -> case_false
+
+pelemAt' :: PIsListLike l a => Term s (PInteger :--> l a :--> a)
+pelemAt' = phoistAcyclic $
+  pfix #$ plam $ \self n xs ->
+    pif
+      (n #== 0)
+      (phead # xs)
+      (self # (n - 1) #$ ptail # xs)
+
+pelemAt' # 5 # (pconstant [1,2,3,4,5])
+-- the function `self` must be passed as an argument to each recursive call.
+-- each recursive call results in:
+--   uplc Apply operations to apply the arguments `n` and `xs` to `self`
+--   lazy ifThenElse (two forces + two delays + builtinIfThenElse)
+--   builtinEqualsInteger
+--   builtinSubtractInteger
+--   uplc Apply operations to apply the arguments (including `self`, `n` and `xs`) to the fixed-point recursive function
+``` 
+As you can see, the naive `elemAt` implementation is quite inefficient. This is a huge efficiency bottleneck for many DApps which use `elemAt` many times to locate elements at indices specified in the redeemer. In an attempt to address this, many protocols use the following heuristic optimization (where the number of skips is determined through trial and error based on the DApps throughput in testing):
+```
+pelemAtFast :: PIsListLike l a => Term s (PInteger :--> l a :--> a)
+pelemAtFast = phoistAcyclic $
+  pfix #$ plam $ \self n xs ->
+    pif
+      (n #> 10)
+      (self # (n - 1) #$ ptail #$ ptail #$ ptail #$ ptail #$ ptail #$ ptail #$ ptail #$ ptail #$ ptail #$ ptail # xs)
+      (pif
+         (n #> 5) 
+         (self # (n - 5) #$ ptail #$ ptail #$ ptail #$ ptail #$ ptail # xs)
+         (pif (n #== 0) (phead # xs) (pelemAt' # (n - 1) # (ptail # xs))))
+```
+This drastically reduces the amount of recursion we have to do which greatly increases the efficiency of this function in practice. However, it should be clear to see that there is still a huge degree of inefficiency in this implementation. Also it is difficulty to determine the correct magic numbers to skip, and the performance
+varies drastically depending on the cut-off values chosen for `n` as-well as the number of different skip-cases (in this case we have skip cases for both `n > 10` and `n > 5`). 
+
+## Specification
+<!-- The technical specification should describe the proposed improvement in sufficient technical detail. In particular, it should provide enough information that an implementation can be performed solely on the basis of the design in the CIP. This is necessary to facilitate multiple, interoperable implementations. This must include how the CIP should be versioned. If a proposal defines structure of on-chain data it must include a CDDL schema in it's specification.-->
+### Function definition
+We define a new Plutus built-in function with the following type signature:
+```haskell
+builtinDropList :: BuiltinInteger -> BuiltinList a -> BuiltinList a
+```
+
+Similar to the behavior of the `indexOfByteString` builtin, this new builtin will simply error if the provided index is out of bounds for the list.
+
+
+### Cost Model
+Although the `BuiltinList` type is a recursive data-type, costing should be relatively straightforward. 
+We propose to define a cost model linear in the size of `n`, the number of elements to drop. What remains is to find a proper coefficient and offset for that linear model, which should be quite easy. 
+
+
+## Rationale: how does this CIP achieve its goals?
+<!-- The rationale fleshes out the specification by describing what motivated the design and what led to particular design decisions. It should describe alternate designs considered and related work. The rationale should provide evidence of consensus within the community and discuss significant objections or concerns raised during the discussion.
+
+It must also explain how the proposal affects the backward compatibility of existing solutions when applicable. If the proposal responds to a CPS, the 'Rationale' section should explain how it addresses the CPS, and answer any questions that the CPS poses for potential solutions.
+-->
+* Easy to implement as it reuses existing code of the Plutus codebase;
+* The built-in is generic enough to cover a wider set of use-cases;
+* The built-in is still relevant even if we get constant lookup index data-structures since there are occasions where BuiltinList would be preferred;
+* This directly addresses the big performance bottleneck that the fixed-point recursion implementation of `elemAt` and `drop` impose on many DApps;
+
+### Alternatives 
+
+- We could decide to accept the heuristic `elemAtFast` implementation as  as an adequate solution.
+- We could provide a more generic builtin that applies a function recursively `n` times (seems complicated and bad idea). 
+- We could try to reduce the overhead introduced by aspects of the `elemAt` by making the language / compiler more performant (still can't imagine we would be able to get anywhere near the performance of this builtin).
+
+## Path to Active
+
+### Acceptance Criteria
+<!-- Describes what are the acceptance criteria whereby a proposal becomes 'Active' -->
+- [] Fully implemented in Cardano.
+
+### Implementation Plan
+<!-- A plan to meet those criteria. Or `N/A` if not applicable. -->
+- [] Passes all requirements of both Plutus and Ledger teams as agreed to improve Plutus script efficiency and usability.
+
+## Copyright
+<!-- The CIP must be explicitly licensed under acceptable copyright terms. -->
+This CIP is licensed under [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/legalcode).
+
+[CC-BY-4.0]: https://creativecommons.org/licenses/by/4.0/legalcode
+[Apache-2.0]: http://www.apache.org/licenses/LICENSE-2.0