CIP-0068? | Datum Metadata Standard

This proposal defines a metadata standard for native assets making use of output datums not only for NFTs but any asset class.

[This is my personal GitHub by I am @WildTangz / @iOddestHumanDad on Twitter]
I like the proposal and think it has some good use cases as you mentioned.  Just to clarify my understanding:

Reference NFT Evolution - Imagine an NFT collection that had a single attribute we wanted to evolve (e.g., IPFS file, metadata, etc.).  With this standard, we could simply allow the owners to keep the NFT tokens they have in their wallets, use our private keys for the reference NFT and simply modify the datum and relock that asset.  All the viewers that used this standard would automatically receive updates.
On-Chain Validation - Imagine an NFT collection that carried governance (like a lightweight DAO).  In this case, the DAO could entirely (or, almost entirely) be run through an on-chain voting protocol instead of requiring owners to validate ownership off-chain with their signatures.  This use case is even better with Vasil reference inputs because users now can simply reference their user token in the transaction they submit and voila, votes are sent.

If I am understanding correctly I think there are a few potential downsides:

Costs - As @Crypto2099 mentioned each asset has to be locked inside of a UTXO with a minimum set by the Vasil parameters.  This doubles the costs for any new NFT project trying to perform a new mint (which can be prohibitive for large collections).
Intellectual Property - The NFT in this case is no longer the sole arbiter of the information about the NFT.  While in some cases (see "Reference NFT Evolution" above) this can be beneficial, in some cases this is actually worse.  For example, several projects (mine included) assign certain IP rights to the asset holder.  Who has those rights in this case? What is the governance for removing/adding rights? What about private key phishes which would lead to a major collection potentially being entirely rugged?
Expiration - This is a problem with existing projects but because you are explicitly tying the policy of the user token to the reference NFT you are not able to evolve the NFT past the expiration slot of the NFT project.  This is likely desired behavior, but it might be worth explicitly mentioning to alert folks that this standard does not obviate the need for choosing an expiration slot wisely.

Couple of questions for you @alessandrokonrad:

If we are going to discuss putting metadata on-chain, is it worth abandoning CIP-0025 for this standard altogether? I am not trying to reinvent the wheel, but if we are going to push innovation on-chain perhaps we should think about a different (or at least, slightly more flexible) format so that the reference NFT can actually represent arbitrary code.  This would be useful for projects like unsig.
Is it wise to use Datum to encode JSON in a fully expanded formats? Since we are already representing JSON as the serialized Datum object, perhaps there is a way to do this more compactly to avoid adding size to the blockchain.  Not sure how efficiently Datum are represented, but some of the on-chain art that fits inside a compressed 16KB metadata today may not fit in the new serialization scheme.

[This is my personal GitHub but I am @WildTangz / @iOddestHumanDad on Twitter]
I like the proposal and think it has some good use cases as you mentioned. Just to clarify my understanding:

Reference NFT Evolution - Imagine an NFT collection that had a single attribute we wanted to evolve (e.g., IPFS file, metadata, etc.).  With this standard, we could simply allow the owners to keep the NFT tokens they have in their wallets, use our private keys for the reference NFT and simply modify the datum and relock that asset.  All the viewers that used this standard would automatically receive updates.
On-Chain Validation - Imagine an NFT collection that carried governance (like a lightweight DAO).  In this case, the DAO could entirely (or, almost entirely) be run through an on-chain voting protocol instead of requiring owners to validate ownership off-chain with their signatures.  This use case is even better with Vasil reference inputs because users now can simply reference their user token in the transaction they submit and voila, votes are sent.

If I am understanding correctly I think there are a few potential downsides:

Costs - As @Crypto2099 mentioned each asset has to be locked inside of a UTXO with a minimum set by the Vasil parameters.  This doubles the costs for any new NFT project trying to perform a new mint (which can be prohibitive for large collections).
Intellectual Property - The NFT in this case is no longer the sole arbiter of the information about the NFT.  While in some cases (see "Reference NFT Evolution" above) this can be beneficial, in some cases this is actually worse.  For example, several projects (mine included) assign certain IP rights to the asset holder.  Who has those rights in this case? What is the governance for removing/adding rights? What about private key phishes which would lead to a major collection potentially being entirely rugged?
Expiration - This is a problem with existing projects but because you are explicitly tying the policy of the user token to the reference NFT you are not able to evolve the NFT past the expiration slot of the NFT project.  This is likely desired behavior, but it might be worth explicitly mentioning to alert folks that this standard does not obviate the need for choosing an expiration slot wisely.

Couple of questions for you @alessandrokonrad:

If we are going to discuss putting metadata on-chain, is it worth abandoning CIP-0025 for this standard altogether? I am not trying to reinvent the wheel, but if we are going to push innovation on-chain perhaps we should think about a different (or at least, slightly more flexible) format so that the reference NFT can actually represent arbitrary code.  This would be useful for projects like unsig.
Is it wise to use Datum to encode JSON in a fully expanded formats? Since we are already representing JSON as the serialized Datum object, perhaps there is a way to do this more compactly to avoid adding size to the blockchain.  Not sure how efficiently Datum are represented, but some of the on-chain art that fits inside a compressed 16KB metadata today may not fit in the new serialization scheme.

Right off the bat, there is no need for overly complicating evolving metadata.  You dont need to hold an asset to update its metadata.  You just mint another with the updated metadata, then burn it.  The holders asset is thusly updated.  No smart contract required, no locked ada, and no dodgy wallet logic to say "If I hold this asset, show this other assets metadata".

Aside from the locked Ada, I see other complications that can come out of this, including introducing attack vectors that are otherwise non existent with native assets.

I'm not sure what attack vectors you mean exactly, but if you talk about usage of Plutus scripts then you face the same risks with any dApp you interact. But you can verify the contracts as they should be open source.

If the goal is to have update-able metadata; I've already solved this with the GYOA platform for Cardano Budz, and am happy to share my concepts with you. It doesnt require any locking of ada, nor a smart contract; which only adds more bloat.

That's not the goal, but it's what the standard allows you to do as it's very generalized. I'm not sure how you achieve utxo authenticity without a native asset. With the standard you should be able to bring the metadata securely into the plutus validator context. But I'm happy to hear your solution.

The next complication is getting all wallet makers to adopt this new standard; which seems fairly unlikely.
I don't think so

It would be much better to have an open dialogue with asset makers. Much like Open Source software, the final product can be substantially better with more people's input.

Well that's why this CIP is open. Anyone is free to contribute to it.

[This is my personal GitHub but I am @WildTangz / @iOddestHumanDad on Twitter]
I like the proposal and think it has some good use cases as you mentioned. Just to clarify my understanding:

Reference NFT Evolution - Imagine an NFT collection that had a single attribute we wanted to evolve (e.g., IPFS file, metadata, etc.).  With this standard, we could simply allow the owners to keep the NFT tokens they have in their wallets, use our private keys for the reference NFT and simply modify the datum and relock that asset.  All the viewers that used this standard would automatically receive updates.
On-Chain Validation - Imagine an NFT collection that carried governance (like a lightweight DAO).  In this case, the DAO could entirely (or, almost entirely) be run through an on-chain voting protocol instead of requiring owners to validate ownership off-chain with their signatures.  This use case is even better with Vasil reference inputs because users now can simply reference their user token in the transaction they submit and voila, votes are sent.

If I am understanding correctly I think there are a few potential downsides:

Costs - As @Crypto2099 mentioned each asset has to be locked inside of a UTXO with a minimum set by the Vasil parameters.  This doubles the costs for any new NFT project trying to perform a new mint (which can be prohibitive for large collections).
Intellectual Property - The NFT in this case is no longer the sole arbiter of the information about the NFT.  While in some cases (see "Reference NFT Evolution" above) this can be beneficial, in some cases this is actually worse.  For example, several projects (mine included) assign certain IP rights to the asset holder.  Who has those rights in this case? What is the governance for removing/adding rights? What about private key phishes which would lead to a major collection potentially being entirely rugged?
Expiration - This is a problem with existing projects but because you are explicitly tying the policy of the user token to the reference NFT you are not able to evolve the NFT past the expiration slot of the NFT project.  This is likely desired behavior, but it might be worth explicitly mentioning to alert folks that this standard does not obviate the need for choosing an expiration slot wisely.

Couple of questions for you @alessandrokonrad:

If we are going to discuss putting metadata on-chain, is it worth abandoning CIP-0025 for this standard altogether? I am not trying to reinvent the wheel, but if we are going to push innovation on-chain perhaps we should think about a different (or at least, slightly more flexible) format so that the reference NFT can actually represent arbitrary code.  This would be useful for projects like unsig.
Is it wise to use Datum to encode JSON in a fully expanded formats? Since we are already representing JSON as the serialized Datum object, perhaps there is a way to do this more compactly to avoid adding size to the blockchain.  Not sure how efficiently Datum are represented, but some of the on-chain art that fits inside a compressed 16KB metadata today may not fit in the new serialization scheme.

Right off the bat, there is no need for overly complicating evolving metadata. You dont need to hold an asset to update its metadata. You just mint another with the updated metadata, then burn it. The holders asset is thusly updated. No smart contract required, no locked ada, and no dodgy wallet logic to say "If I hold this asset, show this other assets metadata".

That is true you can do that, but there are some drawbacks to that approach (namely diluting the supply, albeit temporarily and confusing intrepid blockchain explorers).  The contribution here to update the metadata without changing supply is novel in my opinion.

My primary concern would be the Ada/Lovelace "locked" via the minUTXO but the secondary concern is for ledger bloat due to needing, for example, 10k UTXO on-chain that can never be modified, consolidated, or moved without "rugging" the user tokens. So, particularly for a "locked" token/set... How is this method preferable to the existing CIP-25 metadata standard? I suppose in this case the answer is "it's not" and this particular CIP would actually more be suitable in cases where there is a great and on-going need to modify or change the metadata of a user token based on some sort of validation or approval via the token issuer (policy holder and controller of the reference tokens). i.e. I want to update my phone number in my DID... I go to website example.com and perform an update after authenticating myself, and example.com issues the update to my DID reference token. Still not 100% sure that this is ultimately the best approach due to the aforementioned "issues" but worth considering.

Is there any consideration given or made for the amount of Lovelace that would potentially need to be permanently locked to hold individual metadatums for each token? For a 10k NFT set that is "locked" you're potentially talking about permanently sequestering ~15k ADA just to utilize this method.
Good question and yes, we have. Note that the locking script for the reference NFT can have arbitrary logic. Consider the following script, make every reference NFT burnable to redeem that value at that output (together with its associated NFT of course). This output thus acts as a base value for that token. You even can make it such that once a NFT is burned, it can be minted again by someone else (if they deem the artwork valuable).

Key point here, this is a fundamental property of the ledger and has nothing to do with how you implement your script, or this standard for that matter. The standard opens up all the possibilities that plutus has for the developer, it is our responsibility to be creative with it!

Right off the bat, there is no need for overly complicating evolving metadata. You dont need to hold an asset to update its metadata. You just mint another with the updated metadata, then burn it. The holders asset is thusly updated. No smart contract required, no locked ada, and no dodgy wallet logic to say "If I hold this asset, show this other assets metadata".

Thank you for adding to the dialogue! I know that currently we have a system for updating NFT's in CIP25, but there is a big problem with that way of doing things, it is centralized and errors may occur. The artist must check that the correct metadata is put in while burning/minting it, plutus cannot access that. We need a one size fits all standard that will solve the problems across the board for all token classes. That is what this standard is achieving.

Reference NFT Evolution - Imagine an NFT collection that had a single attribute we wanted to evolve (e.g., IPFS file, metadata, etc.). With this standard, we could simply allow the owners to keep the NFT tokens they have in their wallets, use our private keys for the reference NFT and simply modify the datum and relock that asset. All the viewers that used this standard would automatically receive updates.

Correct, but you are not only limited by using your private keys to update them as a creator of the collection. You can utilize all the power of plutus! With plutus, you can predetermine exactly under what conditions metadata is update so that no central party has to control it.

On-Chain Validation - Imagine an NFT collection that carried governance (like a lightweight DAO). In this case, the DAO could entirely (or, almost entirely) be run through an on-chain voting protocol instead of requiring owners to validate ownership off-chain with their signatures. This use case is even better with Vasil reference inputs because users now can simply reference their user token in the transaction they submit and voila, votes are sent.

This is also correct, thought I must say I dislike the idea of putting votes publicly on a blockchain (you can attack opponents of your view). Instead, people should use protocols like mixing nets to obscure the origin of the vote without losing the ability to verify the vote. A lot of research is already put into this. But that's just my two ada.

Costs - As @Crypto2099 mentioned each asset has to be locked inside of a UTXO with a minimum set by the Vasil parameters. This doubles the costs for any new NFT project trying to perform a new mint (which can be prohibitive for large collections).

You do not have to mint tokens directly. You can use Merkle trees in your minting policy to predetermine exactly what a collection might entail. Also see the post above about implementing a burning function in the policy. Again, the point of the standard is to open up the full power of plutus.

Intellectual Property - The NFT in this case is no longer the sole arbiter of the information about the NFT. While in some cases (see "Reference NFT Evolution" above) this can be beneficial, in some cases this is actually worse. For example, several projects (mine included) assign certain IP rights to the asset holder. Who has those rights in this case? What is the governance for removing/adding rights? What about private key phishes which would lead to a major collection potentially being entirely rugged?

This is more of a legal matter, but my guess is that the one who controls the value has the IP, that is not the reference NFT. This is just a placeholder in a database owned by no one that stores data. I would like to restate that the plutus script controls this reference NFT, not necessarily a normal signing key that is associated with another person's wallet. I thus do not see this problem.

Expiration - This is a problem with existing projects but because you are explicitly tying the policy of the user token to the reference NFT you are not able to evolve the NFT past the expiration slot of the NFT project. This is likely desired behavior, but it might be worth explicitly mentioning to alert folks that this standard does not obviate the need for choosing an expiration slot wisely.

Yeah, the minting with time locking simple scripts is not optimal since it is centralized.

If we are going to discuss putting metadata on-chain, is it worth abandoning CIP-0025 for this standard altogether? I am not trying to reinvent the wheel, but if we are going to push innovation on-chain perhaps we should think about a different (or at least, slightly more flexible) format so that the reference NFT can actually represent arbitrary code. This would be useful for projects like unsig.

This standard allows for the wrapping of existing tokens by creating a script with predetermined metadata associated with the previous collection (using again Merkle trees). So, if the right tools are built, the standard is not necessary anymore.

Is it wise to use Datum to encode JSON in a fully expanded formats? Since we are already representing JSON as the serialized Datum object, perhaps there is a way to do this more compactly to avoid adding size to the blockchain. Not sure how efficiently Datum are represented, but some of the on-chain art that fits inside a compressed 16KB metadata today may not fit in the new serialization scheme.

Fair and good question! We struggled as well with the way to represent data on-chain in the datum when writing this CIP. Initially, we tried to decode the data in CBOR, but this raised issues with on-chain decoding, which consumes a lot of execution units. See (1).
I just commented on all your remarks, hope my answer are okay for you. Thanks for your input and effort regarding this CIP! Much appreciated :)

Reference NFT Evolution - Imagine an NFT collection that had a single attribute we wanted to evolve (e.g., IPFS file, metadata, etc.). With this standard, we could simply allow the owners to keep the NFT tokens they have in their wallets, use our private keys for the reference NFT and simply modify the datum and relock that asset. All the viewers that used this standard would automatically receive updates.

Correct, but you are not only limited by using your private keys to update them as a creator of the collection. You can utilize all the power of plutus! With plutus, you can predetermine exactly under what conditions metadata is update so that no central party has to control it.

On-Chain Validation - Imagine an NFT collection that carried governance (like a lightweight DAO). In this case, the DAO could entirely (or, almost entirely) be run through an on-chain voting protocol instead of requiring owners to validate ownership off-chain with their signatures. This use case is even better with Vasil reference inputs because users now can simply reference their user token in the transaction they submit and voila, votes are sent.

This is also correct, thought I must say I dislike the idea of putting votes publicly on a blockchain (you can attack opponents of your view). Instead, people should use protocols like mixing nets to obscure the origin of the vote without losing the ability to verify the vote. A lot of research is already put into this. But that's just my two ada.

I agree with this.  There is some good research on this from a while back I knew about: https://dl.acm.org/doi/10.1145/1866307.1866346.  It uses xors to send encrypted communications (a communication could just be a vote).

Costs - As @Crypto2099 mentioned each asset has to be locked inside of a UTXO with a minimum set by the Vasil parameters. This doubles the costs for any new NFT project trying to perform a new mint (which can be prohibitive for large collections).

You do not have to mint tokens directly. You can use Merkle trees in your minting policy to predetermine exactly what a collection might entail. Also see the post above about implementing a burning function in the policy. Again, the point of the standard is to open up the full power of plutus.

Sorry I meant not the cost, but the minUTXO you have to lock up.  I get that this is technically "the user's money", but right now you only have to send (1 x minUTXO) back with each asset.  This proposal would double that, lowering creator's fees.

Intellectual Property - The NFT in this case is no longer the sole arbiter of the information about the NFT. While in some cases (see "Reference NFT Evolution" above) this can be beneficial, in some cases this is actually worse. For example, several projects (mine included) assign certain IP rights to the asset holder. Who has those rights in this case? What is the governance for removing/adding rights? What about private key phishes which would lead to a major collection potentially being entirely rugged?

This is more of a legal matter, but my guess is that the one who controls the value has the IP, that is not the reference NFT. This is just a placeholder in a database owned by no one that stores data. I would like to restate that the plutus script controls this reference NFT, not necessarily a normal signing key that is associated with another person's wallet. I thus do not see this problem.

Definitely a legal matter.  This is more along the lines of "the difference between theory and practice is in theory quite small, but in practice quite large".  I don't think the standard itself implies anything from it, it is just that if the standard is widely adopted, it could add friction to the creator ecosystem.  m2c

Expiration - This is a problem with existing projects but because you are explicitly tying the policy of the user token to the reference NFT you are not able to evolve the NFT past the expiration slot of the NFT project. This is likely desired behavior, but it might be worth explicitly mentioning to alert folks that this standard does not obviate the need for choosing an expiration slot wisely.

Yeah, the minting with time locking simple scripts is not optimal since it is centralized.

Agreed.

If we are going to discuss putting metadata on-chain, is it worth abandoning CIP-0025 for this standard altogether? I am not trying to reinvent the wheel, but if we are going to push innovation on-chain perhaps we should think about a different (or at least, slightly more flexible) format so that the reference NFT can actually represent arbitrary code. This would be useful for projects like unsig.

This standard allows for the wrapping of existing tokens by creating a script with predetermined metadata associated with the previous collection (using again Merkle trees). So, if the right tools are built, the standard is not necessary anymore.

But, unless I am mistaken, you are specifying the metadata structure as serialized JSON inside of this proposal.  Is this proposal just about arbitrary bytestrings? For example, I would love a metadata standard that enabled me to do columnar compression of PFP traits (e.g., "Wings" -> 0x00000001, "Tails" -> 0x00000010) and then I can compose them on-chain with an AND.  If I put that in the reference NFT as of CIP-0068?, it would not conform to the standard as I understand it.

Is it wise to use Datum to encode JSON in a fully expanded formats? Since we are already representing JSON as the serialized Datum object, perhaps there is a way to do this more compactly to avoid adding size to the blockchain. Not sure how efficiently Datum are represented, but some of the on-chain art that fits inside a compressed 16KB metadata today may not fit in the new serialization scheme.

Fair and good question! We struggled as well with the way to represent data on-chain in the datum when writing this CIP. Initially, we tried to decode the data in CBOR, but this raised issues with on-chain decoding, which consumes a lot of execution units. See (1).

Fair.  I would like to see support for more columnar compression formats that are used in the Big Data ecosystem (last time I checked Avro was popular), but likely out of scope for this CIP.

I just commented on all your remarks, hope my answer are okay for you. Thanks for your input and effort regarding this CIP! Much appreciated :)

No worries, thanks for the consideration!

The idea of accessing metadata in Plutus Scripts is definitely something that would expand the possibilities of what the Cardano Smart contract can achieve in many aspects.
The proposed solutions suggest for a standard applicable to all versions of Plutus that doesn't require changes to the protocol
I'd like to suggest an alternative way to access metadata through script context.
This idea obviously requires a new (extended) format of the ScriptContext provided to the script, but it comes with the advantage of being easier to use and implement.
The high-level idea is to have a field in the ScriptContext containing the metadata attached to the validating transaction (if any)
as mentioned by @alessandrokonrad in a brief open discussion on Twitter including the metadata in the script context doesn't really solve for the accessibility of metadata on-chain outside the minting transaction.
to overcome this issue could be proposed a standard that allows exposing the metadata through an oracle.
The high-level idea of such a standard is to have a minting policy that checks for an UTxO to be created at the address of such Oracle, containing the information present in the metadata of the minting transaction in the attached Datum; and fails otherwise.
this way the metadata information is stored at the Oracle address and stays on-chain

to overcome this issue could be proposed a standard that allows exposing the metadata through an oracle.
The high-level idea of such a standard is to have a minting policy that checks for an UTxO to be created at the address of such Oracle, containing the information present in the metadata of the minting transaction in the attached Datum; and fails otherwise.
this way the metadata information is stored at the Oracle address and stays on-chain

The issue is solved within the CIP. You have the Reference NFT locked in an output containing the datum with the metadata. The NFT you have in your wallet points to the Reference NFT and its ouput so you know where to get the data from and it can be securely brought it into the plutus validator context.

This CIP needs a Rationale section that explains how it achieves its goals, and in particular whether it is secure.
My reading of it is that this is much more secure than CIP-25, which is I believe seriously flawed (https://github.com/cardano-foundation/CIPs/pull/85#issuecomment-1054123645). The corresponding risk here is that this CIP imposes an assumption that asset names have a particular meaning, which may not be true for an arbitrary asset (and you can't tell without... metadata, and if you already have a functioning metadata channel you don't need this).
However, it seems much harder for an unsuspecting token issuer to be attacked here. They would have to:

Issue tokens which match the asset name label format that you use, without being aware of this standard.
Allow the minting of a token which matched the reference NFT lable and allow it to be controlled by an attacker.

This isn't totally impossible. Consider e.g. a hypothetical dapp that issues tokens for user accounts. They innocently choose the following format for their asset names: (<user number>)<user handle>. If this CIP is widely used, then user 100 would get the token (100)BobTheAttacker and unexpectedly gain the power to set the metadata for all of the user account tokens!
So as before, to be truly safe this standard would have to be opt-in, but there is no way to tell whether a token has opted in without... metadata.
Hence this CIP does still open an attack vector on innocent third party token issuers, and I would like the proposal to clearly explain this and declare that it is to be considered an acceptable risk.

Thanks for the feedback @michaelpj!

This CIP needs a Rationale section that explains how it achieves its goals, and in particular whether it is secure.

Will work on a Rational section to make this more clear.👍🏻

However, it seems much harder for an unsuspecting token issuer to be attacked here. They would have to:

Issue tokens which match the asset name label format that you use, without being aware of this standard.
Allow the minting of a token which matched the reference NFT lable and allow it to be controlled by an attacker.

This isn't totally impossible. Consider e.g. a hypothetical dapp that issues tokens for user accounts. They innocently choose the following format for their asset names: (<user number>)<user handle>. If this CIP is widely used, then user 100 would get the token (100)BobTheAttacker and unexpectedly gain the power to set the metadata for all of the user account tokens!

This is why the reference NFT and user token have to be under the same policy ID to derive the security from the script (ideally a plutus script) and avoid any spoofing attacks. Of course the script logic needs to be written in a way that it adds the necessary constraints to how reference NFT and user token can be minted. This CIP defines only the interface to keep it very flexible, the implementation is up to the token creators. But I agree it should be made more clear that this CIP is only effective if validators are written with certain rules and constraints.
The advantage of this approach is that you have a very strong link between the two tokens just from the policy id and the asset name without the need for additional metadadata.

Very interesting read and many possible gains contained in this specification. The standard proposal elegantly solves the need for making metadata available also for smart contract validators and at the same time to lay ground for functioning as a decentralized on-chain token name registry! The concept of token classes also support the possibility of have different property standards for different token types. Really exciting ideas!
As has been mentioned by several people already, the cost related to locked lovelaces in lots of reference NFTs was an unwanted consequence, especially for NFT user tokens collections.
The specification contains rules that NFT user tokens must refer to exactly one corresponding reference NFT with the exact name, while all FT user tokens (number > 1 of the same type) can all refer to the same reference NFT because they all have the same name.
Have you considered describing the possibility of specifying a regular expression or valid id range in the reference NFT instead of exact comparison of token names? In that case, this could lead to each NFT collection also being able to refer to only one reference NFT that works for the entire collection.

Have you considered describing the possibility of specifying a regular expression or valid id range in the reference NFT instead of exact comparison of token names? In that case, this could lead to each NFT collection also being able to refer to only one reference NFT that works for the entire collection.

Yeah I thought about that, this would work perfectly for NFTs that share the same metadata.
I'm not sure if you also mean having a single reference NFT where you have batched together metadata for different NFTs. If so this has a few downsides. It adds a lot of complexity to the script logic in case you want to update the metadata for a specific NFT. It may work fine for entirely immutable NFTs, but still if you want to reference the metadata in a plutus script you have all the metadata from the other NFTs in the validator as well, which increases script costs and requires extra checks to make sure the correct metadata are chosen.

Of course the script logic needs to be written in a way that it adds the necessary constraints to how reference NFT and user token can be minted. This CIP defines only the interface to keep it very flexible, the implementation is up to the token creators. But I agree it should be made more clear that this CIP is only effective if validators are written with certain rules and constraints.

This is exactly the hole that I'm pointing to. You have no way of knowing whether a minting policy was written in any particular way (without another metadata channel).  What is dangerous is to _assume _ that everyone will follow a policy when it is not enforced.
More strongly, it's not the case that this CIP is "ineffective" if people don't follow the constraints, rather it creates a new attack vector against non-confirming users. I think actively reducing the security of innocent other network participants is pretty bad and needs to be taken seriously.
Like I said, I think the policy is complicated enough that it's fairly unlikely that someone would accidentally produce asset names that appear to follow it. But looking at it from the security angle suggests that it might be a cheap win to make the policy more obscure, so that we can argue that it is really extremely unlikely to be used by accident.

Great work, would open up a lot of possibilities!
But, just wanted to add some long-term perspective on the issue of the locked ADA. We all like to believe Cardano is going to be around for a very long time, so 100 year should not seem unreasonable :-)
In the first year of Cardano NFTs we had around 5M assets issued. Even assuming a linear growth (very unlikely), in 100 year that would mean 500M assets, resulting in 750M ADA locked.
In 1000 years, 1/6 of all ADA will need to be locked.
Of course, not all NFTs will be locked but the amount could still end up being astronomic.
Even if the specification only requires one reference asset per policy, we currently have 55K collections (according to https://www.cnftjungle.io/statistics), meaning that it could still be a problem over the long horizon, because again, the growth will most likely not be linear.
Either the minimum ADA required to be sent/locked with a token is drastically reduced or this becomes unfeasible in the long-run.

But, just wanted to add some long-term perspective on the issue of the locked ADA. We all like to believe Cardano is going to be around for a very long time, so 100 year should not seem unreasonable :-)
In the first year of Cardano NFTs we had around 5M assets issued. Even assuming a linear growth (very unlikely), in 100 year that would mean 500M assets, resulting in 750M ADA locked. In 1000 years, 1/6 of all ADA will need to be locked. Of course, not all NFTs will be locked but the amount could still end up being astronomic.
Even if the specification only requires one reference asset per policy, we currently have 55K collections (according to https://www.cnftjungle.io/statistics), meaning that it could still be a problem over the long horizon, because again, the growth will most likely not be linear.
Either the minimum ADA required to be sent/locked with a token is drastically reduced or this becomes unfeasible in the long-run.

I get your point, but with Plutus you can implement a native burning mechanism, which would allow you to redeem these ADA again. Of course if people continue using only time locked policies it's bad, but this CIP more or less forces you to write a plutus validator, so you can also think about a burning mechanism.

The entire comment is based on the assumption that a large number of NFTs would still be time locked, otherwise it’s not a problem, of course.
Would it be possible to execute the burning mechanism before the NFTs are locked?
So that it would result in both a locked policy and recovering the ADA.
The thing is that locked NFTs are ingrained in the culture at this point, and not just on Cardano, meaning that the demand for them will still be here. A lot of people don’t even consider open NFTs as “true NFTs”.
We’re probably going to see a lot of new applications where open policies are the norm, but for art and collectible NFTs in particular, most of the arguments for locking make sense. They especially make sense if you consider the long-term, decades, centuries, as the risk of policy keys being compromised increases.
Think of traditional art collectors, they naturally take the long-term view. You are way less likely to buy a piece of very expensive art if you know there’s a possibility that it could be compromised later on. And I think we can all agree on the fact that we do want to onboard as many art collectors as possible.
As a personal anecdote, we created the policy for our collectibles in May of last year, and because we wanted to keep things as flexible as possible and be able to take advantage of new developments as they become available, we chose an open policy. But we got complaints about that over the last 14 months. In our case, we could always satisfactorily explain why we made the choice, as it made sense and fits in with a wider vision for the collection, but that’s mostly not the case.
The demand for locked NFTs is not just a Cardano thing either. This could mean that if it’s perceived to be more difficult to implement locked NFTs on Cardano it could hurt adoption. It doesn’t even have to be true, the perception will be enough.

The entire comment is based on the assumption that a large number of NFTs would still be time locked, otherwise it’s not a problem, of course.
Would it be possible to execute the burning mechanism before the NFTs are locked? So that it would result in both a locked policy and recovering the ADA.
The thing is that locked NFTs are ingrained in the culture at this point, and not just on Cardano, meaning that the demand for them will still be here. A lot of people don’t even consider open NFTs as “true NFTs”.
We’re probably going to see a lot of new applications where open policies are the norm, but for art and collectible NFTs in particular, most of the arguments for locking make sense. They especially make sense if you consider the long-term, decades, centuries, as the risk of policy keys being compromised increases. Think of traditional art collectors, they naturally take the long-term view. You are way less likely to buy a piece of very expensive art if you know there’s a possibility that it could be compromised later on. And I think we can all agree on the fact that we do want to onboard as many art collectors as possible.
As a personal anecdote, we created the policy for our collectibles in May of last year, and because we wanted to keep things as flexible as possible and be able to take advantage of new developments as they become available, we chose an open policy. But we got complaints about that over the last 14 months. In our case, we could always satisfactorily explain why we made the choice, as it made sense and fits in with a wider vision for the collection, but that’s mostly not the case.
The demand for locked NFTs is not just a Cardano thing either. This could mean that if it’s perceived to be more difficult to implement locked NFTs on Cardano it could hurt adoption. It doesn’t even have to be true, the perception will be enough.

I think there is a huge misunderstanding in the ecosystem about "locked" vs "open" policies. If we talk purely about native scripts, I agree time locking is the only way to achieve immutability. But with Plutus things change. The possibilities are endless since you can embed any logic you want into the Plutus script. And actually it's far more elegant, people are just so used to time locking and simple scripts. There is not only "open" and "locked" with Plutus. There is more than just these 2 discret states and there are hundreds of different ways to achieve immutability. It just really depends on the logic.
And even if something is "locked" why should you not be able to burn it. As holder you should have all the rights to do that. There should be no way to mint a second NFT, but burning should be entirely possible.

Thank you @michaelpj for the discussion,

This is exactly the hole that I'm pointing to. You have no way of knowing whether a minting policy was written in any particular way (without another metadata channel).  What is dangerous is to assume that everyone will follow a policy when it is not enforced.
More strongly, it's not the case that this CIP is "ineffective" if people don't follow the constraints, rather it creates a new attack vector against non-conforming users. I think actively reducing the security of innocent other network participants is pretty bad and needs to be taken seriously.
Like I said, I think the policy is complicated enough that it's fairly unlikely that someone would accidentally produce asset names that appear to follow it. But looking at it from the security angle suggests that it might be a cheap win to make the policy more obscure, so that we can argue that it is really extremely unlikely to be used by accident.

Correct me if I am wrong, the hole you are pointing to is the fact that developers can wrongly implement logic when writing smart contracts? E.g, unintentionally creating a FT but calling it an NFT with the (222) prefix (by wrongly implementing the forging conditions). I think this is a more general and harder problem, not related to the core of this specific CIP. How do we know that the code does what was intended? In regard to not knowing if a minting policy is correctly implemented, developers can publish their code, the hash of the validator is verifiable related to the policy ID. Trust is established by battle testing policies, common ways of minting certain asset classes will emerge. Lastly, Cardano is an open platform and standards should not be enforced, they should emerge fluidly. Implementing a standard wrongly may be a choice to opt out, though it may render the object worthless or create a new standard.

There seems to be some overlap with CIP-0066? although with different trade-offs. Would the authors (@Padierfind / @alessandrokonrad -- and the others, sorry I don't have the github handles) be open to discuss the two approaches as part of the next CIP bi-weekly meeting (16th of August) ?

There seems to be some overlap with CIP-0066? although with different trade-offs. Would the authors (@Padierfind / @alessandrokonrad -- and the others, sorry I don't have the github handles) be open to discuss the two approaches as part of the next CIP bi-weekly meeting (16th of August) ?

There aren't any overlaps actually. CIP-0066 tries to define a very specific standard around identity making use of the underlying structure of CIP-0025. CIP-0068 has per se nothing to do with identity and instead tries to define a generalized metadata standard for all types of assets. By extending CIP-0068 further someone could make use of identity.
Yes I could try to join, but I don't know 100% yet if I'm available there. @perturbing could also join probably.

Correct me if I am wrong, the hole you are pointing to is the fact that developers can wrongly implement logic when writing smart contracts? E.g, unintentionally creating a FT but calling it an NFT with the (222) prefix (by wrongly implementing the forging conditions).

No, this is not what I'm saying. The developers in my example (let's call it GameCoin) have done nothing wrong. They have used asset names for their own purposes, perfectly reasonably. However, we have chosen to interpret this as meaning something which they did not intend, which is our fault.
If a standard is not enforced in the rules of Cardano then it is necessarily opt-in. But we have no way of knowing whether people have opted in. The implicit strategy in the initial version of the proposal is "if it looks like they've opted in, assume that they have". My point is that this strategy has no guarantee of reliability, and in the cases where we assume wrongly we can degrade the security of other people's projects. Hence: we should think very carefully about this, call it out explicitly, and at the very least try to make the bad case as unlikely as possible.

@michaelpj thank you for the extra clarification, I see your point now. What I still do not see is how this will negatively affect other projects that do not opt in. This proposal does not change their immutable data on chain, only how it may be viewed by some. It changes nothing about the off-chain software stack. Besides, this naming only introduces the idea by 3rd parties that there may exist another token with a similar naming style, nothing more.
An option to solve your raised issue is to introduce a dedicated field that captures these kinds of tokens. This could be an extra bytestring or a split in the current TokenName space of 32 bytes. The latter sounds more logical to me, since the creation of an extra field besides the 32 bytes will open the door for bigger encodings in token names, which leads to misuse of the intended standard.

What I still do not see is how this will negatively affect other projects that do not opt in. This proposal does not change their immutable data on chain, only how it may be viewed by others.

Changing how they are viewed by others is exactly metadata spoofing and yes it's an attack. Of course, this is only a problem if this standard is widely adopted but presumably that's the idea :)

An option to solve your raised issue is to introduce a dedicated field that captures these kinds of tokens.

Changing Cardano itself is obviously a much bigger ask and has a pretty high bar. But metadata is a big enough problem that I think we could consider it if we had a good proposal.

Changing how they are viewed by others is exactly metadata spoofing and yes it's an attack. Of course, this is only a problem if this standard is widely adopted but presumably that's the idea :)

Thank you @michaelpj! I totally see the problem. Here's a solution I propose, which could also rescue the more flawed CIP-0025:
There needs to be a mechanism, where someone has to proactively enable metadata to make use of it for a certain minting policy, so innocent people are not put at risk and off-chain attack vectors are reduced to pretty much 0.
We mint an asset with a special asset name under the minting policy, where we want to enable metadata. This asset carries special metadata in the mint transaction which specify what metadata standard the entire minting policy is following.
So far this doesn't really solve the problem, but here is the trick:
In case the minting policy is a NativeScript:

It must be an any script with the second argument containing the word "metadata" as key hash.

{
  "type": "any",
  "scripts":
  [
    {
    ...
    },
    {
      "type": "sig",
      "keyHash": "metadata"
    },
  ]
}

In case the minting policy is a PlutusScript:

The special asset must have been minted with a very specific redeemer (e.g. index of plutus data contstructor is 1234)

So a third party has to do the following when it sees a specific asset A:

Lookup special asset name under policy id of A
Retrieve metadata from mint transaction to determine metadata standard for minting policy
Determine validity of metadata by looking for special key word "metadata" in script in case of native script policy or by checking the redeemer constructor index of the mint transaction of special asset in case of plutus script policy
Go back to asset A and retrieve metadata according to standard

This approach makes it very obscure and hard to follow metadata standards by accident and requires metadata users to be proactive and register something like a certificate in the first place.

There needs to be a mechanism, where someone has to proactively enable metadata to make use of it for a certain minting policy, so innocent people are not put at risk and off-chain attack vectors are reduced to pretty much 0.

To be clear, I don't think that we have to close this hole. If we think that it's sufficiently unlikely that the bad case will come up we can just accept it. That may well be better than forcing this proposal to include lots of extra complexity. I just want us to be very clear about what we're doing and why.
I guess I'm thinking that if we can make the token name format obscure enough that might be the best compromise. Adding a whole extra mechanism to close this one hole seems maybe not worth it, unless we can make it very simple.

This asset carries special metadata in the mint transaction which specify what metadata standard the entire minting policy is following.

I think it would be best if we can avoid this. One thing I really like about this proposal is that it avoids using transaction metadata. That's going to be a big deal in the post-Mithril world, IMO.

I guess I'm thinking that if we can make the token name format obscure enough that might be the best compromise. Adding a whole extra mechanism to close this one hole seems maybe not worth it, unless we can make it very simple.

I'm actually curious what you think is an obscure enough format. Simply replacing parentheses with colons? :222:, double colons? ::222:: or a completely different format, which may not even be UTF-8 encodable?

I'm really not sure. Something not UTF-8 encodable is pretty obscure indeed, but has the disadvantage of being harder to render. Perhaps using an unusual unicode grapheme as a bracketer?

I'm really not sure. Something not UTF-8 encodable is pretty obscure indeed, but has the disadvantage of being harder to render. Perhaps using an unusual unicode grapheme as a bracketer?

@perturbing had this idea of using the null character. So if we want to make use of the label 222:

in UTF-8: \x00222\x00
in HEX:<PHONE_NUMBER>

Maybe I am misunderstanding as I follow this, but I would recommend not choosing the null character.  null characters can cause major issues for NFT viewers and consuming applications (e.g., many SERDE libraries, JSON interpreters, etc. fail without easy debugging on null chars). This pattern is also generally is a bad marker for delimiting (since it is typically used to represent end of stream/file).
I might choose an obscure mathematical symbol like U+237B: https://www.compart.com/en/unicode/category/So
You could also do a pairing like U+2371/U+2372 so it is a clear enclosure rather than someone accidentally bookending their asset name with some UTF-8 character (I have done this with the ADA symbol, U+20B3: http://www.unicode-symbol.com/u/20B3.html)

I agree with @thaddeusdiamond that perhaps the null character is not a good idea. But we must keep in mind that we would like to use a Unicode character that only consumes 1 byte of our 32 bytes available in the token name.

For now, @alessandrokonrad and I found the characters «222» best suited. They are both 2 bytes, can be types with your keyboard, on Linux hold ctrl + shift and type U followed by the hex representation, 00AB («) and 00BB (»). On Mac, option + q («) and option + alt + q (»). This is obscure enough to not accidentally use it (like the brackets). Furthermore, they are nice in their rendering. What do you think?

Changing how they are viewed by others is exactly metadata spoofing and yes it's an attack. Of course, this is only a problem if this standard is widely adopted but presumably that's the idea :)

Thank you @michaelpj! I totally see the problem. Here's a solution I propose, which could also rescue the more flawed CIP-0025:
There needs to be a mechanism, where someone has to proactively enable metadata to make use of it for a certain minting policy, so innocent people are not put at risk and off-chain attack vectors are reduced to pretty much 0.
We mint an asset with a special asset name under the minting policy, where we want to enable metadata. This asset carries special metadata in the mint transaction which specify what metadata standard the entire minting policy is following. So far this doesn't really solve the problem, but here is the trick:
In case the minting policy is a NativeScript:

It must be an any script with the second argument containing the word "metadata" as key hash.

{
  "type": "any",
  "scripts":
  [
    {
    ...
    },
    {
      "type": "sig",
      "keyHash": "metadata"
    },
  ]
}

In case the minting policy is a PlutusScript:

The special asset must have been minted with a very specific redeemer (e.g. index of plutus data contstructor is 1234)

So a third party has to do the following when it sees a specific asset A:

Lookup special asset name under policy id of A
Retrieve metadata from mint transaction to determine metadata standard for minting policy
Determine validity of metadata by looking for special key word "metadata" in script in case of native script policy or by checking the redeemer constructor index of the mint transaction of special asset in case of plutus script policy
Go back to asset A and retrieve metadata according to standard

This approach makes it very obscure and hard to follow metadata standards by accident and requires metadata users to be proactive and register something like a certificate in the first place. Or differently said the minting policy must show someone willingly wanted to make use of metadata.

I really like this approach of allowing allowing a creator to choose which metadata standard they want to implement. This way innovative developers like @alessandrokonrad can have the programmability they need to build new types of applications via CIP-68, while your standard project can just use CIP-25 and avoid doubling their minting costs.
I also have a few questions:

The goal of this CIP seems to be to introduce NFT metadata into the Plutus validator context to allow for new applications. Is there no way to do this without requiring a reference NFT? Would it be possible to change this at the base layer instead?

How many blockfrost calls would it take a 3rd party to look up the associated metadata for an asset? From the screenshot below it appears we are going from 1 -> 3? This is something to keep in mind as it increases complexity & costs for developers.

Is there a limit to how much metadata you can store in a datum? I thought fields were limited to 64 chars.

@alessandrokonrad is it possible to get some examples as to what you're hoping to do with this standard?

I really like this approach of allowing allowing a creator to choose which metadata standard they want to implement. This way innovative developers like @alessandrokonrad can have the programmability they need to build new types of applications via CIP-68, while your standard project can just use CIP-25 and avoid doubling their minting costs.

We may not need this approach as the asset name labels seem to be enough already if we make them more obscure. So we don't have this extra complexity. Asset name labes also make it possible to choose the standard you want to use for your tokens and actually it's much easier.
Regarding costs, let's be very specific about this. Covering the min ada requirement for a token is not necessarily a cost, maybe temporarily, but if you can redeem the ada at a later point again you didn't lose them. The only real cost you have are network fees.

The goal of this CIP seems to be to introduce NFT metadata into the Plutus validator context to allow for new applications. Is there no way to do this without requiring a reference NFT? Would it be possible to change this at the base layer instead?

Without reference NFT there is no way to know the UTxO you query the metadata from is legit.
What exactly would you change about the base layer? I don't see how this should even be done or the necessity for it.

How many blockfrost calls would it take a 3rd party to look up the associated metadata for an asset? From the screenshot below it appears we are going from 1 -> 3? This is something to keep in mind as it increases complexity & costs for developers.

The only reason you have 1 call right now is because Blockfrost is following CIP-0025. There is also a multi step procedure behind it. If Blockfrost adopts CIP-0068 it will be just 1 call as well. So I don't think that's a good measure and comparison.

Is there a limit to how much metadata you can store in a datum? I thought fields were limited to 64 chars.

There is no limit except overall transaction size limit. The datum itself represents the metadata, there is not a single bytestring holding the entire metadata in a serialized form.

@alessandrokonrad is it possible to get some examples as to what you're hoping to do with this standard?

Trustless token minting
Trustless token bridges
Have an on-chain voting process only with a subset of an NFT collection. The only way to make this possible is if the Plutus validator has access to the metadata.

Something to note about this standard is that it's very generalized and universally applicable to pretty much any token type (e.g. FTs, NFTs, semi fungible, etc.). If you have something very generalized it makes it also easier to compose things.
CIP-0025 is very limited in that respect and only works under very specific constraints and only for certain token types, which is far from ideal.

Just worth mentioning that some of what's described here is also achievable in a completely different way with CIP-54 - although these are not competing standards, I believe they are complimentary. CIP-54 requires that the token is an HTML page with embedded Javascript, but if you're willing to do that, then you can monitor a script datum and change the rendering of the NFT based on that - this happens at the .
There are trade-offs with either approach - one benefit of the CIP-54 approach is that you can still "evolve" NFTs after the policy is fully locked (assuming you designed them to allow this in the first place). As as been mentioned, there are plenty within the community who do not consider an NFT to be an NFT unless the policy is fully locked. One disadvantage is that you cannot change the original NFT's metadata, you can only change how it renders in the browser.
I would suggest that it might be an idea to think of a new name to identify tokens complying with CIP-68, to differentiate them from CIP-54 "Smart NFTs" and just normal CIP-25 NFTs.
Either way, CIP-25 needs to continue to be supported as a base level standard for all NFTs - adding these more advanced features, whether it be CIP-54 or CIP-68, is something that's not necessary or suitable for many NFT authors who just want to publish some jpgs, so I believe CIP-54 and CIP-68 NFTs will always be "special" in that they have additional functionality beyond this basic "standard NFT" functionality.

you can monitor a script datum and change the rendering of the NFT based on that - this happens at the rendering layer and doesn't change the underlying token.

I believe the point of this proposal is to actually store the metadata on the blockchain. Essentially just storing an exterrnal link and then changing what's at the link is another approach which you can take. They have different advantages and disadvantages.

Either way, CIP-25 needs to continue to be supported as a base level standard for all NFTs - adding these more advanced features

CIP-25 (and hence CIP-54) is insecure and requires traversing the entire history of the chain, which may become increasingly undesirable in a post-Mithril world. I hope it will be deprecated in favour of CIP-68, even for the case where you just want to store an external link.

Either way, CIP-25 needs to continue to be supported as a base level standard for all NFTs - adding these more advanced features

CIP-25 (and hence CIP-54) is insecure and requires traversing the entire history of the chain, which may become increasingly undesirable in a post-Mithril world. I hope it will be deprecated in favour of CIP-68, even for the case where you just want to store an external link.

I am not convinced that replacing something simple that everyone already understands, with something more complicated, and more costly, for little appreciable benefit, is going to be something that everyone would buy into. I certainly wouldn't.
I think I understand your point about scanning the blockchain - you mean, to find the original mint transaction? If that's really going to be a big overhead, I suggest it should be fixed in the protocol by maintaining a reference to the minting transaction.
We're going to need to continue to support rendering CIP-25 NFTs long into the future, even if they are depreciated, given that there are already 5m+ of them minted ;) I really think depreciating CIP-25 is an absolute non-starter.

These assets and their metadata do not disappear with this proposed CIP. Projects can deterministically convert their CIP-25 assets to the above standard using a Merkle tree. What I mean by this, one can create a Merkle tree data structure of their collection and their associated metadata. Then they could make a plutus minting policy has the Merkle root hard-coded to lock the old asset and mint another pre-determined collection that verifiable correspond to the old collection. Note that these assets will have a new policy ID, but this ID can be traced back to the original one. Here is a merkle tree library for plutus (1)
Lastly, this proposal is here to better the experience of all assets, there is something fundamentally wrong with CIP-25. This does not mean it was wrong at the time when it was first introduced. Back then, we did not have plutus or reference inputs, technology evolves, and so should standards.

I believe you have just proved my point. I run an NFT marketplace and none of that makes any sense to me at all. I don't see any demand at all from people to convert their NFT to this new format, it seems more like it's designed to get Spacebudz out of a hole.
I would support this standard if it was an optional addition on CIP- 25 but I will not support it as a replacement to CIP-25. I'm not sure why you think CIP-25 is wrong, clearly it's working fine for most people as-is, otherwise there wouldn't be 5m assets minted.

If that's really going to be a big overhead, I suggest it should be fixed in the protocol by maintaining a reference to the minting transaction.

There is no such thing as "the" minting transaction for fungible tokens.
We're not going to jam something ad-hoc into the base layer. But if we can come up with a small, well-scoped improvement that helps the metadata situation, I think we'd be open to it.

I'm not sure why you think CIP-25 is wrong

It's not secure, it has a much worse version of the metadata spoofing risk that this proposal has: https://github.com/cardano-foundation/CIPs/pull/85#issuecomment-1054123645
It requires processing the history of the chain. This is only going to get more painful over time as the chain gets bigger, and when we get Mithril and it's possible to run a node without having the full chain, it'll be a huge penalty to require the history just to look for metadata.

I really like this approach of allowing allowing a creator to choose which metadata standard they want to implement. This way innovative developers like @alessandrokonrad can have the programmability they need to build new types of applications via CIP-68, while your standard project can just use CIP-25 and avoid doubling their minting costs.

We may not need this approach as the asset name labels seem to be enough already if we make them more obscure. So we don't have this extra complexity. Asset name labes also make it possible to choose the standard you want to use for your tokens and actually it's much easier.

Ohh I see! This is where CIP-67 comes into play? Clever 💡

Regarding costs, let's be very specific about this. Covering the min ada requirement for a token is not necessarily a cost, maybe temporarily, but if you can redeem the ada at a later point again you didn't lose them. The only real cost you have are network fees.

I'm not convinced about this. A user minting an NFT will need to lock up 2 * min ADA, one for the token and one for the reference NFT, how can they get either of those back?
If a user minting a 20 ADA NFT gets a signature screen requesting 22-23 ADA, they're going to get the impression that transaction fees on Cardano are high. Perhaps we could just charge 20 ADA and take min ada away from the creator's profits, but depending on your mint price that could be 1-5% just for an additional min ada UTXO.

The goal of this CIP seems to be to introduce NFT metadata into the Plutus validator context to allow for new applications. Is there no way to do this without requiring a reference NFT? Would it be possible to change this at the base layer instead?

Without reference NFT there is no way to know the UTxO you query the metadata from is legit. What exactly would you change about the base layer? I don't see how this should even be done or the necessity for it.

Was just curious if there were any other implementations you considered besides this approach! 🙇

How many blockfrost calls would it take a 3rd party to look up the associated metadata for an asset? From the screenshot below it appears we are going from 1 -> 3? This is something to keep in mind as it increases complexity & costs for developers.

The only reason you have 1 call right now is because Blockfrost is following CIP-0025. There is also a multi step procedure behind it. If Blockfrost adopts CIP-0068 it will be just 1 call as well. So I don't think that's a good measure and comparison.

Fair enough.

Is there a limit to how much metadata you can store in a datum? I thought fields were limited to 64 chars.

There is no limit except overall transaction size limit. The datum itself represents the metadata, there is not a single bytestring holding the entire metadata in a serialized form.

Thank you for answer my noob question 😛

It's not secure, it has a much worse version of the metadata spoofing risk that this proposal has: CIP-
NFTMetdataStandard #85 (comment)

I'm not sure what you mean by "secure", but I consider it perfectly secure.

It requires processing the history of the chain. This is only going to get more painful over time as the chain gets bigger, and when we get Mithril and it's possible to run a node without having the full chain, it'll be a huge penalty to require the history just to look for metadata.

Not really. There are many pain points in Cardano, but scanning the chain for metadata isn't one of them. To do anything on Cardano you need to run a full chain indexer, so we've all got SQL databases, and those are indexed such that it's not an expensive operation to scan for a particular token. We do this all the time, it is not anywhere near being a pain point.
While I admit that it would be nice to have the NFT metadata following the NFT so that it's actually attached to the token rather than the mint transaction, I just feel like there's a lot of overcomplication in this CIP, to achieve something that, to me at least, is a marginal benefit at best.
With regard to fungible tokens - I don't have a problem with just referring to the most recent mint transaction. It's not as "clean" as I would like, but it's definitely not broken.
I will also point out that minting policy scripts have been able to do some quite advanced things, since inception, and yet everyone mints on the basic time-locked policy - they're just looking for the simplest method to get their mint done and in most cases not interested in taking advantage of the advanced features that policy scripts can offer. I logically extend this to the assumption that 99% of the NFT projects that are being minted have no interest in the functionality being offered here, and for them it's just an additional layer of confusion and cost, for dubious benefits.
I will say that this exact same argument applies to the CIP I wrote - CIP-54, which is why I haven't specified that CIP-25 should be depreciated.
Regarding the min-UTXO amount - this is already a big enough pain point for people on Cardano. I have repeatedly had to explain this to my non-technical friends, as it makes absolutely no sense to them when they're used to traditional banking which doesn't have these weird minimum amounts. They get very annoyed when having NFTs in their wallet means they can't withdraw the entire ADA balance - they're used to being able to run their bank balance to zero. So just another argument in favour of not locking more of people's funds than we absolutely have to!

With regard to fungible tokens - I don't have a problem with just referring to the most recent mint transaction. It's not as "clean" as I would like, but it's definitely not broken.

Well, that should have been specified in CIP-25, which was written over a year ago. It's far too late to be discussing this. If I get one meta-level point home here, it's that we really need to discuss security in CIPs and take into account the full generality of the system.
Your proposal to take the last transaction is the only sane way to patch CIP-25, but it's still broken as I'll show below.

I'm not sure what you mean by "secure", but I consider it perfectly secure.

Let me give another example since it seems like the point isn't getting home. Consider the Djed stablecoin. A normal interaction with the Djed contract mints additional stablecoins when the user puts Ada in, this is not a privileged action. Hence, according to (patched) CIP-25, the last person to mint a Djed stablecoin can set the metadata for all Djed stablecoins. The contract cannot prevent this, since scripts can't see metadata.
CIP-25 works only for token schemes where tokens are minted only by the organizer of the scheme. But that is not always the case and you cannot assume that it is. The problem with CIP-25 is not that it only works for organizer-issued tokens, but that it  assumes that all tokens behave like that, and this makes other tokens vulnerable to metadata spoofing.

I will also point out that minting policy scripts have been able to do some quite advanced things, since inception, and yet everyone mints on the basic time-locked policy - they're just looking for the simplest method to get their mint done and in most cases not interested in taking advantage of the advanced features that policy scripts can offer. I logically extend this to the assumption that 99% of the NFT projects that are being minted have no interest in the functionality being offered here, and for them it's just an additional layer of confusion and cost, for dubious benefits.

It's nice that the tokens which you're aware of or care about are all doing relatively simple things with tokens. I don't think that's a good reason to adopt proposals which open attack vectors against other people using the platform in legitimate ways (like, say, Djed).
Again: "99% of usage follows pattern X" is a good reason to have a proposal that only works for pattern X, but it's not a good reason to have a proposal that makes not-X attackable.

Not really. There are many pain points in Cardano, but scanning the chain for metadata isn't one of them. To do anything on Cardano you need to run a full chain indexer, so we've all got SQL databases, and those are indexed such that it's not an expensive operation to scan for a particular token. We do this all the time, it is not anywhere near being a pain point.

This is a very short sighted view IMO. The chain will only get bigger, and requiring the whole chain is a significant barrier to usage, especially if you care about low-spec devices. And if there are multiple reasons X, Y and Z that force us to do some undesirable action A, that's not a reason to not try to fix X.

CIP-25 works only for token schemes where tokens are minted only by the organizer of the scheme. But that is not always the case and you cannot assume that it is. The problem with CIP-25 is not that it only works for organizer-issued tokens, but that it assumes that all tokens behave like that, and this makes other tokens vulnerable to metadata spoofing.

Okay well I have an easy fix for that - CIP25 only applies to NFTs, as it was always intended. Problem solved.

This is a very short sighted view IMO. The chain will only get bigger, and requiring the whole chain is a significant barrier to usage, especially if you care about low-spec devices. And if there are multiple reasons X, Y and Z that force us to do some undesirable action A, that's not a reason to not try to fix X.

Like I say, I would like the token metadata to follow the token rather than being stuck in the mint transaction, but I don't think this convoluted method of minting two tokens and locking one in a script address is the best way to achieve that. I think it would be better to just have the node attach this information directly to the right object in the first place (attach the metadata to the token not the transaction).

Okay well I have an easy fix for that - CIP25 only applies to NFTs, as it was always intended. Problem solved.

How do you know if a token is a NFT? That's... metadata. You can't make CIP-25 opt-in, because whether or not you've opted in is... metadata.

Okay well I have an easy fix for that - CIP25 only applies to NFTs, as it was always intended. Problem solved.

How do you know if a token is a NFT? That's... metadata. You can't make CIP-25 opt-in, because whether or not you've opted in is... metadata.

Well that just returns me to an earlier point - even if you depreciate CIP-25, you will still have to support it given the huge number of tokens that have already been minted, so this CIP doesn't fix that issue.
I tell whether a token is an FT or NFT by counting the live instances of it, this also requires scanning the chain. Again, returning me to the point that you must run a full chain indexer to do anything useful with Cardano.
As I've already said, I would prefer this information to follow the token around, since it is really token metadata not transaction metadata, but I think that is a different CIP. I don't like this hacky approach to achieve something which I think could and should be done much cleaner in the node itself.

@kieransimkin, again thank you for raising your concerns, I hope my answers and that of others help understand the global view on the problem, it is not an easy one.

Well that just returns me to an earlier point - even if you depreciate CIP-25, you will still have to support it given the huge number of tokens that have already been minted, so this CIP doesn't fix that issue.

I thought about this too, since the minting policies of these tokens are already closed (as they should be), we cannot just mint a similar one under the same policy ID following this CIP. What we can do is create a bridge between the two standards, here old NFT's can be locked and new NFT's can be verifiably minted. This is an involved minting script that is not made yet, but I think it is possible with the use of Merkle tree's (something that all future projects might want to consider). These allow a user to construct zero knowledge proofs that can allow them to permissionless mint a certain token with pre-determined metadata. Now the bridge only checks that a certain token is locked and checks that the new token is minted with the correct metadata (it has to be the same as in the proof).

I don't like this hacky approach to achieve something which I think could and should be done much cleaner in the node itself.

I will shed some light on how the current idea came about and abstract a bit away of the details. We want to make data available at the edge of the UTxO graph, these are the active parts of the ledger which hold and propagate the state in chunks which we call unspent transaction outputs.
Now, in general, we want a choice on how we handle data. Sometimes we want the data to be persistent and available on the active part of the ledger, and every so often we want data to only be recorder by the ledger, I think you agree on that.
How the ledger currently is, implements this choice, for the former you attach data in the datum, for the latter you use metadata in a transaction. Of course, these ways of storing and handling are priced different, as they imply different things. There are advantages and trade of between both. But these emerge not from this CIP, only from what the ledger gives us at the moment.

I tell whether a token is an FT or NFT by counting the live instances of it, this also requires scanning the chain.

This doesn't work because just because there is currently only one, doesn't mean there will always be only one, i.e. this cannot distinguish NFTs and FTs of which only one has currently been minted.
Also, I should have said, this is a distraction because the key thing is not FT vs NFT, but organizer-minted versus user-minted. You can have NFTs that are minted by users, and those would also be vulnerable to metadata spoofing. And organizer-minted versus user-minted really isn't something you can see in the base layer at all.

When I talk about deprecating CIP-25 I mean something like: deprecate the CIP, try and persuade people not to mint new tokens using it, etc. Not "immediately drop support from wallets" or something. I think if we ever do get metadata spoofing attacks based on CIP-25 there will be quite a bit of pressure to do something like that.

Some (AI-generated) summary of the conversations above (I invite the authors / commenters to reach out if they disagree with the summary, but from having also read through the conversation, I find it relatively accurate).

The main conversation is about a proposal for a new standard that would allow for metadata to be securely attached to NFTs. The standard would require that the NFTs be locked in a UTXO, which would then be referenced by a user token. The metadata would be stored in a datum object within the UTXO. There are some concerns about the cost of locking the NFTs in a UTXO, as well as the potential for ledger bloat. There is also some discussion about whether or not this standard would be preferable to the existing CIP-25 metadata standard.
A conversation between @Michaelpj and @alessandrokonrad: @Michaelpj is pointing out that this could be used to attack innocent third-party token issuers, who might not be aware of the standard or might not have implemented it correctly. @Alessandrokonrad suggests a solution where there would be a mechanism to enable metadata for a minting policy, so that innocent people are not put at risk. @Michaelpj suggests that making the token name format obscure enough might be the best compromise.
Another conversation is between @cent-development, @alessandrokonrad, and @newmindflow. They are discussing the implications of the standard proposal for making metadata available for smart contract validators and the possibility of using a single reference NFT for all FT user tokens. @newmindflow brings up the point that there could be a problem with the long-term growth of NFTs if the minimum ADA required to be sent/locked with a token is not reduced. @alessandrokonrad suggests that a burning mechanism could be used to redeem the ADA. @newmindflow agrees that this could work if the burning mechanism is executed before the NFTs are locked.
A final conversation is about the trade-offs between two different ways of storing data for NFTs on the Cardano blockchain. One way is to store the data on the blockchain itself (CIP-68), and the other way is to store a link to the data off-chain (CIP-54). The advantages of CIP-68 are that it is more secure and doesn't require scanning the entire history of the chain. The disadvantages are that it is more complicated and expensive, and that it is not compatible with the existing 5 million NFTs minted under CIP-25.

Adding my own grain of salt to this: it occurs to me that one source of disagreement being the "depreciation of CIP-0025", which is maybe something that has been slightly extrapolated from the proposal? I don't think CIP-0068 comes to deprecate CIP-0068, but it certainly positions itself as more flexible and more secure alternative way to associate metadata to assets on-chain, albeit introducing of course its sheer amount of complexity to achieve it. Whether or not you want to embrace this standard or prefer to keep using CIP-0025 is a different story. If the plan is to deprecate CIP-0025 and mark it as obsolete, then I'd suggest to have this conversation in due time and on the PR proposing that change. This conversation is otherwise orthogonal to this proposal.

This doesn't work because just because there is currently only one, doesn't mean there will always be only one, i.e. this cannot distinguish NFTs and FTs of which only one has currently been minted.

Well it's non-trivial to determine if a policy is locked, and I generally only care if the token is an NFT at the present time, I cannot and do not try to predict the future.

When I talk about deprecating CIP-25 I mean something like: deprecate the CIP, try and persuade people not to mint new tokens using it, etc. Not "immediately drop support from wallets" or something. I think if we ever do get metadata spoofing attacks based on CIP-25 there will be quite a bit of pressure to do something like that.

If the user mints the token, they should be able to put what they like in the metadata. If you want to control what's in the mint, you should generate the transaction server-side and sign it with the policy key, then pass it to the user to sign with their key. If people are minting tokens entirely in the client side, without requiring something non-client side to sign with the policy key, then they should expect that the user can change the token metadata. This is not a security flaw in my opinion, it's perhaps a caveat worthy of mention, but it's not inherently insecure.
I still don't think there is a security issue here, as long as people using the standard understand it, which is a lot easier with CIP25 than it is with this standard - CIP25 requires much less fundamental blockchain knowledge than this new standard - CIP25 is written in terms that are understandable to a non-blockchain developer, so I think it's safer than expecting people to mint via a regime which, to my mind at least, is more complicated. Complexity in itself can be a vulnerability if it means people are using the standard without fully understanding the implications of it.

I still don't think there is a security issue here, as long as people using the standard understand it

I believe you're missing the point about the mentioned security flaws. This is something we mentioned in the last CIP editor meeting and suggested @alessandrokonrad to perhaps emphasize a bit more in the motivation section about the flaws inherently affecting CIP-0025 and standards built upon it.
The issue is not for people using it. It's for those who don't. If you are unaware that this CIP exists, and wants to mint token in a different way, possibly providing metadata using a different mechanism, then you've open up your protocol to metadata spoofing since platforms will today, by default, interpret anything from a minting transaction as legitimate metadata. If you do differently, you're likely screwed. Such standard is therefore unsound unless everyone is using it. Yet, since it can't be enforced at the protocol-level, there's no such guarantee. While CIPs have grown popular, there are still many people out there -- especially new joiners -- who do not know about them and may launch projects at scale irrespective of what CIP-0025 is specifying.
CIP-0068 on the contrary is much harder to spoof, because the conditions to be recognized as a CIP-0068 metadata are far less "accidental" and can probably be made even less likely.

@kieransimkin: I still don't think there is a security issue here, as long as people using the standard understand it

I believe you're missing the point about the mentioned security flaws. This is something we mentioned in the last CIP editor meeting and suggested @alessandrokonrad to perhaps emphasize a bit more in the motivation section: the flaws inherently affecting CIP-0025 and standards built upon it.
The issue is not for people using it. It's for those who don't. If you are unaware that this CIP exists, and wants to mint token in a different way, possibly providing metadata using a different mechanism, then you've open up your protocol to metadata spoofing since platforms will today, by default, interpret anything from a minting transaction as legitimate metadata. If you do differently, you're likely screwed. Such standard is therefore unsound unless everyone is using it. Yet, since it can't be enforced at the protocol-level, there's no such guarantee. While CIPs have grown popular, there are still many people out there -- especially new joiners -- who do not know about them and may launch projects at scale irrespective of what CIP-0025 is specifying.
CIP-0068 on the contrary is much harder to spoof, because the conditions to be recognized as a CIP-0068 metadata are far less "accidental" and can probably be made even less likely.

I wanna add to this that CIP-0025 overshadows every new metadata standard by default if not really adopted. But that shouldn't be the case, the worst case should be not displaying anything by 3rd parties, but there is not guarantee for that.

While this is an interesting proposal, the concerns raised by some are important considerations.  It is also not clear to me why it is important to design a solution around the need to have mutable/programable metadata on-chain.  It feels like we are trying to replicate other blockchains, rather than trying to innovate in the Cardano way.
I realise that there will be scenarios where information related to an NFT needs to be mutated, such as gaming data. But there are ways of doing this (using the blockchain for proofs) without needing to update the NFT itself.  With NFTs in Ethereum existing as smart contracts, it is their simplistic way of achieving that objective, but Cardano isn't bound by smart contracts for tokens, so we don't need to just copy their approach.
I have been trying to promote this broader way of thinking since the topic of metadata was first raised, including taking the opportunity to think beyond Cardano (i.e. an inter chain approach). My suggestion therefore is a compromise, whereby the token CAN be minted with immutable metadata (like a CIP-0025 compliant time locked NFT) to address the concern raised by @kieransimkin, but the token MAY also be referenced via a DID (if the creator wants to extend its capabilities), the URI of which is constructed from the asset ID, e.g. did:cnft: (addressing the naming concerns raised by @michaelpj with the "reference NFT" approach). If the NFT creator wants to lock a "reference NFT" in a smart contract (as per the suggestion in this proposal), then it can be referenced as a linked resource in the DID.  This would only put the additional cost burden (as highlighted by @newmindflow and others) on tokens that need this option.
This would help address the issues raised in this CIP, by allowing Services in the DID to define serviceEndpoints for additional metadata (programmable or otherwise, on-chain or off-chain).  It would also provide a mechanism to address the verification challenge posed in CIP-0066, by using the Verifiable Credentials standard that already works with DIDs.  It could also move our NFTs towards a cross chain interoperability standard (others are exploring this approach), if that is important to the NFT creator.  By using a DID as a common interface, alternate standards for different token types (see the Token Taxonomy Framework) can be introduced as required.
I would be interested in people's thoughts on this, as I may consider submitting my own CIP with this approach for consideration.

@phillewis I think that taking the policy ID + asset name and using it to look up metadata in an external system (whether it be DID-based or otherwise) is fairly reasonable, that was the basis of CIP-26, for example. You still need to prevent credentials being registered by the wrong person, but I think that's soluble, and is something that you can iterate on off-chain without having to change the base mechanism. So it might well be viable.

@phillewis I think that taking the policy ID + asset name and using it to look up metadata in an external system (whether it be DID-based or otherwise) is fairly reasonable, that was the basis of CIP-26, for example. You still need to prevent credentials being registered by the wrong person, but I think that's soluble, and is something that you can iterate on off-chain without having to change the base mechanism. So it might well be viable.

Thanks for the feedback @michaelpj. If there are others interested in exploring this approach, would you be interested in connecting to help flesh it out further?

@kieransimkin: I still don't think there is a security issue here, as long as people using the standard understand it

I believe you're missing the point about the mentioned security flaws. This is something we mentioned in the last CIP editor meeting and suggested @alessandrokonrad to perhaps emphasize a bit more in the motivation section: the flaws inherently affecting CIP-0025 and standards built upon it.

Either I'm still missing the point, or I just don't agree with it. The minting transaction is the authoritative source of the metadata in CIP-25. If people are not following CIP-25, their NFTs are not valid and all bets are off. Minting an NFT requires a signature from the policy key, so how is that insecure? It is correct to interpret anything in the minting transaction as the correct metadata, because the minting transaction requires a signature by the policy. If you have the private key of the policy, you have the right to put whatever you like in the metadata, this is not a security issue?

Minting an NFT requires a signature from the policy key, so how is that insecure?

That's not always the case. You could have complex plutus minting policies where there isn't an owner behind it, but just some conditions need to be met in order to mint. Imagine a DEX where you mint liquidity tokens. The process is autonomous and decentralized. What stops someone from injecting metadata according to CIP-0025 into the mint transaction.

Minting an NFT requires a signature from the policy key, so how is that insecure?

That's not always the case. You could have complex plutus minting policies where there isn't an owner behind it, but just some conditions need to be met in order to mint. Imagine a DEX where you mint liquidity tokens. The process is autonomous and decentralized. What stops someone from injecting metadata according to CIP-0025 into the mint transaction.

Ah okay thanks for the explanation - I get it now. I am still stuck in the mindset that all NFTs will be minted with the same locked policy, because that is still the only policy that is documented at all on the official site.
My counter to this, is that if you're savvy enough to be writing your own minting policy scripts, you're going to understand the implications of doing so (obviously better than I do).
This just puts me back to my original point though - if we're aiming to completely replace CIP-25 - I would much prefer a cleaner solution whereby the token metadata becomes a first class primitive in the blockchain itself, rather than just stuffing it in some ad-hoc metadata/datum field, as this feels like a hack to me, in much the same way CIP-25 feels like a hack. I don't really wanna replace one hacky way of storing the metadata with another one that that actually feels worse, and more confusing for people (thus potentially more prone to errors).

My counter to this, is that if you're savvy enough to be writing your own minting policy scripts, you're going to understand the implications of doing so (obviously better than I do).

We could use the same DEX example here. Regardless of how skilled the devs are, why would they even care about metadata in the first place for liquidity tokens and so they may not even be aware of CIP-0025.
The other thing is Plutus validators do not have access to the auxiliary data, which makes it impossible for people who are aware of the standard to create decentralized minting policies without being victim of spoofing attacks.

This just puts me back to my original point though - if we're aiming to completely replace CIP-25 - I would much prefer a cleaner solution whereby the token metadata becomes a first class primitive in the blockchain itself, rather than just stuffing it in some ad-hoc metadata/datum field, as this feels like a hack to me, in much the same way CIP-25 feels like a hack. I don't really wanna replace one hacky way of storing the metadata with another one that that actually feels worse, and more confusing for people (thus potentially more prone to errors).

CIP-0068 gives you a lot of flexibility and is imo not hacky. You separate control over the NFT/token itself and metadata, which allows yout to create far more sophisticated and interesting token standards, but of course still allows you to do the same as with CIP-0025. The datum is meant to store information inside, you could then also reference these data and bring them into plutus validators. And since these data are part of the ledger state they could be useful for Mithril, which allows you to query and fully verify the metadata as light client.

I would much prefer a cleaner solution whereby the token metadata becomes a first class primitive in the blockchain itself

This CIP defines just that, a universal place where metadata associated with any native asset can reside onchain (in a standardized way). It does not fill in the blanks on how and in what format this data should be formatted (we suggest two standards for FT/NFT's but more are possible). This CIP addresses the abstraction on how data of any token is linked to its value. And a good thing to remember here is that metadata should be strictly separated from its representation of value, this because this value is moved a lot. Were the metadata attached to the native asset that is the transferable value (or the UTxO that has this token), this data would fill up valuable transaction space. With reference inputs, you can separate this data and value while still preserving the option of having the data available in a transaction. That is the core of this proposal. Again, it does not fill in the blanks, how you manage the metadata is up to the asset label standard used.

I would propose to make the 222 NFT Standard name property in files_details optional, and actually to also make it optional in CIP 25 (which would be backward compatible).
In practice, the name is often not used and omitted by creators, which shows that it's often not needed.

The CIP meeting just finished has noted that this CIP is considered ready for acceptance, pending only the resolution of the questions still being discussed in related CIP-0067 (https://github.com/cardano-foundation/CIPs/pull/298).

(cc: @KtorZ) @alessandrokonrad @perturbing the consensus at today's CIP meeting # 54 is that this proposal is ready to be merged, if you will please:

add the number 68 (no leading zeroes) in the CIP header
change status to Proposed (it can move ahead while implementations are still being worked on)
add a clear Path to Active section about the next steps for acceptance & use by the community

We'll be reviewing the Path to active in the next session. As a reminder, this should emphasis on demonstrating adoption of the CIP, which includes working implementations and adoptions across various actors of the ecosystem through observable metrics.


[AddToken] HoG on GNOSIS_CHAIN

Network
GNOSIS_CHAIN
Token Symbol
HoG
Token Name
Humans of Gnosis
Decimals
18
Address
0x6de572faa138048ce8142c4a206eb09a8ec39e45
Image URL
https://gnosis.mypinata.cloud/ipfs/QmWBcV2GweaRizjDDezwSLjSLjEfZBLZHxRqQe8WPfTmB7
Reason
The idea behind Circles is that people can create and use their own money. We are now adding a very important piece to to it: groups/communities. In addition to the pure p2p layer of Circles we are adding “group currencies”. As a first experiment we are launching the “Humans of Gnosis” group. As a big milestone we want to allow anyone to buy a DappCon ticket this year with “HoG” (Human of Gnosis) tokens.
Some people won't be able to mint enough HoG thus we would like them to purchase them via Cowswap.
We will provide liquidity for the token once we open the ticket sale.

Submitted on behalf of @MareenG
Original request https://github.com/cowprotocol/token-lists/issues/193