Generation of oneOf definitions when title is set

📝 Context
This PR addresses an issue encountered while generating Go code from contract schemas derived from our contracts definitions. The problem arises when executing go-codegen on our axone-objectarium.json schemas. Specifically, all definition types from ExecuteMsg and QueryMsg are not being generated.
 `axone-objectarium.json` schema

{
"contract_name": "axone-objectarium",
"contract_version": "5.0.0",
"idl_version": "1.0.0",
"instantiate": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "InstantiateMsg",
"description": "Instantiate messages",
"type": "object",
"required": [
"bucket"
],
"properties": {
"bucket": {
"description": "The name of the bucket. The name could not be empty or contains whitespaces. If name contains whitespace, they will be removed.",
"type": "string"
},
"config": {
"description": "The configuration of the bucket.",
"default": {
"accepted_compression_algorithms": [
"passthrough",
"snappy",
"lzma"
],
"hash_algorithm": "sha256"
},
"allOf": [
{
"$ref": "#/definitions/BucketConfig"
}
]
},
"limits": {
"description": "The limits of the bucket.",
"default": {
"max_object_pins": null,
"max_object_size": null,
"max_objects": null,
"max_total_size": null
},
"allOf": [
{
"$ref": "#/definitions/BucketLimits"
}
]
},
"pagination": {
"description": "The configuration for paginated query.",
"default": {
"default_page_size": 10,
"max_page_size": 30
},
"allOf": [
{
"$ref": "#/definitions/PaginationConfig"
}
]
}
},
"additionalProperties": false,
"definitions": {
"BucketConfig": {
"description": "BucketConfig is the type of the configuration of a bucket.\n\nThe configuration is set at the instantiation of the bucket, and is immutable and cannot be changed. The configuration is optional and if not set, the default configuration is used.",
"type": "object",
"properties": {
"accepted_compression_algorithms": {
"description": "The acceptable compression algorithms for the objects in the bucket. If this parameter is not set, then all compression algorithms are accepted. If this parameter is set, then only the compression algorithms in the array are accepted.\n\nWhen an object is stored in the bucket without a specified compression algorithm, the first algorithm in the array is used. Therefore, the order of the algorithms in the array is significant. Typically, the most efficient compression algorithm, such as the NoCompression algorithm, should be placed first in the array.\n\nAny attempt to store an object using a different compression algorithm than the ones specified here will fail.",
"default": [
"passthrough",
"snappy",
"lzma"
],
"type": "array",
"items": {
"$ref": "#/definitions/CompressionAlgorithm"
}
},
"hash_algorithm": {
"description": "The algorithm used to hash the content of the objects to generate the id of the objects. The algorithm is optional and if not set, the default algorithm is used.\n\nThe default algorithm is Sha256 if not set.",
"default": "sha256",
"allOf": [
{
"$ref": "#/definitions/HashAlgorithm"
}
]
}
},
"additionalProperties": false
},
"BucketLimits": {
"description": "BucketLimits is the type of the limits of a bucket.\n\nThe limits are optional and if not set, there is no limit.",
"type": "object",
"properties": {
"max_object_pins": {
"description": "The maximum number of pins in the bucket for an object.",
"anyOf": [
{
"$ref": "#/definitions/Uint128"
},
{
"type": "null"
}
]
},
"max_object_size": {
"description": "The maximum size of the objects in the bucket.",
"anyOf": [
{
"$ref": "#/definitions/Uint128"
},
{
"type": "null"
}
]
},
"max_objects": {
"description": "The maximum number of objects in the bucket.",
"anyOf": [
{
"$ref": "#/definitions/Uint128"
},
{
"type": "null"
}
]
},
"max_total_size": {
"description": "The maximum total size of the objects in the bucket.",
"anyOf": [
{
"$ref": "#/definitions/Uint128"
},
{
"type": "null"
}
]
}
},
"additionalProperties": false
},
"CompressionAlgorithm": {
"description": "CompressionAlgorithm is an enumeration that defines the different compression algorithms supported for compressing the content of objects. The compression algorithm specified here are relevant algorithms for compressing data on-chain, which means that they are fast to compress and decompress, and have a low computational cost.\n\nThe order of the compression algorithms is based on their estimated computational cost (quite opinionated) during both compression and decompression, ranging from the lowest to the highest. This particular order is utilized to establish the default compression algorithm for storing an object.",
"oneOf": [
{
"title": "Passthrough",
"description": "Represents no compression algorithm. The object is stored as is without any compression.",
"type": "string",
"enum": [
"passthrough"
]
},
{
"title": "Snappy",
"description": "Represents the Snappy algorithm. Snappy is a compression/decompression algorithm that does not aim for maximum compression. Instead, it aims for very high speeds and reasonable compression.\n\nSee the snappy web page for more information.",
"type": "string",
"enum": [
"snappy"
]
},
{
"title": "Lzma",
"description": "Represents the LZMA algorithm. LZMA is a lossless data compression/decompression algorithm that features a high compression ratio and a variable compression-dictionary size up to 4 GB.\n\nSee the LZMA wiki page for more information.",
"type": "string",
"enum": [
"lzma"
]
}
]
},
"HashAlgorithm": {
"description": "HashAlgorithm is an enumeration that defines the different hash algorithms supported for hashing the content of objects.",
"oneOf": [
{
"title": "MD5",
"description": "Represents the MD5 algorithm. MD5 is a widely used cryptographic hash function that produces a 128-bit hash value. The computational cost of MD5 is relatively low compared to other hash functions, but its short hash length makes it easier to find hash collisions. It is now considered insecure for cryptographic purposes, but can still used in non-security contexts.\n\nMD5 hashes are stored on-chain as 32 hexadecimal characters.\n\nSee the MD5 Wikipedia page for more information.",
"type": "string",
"enum": [
"m_d5"
]
},
{
"title": "SHA1",
"description": "Represents the SHA-224 algorithm. SHA-224 is a variant of the SHA-2 family of hash functions that produces a 224-bit hash value. It is similar to SHA-256, but with a shorter output size. The computational cost of SHA-224 is moderate, and its relatively short hash length makes it easier to store and transmit.\n\nSHA-224 hashes are stored on-chain as 56 hexadecimal characters.\n\nSee the SHA-2 Wikipedia page for more information.",
"type": "string",
"enum": [
"sha224"
]
},
{
"title": "SHA256",
"description": "Represents the SHA-256 algorithm. SHA-256 is a member of the SHA-2 family of hash functions that produces a 256-bit hash value. It is widely used in cryptography and other security-related applications. The computational cost of SHA-256 is moderate, and its hash length strikes a good balance between security and convenience.\n\nSHA-256 hashes are stored on-chain as 64 hexadecimal characters.\n\nSee the SHA-2 Wikipedia page for more information.",
"type": "string",
"enum": [
"sha256"
]
},
{
"title": "SHA384",
"description": "Represents the SHA-384 algorithm. SHA-384 is a variant of the SHA-2 family of hash functions that produces a 384-bit hash value. It is similar to SHA-512, but with a shorter output size. The computational cost of SHA-384 is relatively high, but its longer hash length provides better security against hash collisions.\n\nSHA-384 hashes are stored on-chain as 96 hexadecimal characters.\n\nSee the SHA-2 Wikipedia page for more information.",
"type": "string",
"enum": [
"sha384"
]
},
{
"title": "SHA512",
"description": "Represents the SHA-512 algorithm. SHA-512 is a member of the SHA-2 family of hash functions that produces a 512-bit hash value. It is widely used in cryptography and other security-related applications. The computational cost of SHA-512 is relatively high, but its longer hash length provides better security against hash collisions.\n\nSHA-512 hashes are stored on-chain as 128 hexadecimal characters.\n\nSee the SHA-2 Wikipedia page for more information.",
"type": "string",
"enum": [
"sha512"
]
}
]
},
"PaginationConfig": {
"description": "PaginationConfig is the type carrying configuration for paginated queries.\n\nThe fields are optional and if not set, there is a default configuration.",
"type": "object",
"properties": {
"default_page_size": {
"description": "The default number of elements in a page.\n\nShall be less or equal than max_page_size. Default to '10' if not set.",
"default": 10,
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"max_page_size": {
"description": "The maximum elements a page can contain.\n\nShall be less than u32::MAX - 1. Default to '30' if not set.",
"default": 30,
"type": "integer",
"format": "uint32",
"minimum": 0.0
}
},
"additionalProperties": false
},
"Uint128": {
"description": "A thin wrapper around u128 that is using strings for JSON encoding/decoding, such that the full u128 range can be used for clients that convert JSON numbers to floats, like JavaScript and jq.\n\n# Examples\n\nUse from to create instances of this and u128 to get the value out:\n\n# use cosmwasm_std::Uint128; let a = Uint128::from(123u128); assert_eq!(a.u128(), 123);\n\nlet b = Uint128::from(42u64); assert_eq!(b.u128(), 42);\n\nlet c = Uint128::from(70u32); assert_eq!(c.u128(), 70);",
"type": "string"
}
}
},
"execute": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "ExecuteMsg",
"description": "Execute messages",
"oneOf": [
{
"title": "StoreObject",
"description": "StoreObject store an object to the bucket and make the sender the owner of the object. The object is referenced by the hash of its content and this value is returned. If the object is already stored, it is a no-op. It may be pinned though.\n\nThe "pin" parameter specifies if the object should be pinned for the sender. In such case, the object cannot be removed (forget) from the storage.\n\nThe "compression_algorithm" parameter specifies the algorithm for compressing the object before storing it in the storage, which is optional. If no algorithm is specified, the algorithm used is the first algorithm of the bucket configuration limits. Note that the chosen algorithm can save storage space, but it will increase CPU usage. Depending on the chosen compression algorithm and the achieved compression ratio, the gas cost of the operation will vary, either increasing or decreasing.",
"type": "object",
"required": [
"store_object"
],
"properties": {
"store_object": {
"type": "object",
"required": [
"data",
"pin"
],
"properties": {
"compression_algorithm": {
"description": "Specifies the compression algorithm to use when storing the object. If None, the first algorithm specified in the list of accepted compression algorithms of the bucket is used (see [BucketLimits::accepted_compression_algorithms]).",
"anyOf": [
{
"$ref": "#/definitions/CompressionAlgorithm"
},
{
"type": "null"
}
]
},
"data": {
"description": "The content of the object to store.",
"allOf": [
{
"$ref": "#/definitions/Binary"
}
]
},
"pin": {
"description": "Specifies if the object should be pinned for the sender.",
"type": "boolean"
}
},
"additionalProperties": false
}
},
"additionalProperties": false
},
{
"title": "ForgetObject",
"description": "ForgetObject first unpin the object from the bucket for the considered sender, then remove it from the storage if it is not pinned anymore. If the object is pinned for other senders, it is not removed from the storage and an error is returned. If the object is not pinned for the sender, this is a no-op.",
"type": "object",
"required": [
"forget_object"
],
"properties": {
"forget_object": {
"type": "object",
"required": [
"id"
],
"properties": {
"id": {
"type": "string"
}
},
"additionalProperties": false
}
},
"additionalProperties": false
},
{
"title": "PinObject",
"description": "PinObject pins the object in the bucket for the considered sender. If the object is already pinned for the sender, this is a no-op. While an object is pinned, it cannot be removed from the storage.",
"type": "object",
"required": [
"pin_object"
],
"properties": {
"pin_object": {
"type": "object",
"required": [
"id"
],
"properties": {
"id": {
"type": "string"
}
},
"additionalProperties": false
}
},
"additionalProperties": false
},
{
"title": "UnpinObject",
"description": "UnpinObject unpins the object in the bucket for the considered sender. If the object is not pinned for the sender, this is a no-op. The object can be removed from the storage if it is not pinned anymore.",
"type": "object",
"required": [
"unpin_object"
],
"properties": {
"unpin_object": {
"type": "object",
"required": [
"id"
],
"properties": {
"id": {
"type": "string"
}
},
"additionalProperties": false
}
},
"additionalProperties": false
}
],
"definitions": {
"Binary": {
"description": "Binary is a wrapper around Vec to add base64 de/serialization with serde. It also adds some helper methods to help encode inline.\n\nThis is only needed as serde-json-{core,wasm} has a horrible encoding for Vec. See also https://github.com/CosmWasm/cosmwasm/blob/main/docs/MESSAGE_TYPES.md.",
"type": "string"
},
"CompressionAlgorithm": {
"description": "CompressionAlgorithm is an enumeration that defines the different compression algorithms supported for compressing the content of objects. The compression algorithm specified here are relevant algorithms for compressing data on-chain, which means that they are fast to compress and decompress, and have a low computational cost.\n\nThe order of the compression algorithms is based on their estimated computational cost (quite opinionated) during both compression and decompression, ranging from the lowest to the highest. This particular order is utilized to establish the default compression algorithm for storing an object.",
"oneOf": [
{
"title": "Passthrough",
"description": "Represents no compression algorithm. The object is stored as is without any compression.",
"type": "string",
"enum": [
"passthrough"
]
},
{
"title": "Snappy",
"description": "Represents the Snappy algorithm. Snappy is a compression/decompression algorithm that does not aim for maximum compression. Instead, it aims for very high speeds and reasonable compression.\n\nSee the snappy web page for more information.",
"type": "string",
"enum": [
"snappy"
]
},
{
"title": "Lzma",
"description": "Represents the LZMA algorithm. LZMA is a lossless data compression/decompression algorithm that features a high compression ratio and a variable compression-dictionary size up to 4 GB.\n\nSee the LZMA wiki page for more information.",
"type": "string",
"enum": [
"lzma"
]
}
]
}
}
},
"query": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "QueryMsg",
"description": "Query messages",
"oneOf": [
{
"title": "Bucket",
"description": "Bucket returns the bucket information.",
"type": "object",
"required": [
"bucket"
],
"properties": {
"bucket": {
"type": "object",
"additionalProperties": false
}
},
"additionalProperties": false
},
{
"title": "Object",
"description": "Object returns the object information with the given id.",
"type": "object",
"required": [
"object"
],
"properties": {
"object": {
"type": "object",
"required": [
"id"
],
"properties": {
"id": {
"description": "The id of the object to get.",
"type": "string"
}
},
"additionalProperties": false
}
},
"additionalProperties": false
},
{
"title": "Objects",
"description": "Objects returns the list of objects in the bucket with support for pagination.",
"type": "object",
"required": [
"objects"
],
"properties": {
"objects": {
"type": "object",
"properties": {
"address": {
"description": "The owner of the objects to get.",
"type": [
"string",
"null"
]
},
"after": {
"description": "The point in the sequence to start returning objects.",
"type": [
"string",
"null"
]
},
"first": {
"description": "The number of objects to return.",
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
}
},
"additionalProperties": false
}
},
"additionalProperties": false
},
{
"title": "ObjectData",
"description": "ObjectData returns the content of the object with the given id.",
"type": "object",
"required": [
"object_data"
],
"properties": {
"object_data": {
"type": "object",
"required": [
"id"
],
"properties": {
"id": {
"description": "The id of the object to get.",
"type": "string"
}
},
"additionalProperties": false
}
},
"additionalProperties": false
},
{
"title": "ObjectPins",
"description": "ObjectPins returns the list of addresses that pinned the object with the given id with support for pagination.",
"type": "object",
"required": [
"object_pins"
],
"properties": {
"object_pins": {
"type": "object",
"required": [
"id"
],
"properties": {
"after": {
"description": "The point in the sequence to start returning pins.",
"type": [
"string",
"null"
]
},
"first": {
"description": "The number of pins to return.",
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
},
"id": {
"description": "The id of the object to get the pins for.",
"type": "string"
}
},
"additionalProperties": false
}
},
"additionalProperties": false
}
]
},
"migrate": null,
"sudo": null,
"responses": {
"bucket": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "BucketResponse",
"description": "BucketResponse is the response of the Bucket query.",
"type": "object",
"required": [
"config",
"limits",
"name",
"pagination",
"stat"
],
"properties": {
"config": {
"description": "The configuration of the bucket.",
"allOf": [
{
"$ref": "#/definitions/BucketConfig"
}
]
},
"limits": {
"description": "The limits of the bucket.",
"allOf": [
{
"$ref": "#/definitions/BucketLimits"
}
]
},
"name": {
"description": "The name of the bucket.",
"type": "string"
},
"pagination": {
"description": "The configuration for paginated query.",
"allOf": [
{
"$ref": "#/definitions/PaginationConfig"
}
]
},
"stat": {
"description": "The statistics of the bucket.",
"allOf": [
{
"$ref": "#/definitions/BucketStat"
}
]
}
},
"additionalProperties": false,
"definitions": {
"BucketConfig": {
"description": "BucketConfig is the type of the configuration of a bucket.\n\nThe configuration is set at the instantiation of the bucket, and is immutable and cannot be changed. The configuration is optional and if not set, the default configuration is used.",
"type": "object",
"properties": {
"accepted_compression_algorithms": {
"description": "The acceptable compression algorithms for the objects in the bucket. If this parameter is not set, then all compression algorithms are accepted. If this parameter is set, then only the compression algorithms in the array are accepted.\n\nWhen an object is stored in the bucket without a specified compression algorithm, the first algorithm in the array is used. Therefore, the order of the algorithms in the array is significant. Typically, the most efficient compression algorithm, such as the NoCompression algorithm, should be placed first in the array.\n\nAny attempt to store an object using a different compression algorithm than the ones specified here will fail.",
"default": [
"passthrough",
"snappy",
"lzma"
],
"type": "array",
"items": {
"$ref": "#/definitions/CompressionAlgorithm"
}
},
"hash_algorithm": {
"description": "The algorithm used to hash the content of the objects to generate the id of the objects. The algorithm is optional and if not set, the default algorithm is used.\n\nThe default algorithm is Sha256 if not set.",
"default": "sha256",
"allOf": [
{
"$ref": "#/definitions/HashAlgorithm"
}
]
}
},
"additionalProperties": false
},
"BucketLimits": {
"description": "BucketLimits is the type of the limits of a bucket.\n\nThe limits are optional and if not set, there is no limit.",
"type": "object",
"properties": {
"max_object_pins": {
"description": "The maximum number of pins in the bucket for an object.",
"anyOf": [
{
"$ref": "#/definitions/Uint128"
},
{
"type": "null"
}
]
},
"max_object_size": {
"description": "The maximum size of the objects in the bucket.",
"anyOf": [
{
"$ref": "#/definitions/Uint128"
},
{
"type": "null"
}
]
},
"max_objects": {
"description": "The maximum number of objects in the bucket.",
"anyOf": [
{
"$ref": "#/definitions/Uint128"
},
{
"type": "null"
}
]
},
"max_total_size": {
"description": "The maximum total size of the objects in the bucket.",
"anyOf": [
{
"$ref": "#/definitions/Uint128"
},
{
"type": "null"
}
]
}
},
"additionalProperties": false
},
"BucketStat": {
"title": "BucketStat",
"description": "BucketStat is the type of the statistics of a bucket.",
"type": "object",
"required": [
"compressed_size",
"object_count",
"size"
],
"properties": {
"compressed_size": {
"description": "The total size of the objects contained in the bucket after compression.",
"allOf": [
{
"$ref": "#/definitions/Uint128"
}
]
},
"object_count": {
"description": "The number of objects in the bucket.",
"allOf": [
{
"$ref": "#/definitions/Uint128"
}
]
},
"size": {
"description": "The total size of the objects contained in the bucket.",
"allOf": [
{
"$ref": "#/definitions/Uint128"
}
]
}
},
"additionalProperties": false
},
"CompressionAlgorithm": {
"description": "CompressionAlgorithm is an enumeration that defines the different compression algorithms supported for compressing the content of objects. The compression algorithm specified here are relevant algorithms for compressing data on-chain, which means that they are fast to compress and decompress, and have a low computational cost.\n\nThe order of the compression algorithms is based on their estimated computational cost (quite opinionated) during both compression and decompression, ranging from the lowest to the highest. This particular order is utilized to establish the default compression algorithm for storing an object.",
"oneOf": [
{
"title": "Passthrough",
"description": "Represents no compression algorithm. The object is stored as is without any compression.",
"type": "string",
"enum": [
"passthrough"
]
},
{
"title": "Snappy",
"description": "Represents the Snappy algorithm. Snappy is a compression/decompression algorithm that does not aim for maximum compression. Instead, it aims for very high speeds and reasonable compression.\n\nSee the snappy web page for more information.",
"type": "string",
"enum": [
"snappy"
]
},
{
"title": "Lzma",
"description": "Represents the LZMA algorithm. LZMA is a lossless data compression/decompression algorithm that features a high compression ratio and a variable compression-dictionary size up to 4 GB.\n\nSee the LZMA wiki page for more information.",
"type": "string",
"enum": [
"lzma"
]
}
]
},
"HashAlgorithm": {
"description": "HashAlgorithm is an enumeration that defines the different hash algorithms supported for hashing the content of objects.",
"oneOf": [
{
"title": "MD5",
"description": "Represents the MD5 algorithm. MD5 is a widely used cryptographic hash function that produces a 128-bit hash value. The computational cost of MD5 is relatively low compared to other hash functions, but its short hash length makes it easier to find hash collisions. It is now considered insecure for cryptographic purposes, but can still used in non-security contexts.\n\nMD5 hashes are stored on-chain as 32 hexadecimal characters.\n\nSee the MD5 Wikipedia page for more information.",
"type": "string",
"enum": [
"m_d5"
]
},
{
"title": "SHA1",
"description": "Represents the SHA-224 algorithm. SHA-224 is a variant of the SHA-2 family of hash functions that produces a 224-bit hash value. It is similar to SHA-256, but with a shorter output size. The computational cost of SHA-224 is moderate, and its relatively short hash length makes it easier to store and transmit.\n\nSHA-224 hashes are stored on-chain as 56 hexadecimal characters.\n\nSee the SHA-2 Wikipedia page for more information.",
"type": "string",
"enum": [
"sha224"
]
},
{
"title": "SHA256",
"description": "Represents the SHA-256 algorithm. SHA-256 is a member of the SHA-2 family of hash functions that produces a 256-bit hash value. It is widely used in cryptography and other security-related applications. The computational cost of SHA-256 is moderate, and its hash length strikes a good balance between security and convenience.\n\nSHA-256 hashes are stored on-chain as 64 hexadecimal characters.\n\nSee the SHA-2 Wikipedia page for more information.",
"type": "string",
"enum": [
"sha256"
]
},
{
"title": "SHA384",
"description": "Represents the SHA-384 algorithm. SHA-384 is a variant of the SHA-2 family of hash functions that produces a 384-bit hash value. It is similar to SHA-512, but with a shorter output size. The computational cost of SHA-384 is relatively high, but its longer hash length provides better security against hash collisions.\n\nSHA-384 hashes are stored on-chain as 96 hexadecimal characters.\n\nSee the SHA-2 Wikipedia page for more information.",
"type": "string",
"enum": [
"sha384"
]
},
{
"title": "SHA512",
"description": "Represents the SHA-512 algorithm. SHA-512 is a member of the SHA-2 family of hash functions that produces a 512-bit hash value. It is widely used in cryptography and other security-related applications. The computational cost of SHA-512 is relatively high, but its longer hash length provides better security against hash collisions.\n\nSHA-512 hashes are stored on-chain as 128 hexadecimal characters.\n\nSee the SHA-2 Wikipedia page for more information.",
"type": "string",
"enum": [
"sha512"
]
}
]
},
"PaginationConfig": {
"description": "PaginationConfig is the type carrying configuration for paginated queries.\n\nThe fields are optional and if not set, there is a default configuration.",
"type": "object",
"properties": {
"default_page_size": {
"description": "The default number of elements in a page.\n\nShall be less or equal than max_page_size. Default to '10' if not set.",
"default": 10,
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"max_page_size": {
"description": "The maximum elements a page can contain.\n\nShall be less than u32::MAX - 1. Default to '30' if not set.",
"default": 30,
"type": "integer",
"format": "uint32",
"minimum": 0.0
}
},
"additionalProperties": false
},
"Uint128": {
"description": "A thin wrapper around u128 that is using strings for JSON encoding/decoding, such that the full u128 range can be used for clients that convert JSON numbers to floats, like JavaScript and jq.\n\n# Examples\n\nUse from to create instances of this and u128 to get the value out:\n\n# use cosmwasm_std::Uint128; let a = Uint128::from(123u128); assert_eq!(a.u128(), 123);\n\nlet b = Uint128::from(42u64); assert_eq!(b.u128(), 42);\n\nlet c = Uint128::from(70u32); assert_eq!(c.u128(), 70);",
"type": "string"
}
}
},
"object": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "ObjectResponse",
"description": "ObjectResponse is the response of the Object query.",
"type": "object",
"required": [
"compressed_size",
"compression_algorithm",
"id",
"is_pinned",
"owner",
"size"
],
"properties": {
"compressed_size": {
"description": "The size of the object when compressed. If the object is not compressed, the value is the same as size.",
"allOf": [
{
"$ref": "#/definitions/Uint128"
}
]
},
"compression_algorithm": {
"description": "The compression algorithm used to compress the content of the object.",
"allOf": [
{
"$ref": "#/definitions/CompressionAlgorithm"
}
]
},
"id": {
"description": "The id of the object.",
"type": "string"
},
"is_pinned": {
"description": "Tells if the object is pinned by at least one address.",
"type": "boolean"
},
"owner": {
"description": "The owner of the object.",
"type": "string"
},
"size": {
"description": "The size of the object.",
"allOf": [
{
"$ref": "#/definitions/Uint128"
}
]
}
},
"additionalProperties": false,
"definitions": {
"CompressionAlgorithm": {
"description": "CompressionAlgorithm is an enumeration that defines the different compression algorithms supported for compressing the content of objects. The compression algorithm specified here are relevant algorithms for compressing data on-chain, which means that they are fast to compress and decompress, and have a low computational cost.\n\nThe order of the compression algorithms is based on their estimated computational cost (quite opinionated) during both compression and decompression, ranging from the lowest to the highest. This particular order is utilized to establish the default compression algorithm for storing an object.",
"oneOf": [
{
"title": "Passthrough",
"description": "Represents no compression algorithm. The object is stored as is without any compression.",
"type": "string",
"enum": [
"passthrough"
]
},
{
"title": "Snappy",
"description": "Represents the Snappy algorithm. Snappy is a compression/decompression algorithm that does not aim for maximum compression. Instead, it aims for very high speeds and reasonable compression.\n\nSee the snappy web page for more information.",
"type": "string",
"enum": [
"snappy"
]
},
{
"title": "Lzma",
"description": "Represents the LZMA algorithm. LZMA is a lossless data compression/decompression algorithm that features a high compression ratio and a variable compression-dictionary size up to 4 GB.\n\nSee the LZMA wiki page for more information.",
"type": "string",
"enum": [
"lzma"
]
}
]
},
"Uint128": {
"description": "A thin wrapper around u128 that is using strings for JSON encoding/decoding, such that the full u128 range can be used for clients that convert JSON numbers to floats, like JavaScript and jq.\n\n# Examples\n\nUse from to create instances of this and u128 to get the value out:\n\n# use cosmwasm_std::Uint128; let a = Uint128::from(123u128); assert_eq!(a.u128(), 123);\n\nlet b = Uint128::from(42u64); assert_eq!(b.u128(), 42);\n\nlet c = Uint128::from(70u32); assert_eq!(c.u128(), 70);",
"type": "string"
}
}
},
"object_data": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Binary",
"description": "Binary is a wrapper around Vec to add base64 de/serialization with serde. It also adds some helper methods to help encode inline.\n\nThis is only needed as serde-json-{core,wasm} has a horrible encoding for Vec. See also https://github.com/CosmWasm/cosmwasm/blob/main/docs/MESSAGE_TYPES.md.",
"type": "string"
},
"object_pins": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "ObjectPinsResponse",
"description": "ObjectPinsResponse is the response of the GetObjectPins query.",
"type": "object",
"required": [
"data",
"page_info"
],
"properties": {
"data": {
"description": "The list of addresses that pinned the object.",
"type": "array",
"items": {
"type": "string"
}
},
"page_info": {
"description": "The page information.",
"allOf": [
{
"$ref": "#/definitions/PageInfo"
}
]
}
},
"additionalProperties": false,
"definitions": {
"PageInfo": {
"title": "PageInfo",
"description": "PageInfo is the page information returned for paginated queries.",
"type": "object",
"required": [
"cursor",
"has_next_page"
],
"properties": {
"cursor": {
"description": "The cursor to the next page.",
"type": "string"
},
"has_next_page": {
"description": "Tells if there is a next page.",
"type": "boolean"
}
},
"additionalProperties": false
}
}
},
"objects": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "ObjectsResponse",
"description": "ObjectsResponse is the response of the Objects query.",
"type": "object",
"required": [
"data",
"page_info"
],
"properties": {
"data": {
"description": "The list of objects in the bucket.",
"type": "array",
"items": {
"$ref": "#/definitions/ObjectResponse"
}
},
"page_info": {
"description": "The page information.",
"allOf": [
{
"$ref": "#/definitions/PageInfo"
}
]
}
},
"additionalProperties": false,
"definitions": {
"CompressionAlgorithm": {
"description": "CompressionAlgorithm is an enumeration that defines the different compression algorithms supported for compressing the content of objects. The compression algorithm specified here are relevant algorithms for compressing data on-chain, which means that they are fast to compress and decompress, and have a low computational cost.\n\nThe order of the compression algorithms is based on their estimated computational cost (quite opinionated) during both compression and decompression, ranging from the lowest to the highest. This particular order is utilized to establish the default compression algorithm for storing an object.",
"oneOf": [
{
"title": "Passthrough",
"description": "Represents no compression algorithm. The object is stored as is without any compression.",
"type": "string",
"enum": [
"passthrough"
]
},
{
"title": "Snappy",
"description": "Represents the Snappy algorithm. Snappy is a compression/decompression algorithm that does not aim for maximum compression. Instead, it aims for very high speeds and reasonable compression.\n\nSee the snappy web page for more information.",
"type": "string",
"enum": [
"snappy"
]
},
{
"title": "Lzma",
"description": "Represents the LZMA algorithm. LZMA is a lossless data compression/decompression algorithm that features a high compression ratio and a variable compression-dictionary size up to 4 GB.\n\nSee the LZMA wiki page for more information.",
"type": "string",
"enum": [
"lzma"
]
}
]
},
"ObjectResponse": {
"title": "ObjectResponse",
"description": "ObjectResponse is the response of the Object query.",
"type": "object",
"required": [
"compressed_size",
"compression_algorithm",
"id",
"is_pinned",
"owner",
"size"
],
"properties": {
"compressed_size": {
"description": "The size of the object when compressed. If the object is not compressed, the value is the same as size.",
"allOf": [
{
"$ref": "#/definitions/Uint128"
}
]
},
"compression_algorithm": {
"description": "The compression algorithm used to compress the content of the object.",
"allOf": [
{
"$ref": "#/definitions/CompressionAlgorithm"
}
]
},
"id": {
"description": "The id of the object.",
"type": "string"
},
"is_pinned": {
"description": "Tells if the object is pinned by at least one address.",
"type": "boolean"
},
"owner": {
"description": "The owner of the object.",
"type": "string"
},
"size": {
"description": "The size of the object.",
"allOf": [
{
"$ref": "#/definitions/Uint128"
}
]
}
},
"additionalProperties": false
},
"PageInfo": {
"title": "PageInfo",
"description": "PageInfo is the page information returned for paginated queries.",
"type": "object",
"required": [
"cursor",
"has_next_page"
],
"properties": {
"cursor": {
"description": "The cursor to the next page.",
"type": "string"
},
"has_next_page": {
"description": "Tells if there is a next page.",
"type": "boolean"
}
},
"additionalProperties": false
},
"Uint128": {
"description": "A thin wrapper around u128 that is using strings for JSON encoding/decoding, such that the full u128 range can be used for clients that convert JSON numbers to floats, like JavaScript and jq.\n\n# Examples\n\nUse from to create instances of this and u128 to get the value out:\n\n# use cosmwasm_std::Uint128; let a = Uint128::from(123u128); assert_eq!(a.u128(), 123);\n\nlet b = Uint128::from(42u64); assert_eq!(b.u128(), 42);\n\nlet c = Uint128::from(70u32); assert_eq!(c.u128(), 70);",
"type": "string"
}
}
}
},
"description": "# Objectarium\n\nA  CosmWasm Smart Contract which enables the storage of arbitrary unstructured Objects in any Cosmos blockchains.\n\n## Purpose\n\nThe smart contract serves as a robust storage solution, allowing for the storage of arbitrary objects on any blockchain within the Cosmos blockchains network, utilizing the CosmWasm framework. The key features of the contract include:\n\nVersatile Data Storage:\nThe contract is designed to accommodate any type of data, be it text, images, or complex data structures. This flexibility makes it an ideal choice for a wide range of decentralized applications (dApps) that require diverse storage needs.\n\nOn-chain Data:\nBy design, the contract stores data on the blockchain, ensuring that it is immutable and publicly accessible. This is particularly useful for applications that require a high level of transparency, and also for any other smart contract that needs to store data on the blockchain.\n\nPinning and Unpinning:\nOne unique feature is the ability to 'pin' and 'unpin' objects associated with a specific sender address. Pinning ensures that the object remains stored and accessible, while unpinning releases it from being permanently stored, offering a level of control over data persistence.\n\nObject Removal:\nThe contract also includes a 'forget' function, allowing for the removal of objects that are no longer pinned. This is particularly useful for managing storage costs and ensuring that only relevant data remains on the blockchain.\n\nCost Management:\nFeatures like pinning, unpinning, and discarding objects offer a strategic way to control storage costs. Additionally, setting limits on contract size — for instance in terms of object count and their individual sizes — serves as a practical tool to regulate storage costs.\n\n## Rationale\n\nIn a sense, we can consider blockchains built on the Cosmos L0 layer as decentralized databases, and their nature can be shaped and modeled through the smart contracts or modules. Given this, it provides a great opportunity to address the wide range of data management needs. One such important area is the management of unstructured, immutable data, which is written once but accessed frequently — commonly known as object storage. This is the primary focus of axone-objectarium: a specialized smart contract designed to offer a versatile and efficient approach to handling on-chain, unstructured, immutable data in a decentralized manner.\n\n## Terminology\n\n### Object\n\nIn the context of the axone-objectarium smart contract, an object refers to a piece of data stored on the blockchain. It can represent various types of information, such as documents, binary files, or any other digital content. Objects are immutable once stored and are identified by their cryptographic hash, which can be generated using algorithms like MD5 or SHA256. This ensures the integrity and security of the stored data, as any modification to the object would result in a different hash value.\n\n### Bucket\n\nThe smart contract is organized around buckets. A bucket represents a logical container within the axone-objectarium smart contract instance that groups related Objects together. It acts as a storage unit for Objects and provides a context for managing and organizing them. Each bucket has a unique name and is associated with a set of configurations and limits that define its behaviour and characteristics.\n\n### Pin\n\nPin refers to a mechanism that allows users to mark or "pin" specific objects within a bucket. Pinning an object serves as a way to ensure that the object remains in storage and cannot be removed (this is called "forgotten"). It provides protection and guarantees that the pinned object will persist in the protocol. When an object is pinned, it is associated with the identity (or sender) that performed the pinning action.\n\n## Usage\n\nThe unstructured nature of the data stored in the chain opens up a plethora of possibilities for decentralized applications that require this type of versatile storage.\n\n### In the AXONE protocol\n\nThe primary function of this smart contract within the AXONE protocol is to enable the persistence of governance rules, which are encoded in Prolog. These programs are stored in an immutable format within the protocol and can be referenced by their unique identifiers in situations where there is a need to refer to these rules.\n\n### In the wild world\n\nA plethora of possibilities opens up for decentralized applications (dApps) that require this kind of versatile storage. However, it's important to consider the following constraints: the data is immutable, the cost of recording the data is proportional to its size, and the data is publicly accessible.\n\n## Play\n\n### Instantiation\n\nThe axone-objectarium can be instantiated as follows, refer to the schema for more information on configuration, limits and pagination configuration:\n\nbash\naxoned tx wasm instantiate $CODE_ID \\\n    --label \"my-storage\" \\\n    --from $ADDR \\\n    --admin $ADMIN_ADDR \\\n    --gas 1000000 \\\n    '{\"bucket\":\"my-bucket\"}'\n\n\n### Execution\n\nWe can store an object by providing its data in base64 encoded, we can pin the stored object to prevent it from being removed:\n\nbash\naxoned tx wasm execute $CONTRACT_ADDR \\\n    --from $ADDR \\\n    --gas 1000000 \\\n    \"{\\\"store_object\\\":{\\\"data\\\": \\\"$(cat my-data | base64)\\\",\\\"pin\\\":true}}\"\n\n\nThe object id is stable as it is a hash, we can't store an object twice.\n\nWith the following commands we can pin and unpin existing objects:\n\nbash\naxoned tx wasm execute $CONTRACT_ADDR \\\n    --from $ADDR \\\n    --gas 1000000 \\\n    \"{\\\"pin_object\\\":{\\\"id\\\": \\\"$OBJECT_ID\\\"}}\"\n\naxoned tx wasm execute $CONTRACT_ADDR \\\n    --from $ADDR \\\n    --gas 1000000 \\\n    \"{\\\"unpin_object\\\":{\\\"id\\\": \\\"$OBJECT_ID\\\"}}\"\n\n\nAnd if an object is not pinned, or pinned by the sender of transaction, we can remove it:\n\nbash\naxoned tx wasm execute $CONTRACT_ADDR \\\n    --from $ADDR \\\n    --gas 1000000 \\\n    \"{\\\"forget_object\\\":{\\\"id\\\": \\\"$OBJECT_ID\\\"}}\"\n\n\n### Querying\n\nQuery an object by its id:\n\nbash\naxoned query wasm contract-state smart $CONTRACT_ADDR \\\n    \"{\\\"object\\\": {\\\"id\\\": \\\"$OBJECT_ID\\\"}}\"\n\n\nOr its data:\n\nbash\naxoned query wasm contract-state smart $CONTRACT_ADDR \\\n    \"{\\\"object_data\\\": {\\\"id\\\": \\\"$OBJECT_ID\\\"}}\"\n\n\nWe can also list the objects, eventually filtering on the object owner:\n\nbash\naxoned query wasm contract-state smart $CONTRACT_ADDR \\\n    \"{\\\"objects\\\": {\\\"address\\\": \\\"axone1p8u47en82gmzfm259y6z93r9qe63l25d858vqu\\\"}}\"\n\n\nAnd navigate in a cursor based pagination:\n\nbash\naxoned query wasm contract-state smart $CONTRACT_ADDR \\\n    \"{\\\"objects\\\": {\\\"first\\\": 5, \\\"after\\\": \\\"23Y5t5DBe7DkPwfJo3Sd26Y8Z9epmtpA1FTpdG7DiG6MD8vPRTzzbQ9TccmyoBcePkPK6atUiqcAzJVo3TfYNBGY\\\"}}\"\n\n\nWe can also query object pins with the same cursor based pagination:\n\nbash\naxoned query wasm contract-state smart $CONTRACT_ADDR \\\n    \"{\\\"object_pins\\\": {\\\"id\\\": \\\"$OBJECT_ID\\\", \\\"first\\\": 5, \\\"after\\\": \\\"23Y5t5DBe7DkPwfJo3Sd26Y8Z9epmtpA1FTpdG7DiG6MD8vPRTzzbQ9TccmyoBcePkPK6atUiqcAzJVo3TfYNBGY\\\"}}\"\n",
"title": "axone-objectarium"
}

🔎 Example :
// Execute messages
type ExecuteMsg struct {
	/*
	   StoreObject store an object to the bucket and make the sender the owner of the object. The object is referenced by the hash of its content and this value is returned. If the object is already stored, it is a no-op. It may be pinned though.

	   The "pin" parameter specifies if the object should be pinned for the sender. In such case, the object cannot be removed (forget) from the storage.

	   The "compression_algorithm" parameter specifies the algorithm for compressing the object before storing it in the storage, which is optional. If no algorithm is specified, the algorithm used is the first algorithm of the bucket configuration limits. Note that the chosen algorithm can save storage space, but it will increase CPU usage. Depending on the chosen compression algorithm and the achieved compression ratio, the gas cost of the operation will vary, either increasing or decreasing.
	*/
	StoreObject *ExecuteMsg_StoreObject `json:"StoreObject,omitempty"`
	// ForgetObject first unpin the object from the bucket for the considered sender, then remove it from the storage if it is not pinned anymore. If the object is pinned for other senders, it is not removed from the storage and an error is returned. If the object is not pinned for the sender, this is a no-op.
	ForgetObject *ExecuteMsg_ForgetObject `json:"ForgetObject,omitempty"`
	// PinObject pins the object in the bucket for the considered sender. If the object is already pinned for the sender, this is a no-op. While an object is pinned, it cannot be removed from the storage.
	PinObject *ExecuteMsg_PinObject `json:"PinObject,omitempty"`
	// UnpinObject unpins the object in the bucket for the considered sender. If the object is not pinned for the sender, this is a no-op. The object can be removed from the storage if it is not pinned anymore.
	UnpinObject *ExecuteMsg_UnpinObject `json:"UnpinObject,omitempty"`
}

In the above ExecuteMsg example, the struct is generated successfully. However, the associated definitions such as ExecuteMsg_StoreObject are not generated at the end of the file. This issue also extends to other definitions in ExecuteMsg and QueryMsg.
🕵♀ Analysis
Upon investigation, it was found that our generated schema adds a title to each schema property related to ExecuteMsg enum and QueryMsg enum elements. This is intentional as our documentation is generated based on these titles. To address the issue, I made a minor modification to the code to prevent the skipping of property generation when a title is set. This change appears to resolve the issue effectively.
I would appreciate feedback on why the absence of a title results in the non-generation of property definitions. Additionally, I would like to understand if my modification could potentially introduce any unforeseen impacts.

Thank you for the contribution. I will tag a release after I close 1-2 more issues in a couple days.


using .set file

Hi, I have looked into the code but still find it difficult to use a .set file. I would like to use a .set file to set x, y, z coordinates. And I have tried:
XXX: Data selection
Data: conductance in 2e2/h
But nothing happens to the coordinates after I open qtplot with the data.
Any advice?

It seems the .set file is just used for a record of instrument settings, not for plot settings.


Support generating pages

Add an option to the talon.gen.resource to generate a page.

I believe this is a duplicate of #52?

I would be a dup if #52 had some description of done for the issue.

Fair point :)


versioning: sort by versions

Add a new sort option:

UI needs to show a new "Versions" sort option
Backend needs to support the version sort option as well.

For now just frontend until backend is done.


Feature - API to write toml (and with comments)

Would it be possible to have an API to write toml values back to a stream? I'm imagining a method such as Toml.writeTo(Writer) throws IOException
I noticed from the source code that the toString() of TomlTable and TomlArray looks like it writes toml, but of course, the output of toString() is not an API guarantee.
Also, on another note, would it be possible to add comments to toml values while writing them?

I've actually thought about this for some time.
There's a couple of issues that I'd need to resolve before I commit to a serialization API.
I don't differentiate between the different types of tables beyond the parser.
I'd either need to provide some way for the user to specify which "type of table" is desired.
I don't preserve key structure.
I mean, I obviously preserve the structure, but rather, I don't preserve the "pattern" that it used?
As in, someone could write:
a.b.c = "asd"

but without some token preserving it could be turned into:
[a.b]
c = "asd"

or something else bizarre.
The serialiser/formatter/writer would act something like a normaliser,
which isn't the end of the world, but that leads me onto the next issue.
Comments aren't preserved at all.
The parser isn't even aware of them, and by extension, neither are the data structures (TomlTable and TomlArray).
Adding support for comments would be a bit tricky, as I wouldn't want to pollute the data structures with Comment objects.
At least, I would like to avoid it.
So, in a nutshell, it would be possible, and as you rightfully noted, the toString() is already very similar.
But users would need to be made aware that, while it would be internally consistent, the style itself would be undefined.
And comments are a whole other ball game.

I see the problem. For my use-case, I only need to write valid toml, but not necessarily round-trip reading and writing with exact text preserved.
I don't imagine that the style of written toml being undefined would be a huge issue for API users. It's certainly better than not being able to write toml at all. Maybe it would make sense to implement toml writing, with an undefined style for now, and if desired at a later time, the exact style of toml could be preserved.
For comments, I'll go ahead and open a separate issue. I'm sure those would be more complicated.


Fix for #740.

This fixes #740.
Commits https://github.com/fonttools/fonttools/commit/3063def35b7ad733bf584b45452f9ef8f6efbab4 and https://github.com/fonttools/fonttools/commit/5b47971 introduced a separate fontDictOperators list for FontDict, only listing those TopDict key/value pairs that are actually used in the FontDict context. It provided a fallback that TTX files containing such "useless" key/value pairs would not be rejected.
However, the code still rejected binary fonts that contained such values, even though it didn't before, and yes, such fonts exist. Also: such fonts are not broken per spec, they just contain some fields that otherwise no one ever looks at, so it's a little harsh to reject them.
This patch removes most of the special FontDict code, and uses everything from TopDict, except the order attribute: it sets that to a list of the relevant keys for the FontDict. The effect of this is that "useless" key/value pairs are ignored, not just upon reading XML, but also upon decompilation and compilation of binary fonts. It improves on the previous XML reading behavior in that it no longer silently ignores key typos in the TTX input.
Ideally, we would output everything that is actually in the FontDict to TTX, and only ignore the values when compiling, but I didn't find a clean solution for that, so I decided to just fix the issue.

Cc @readroberts

@anthrotype, should I worry that those AppVeyor tests are failing? I'm afraid I can't make sense out of the error details :(

The two additional commits warn when keys are ignored:

upon compile, a log.log(logging.WARNING, "...") is issued. @anthrotype is that the preferred way of doing this?
upon exporting TTX a comment is added to the XML that warns about the ignored keys.

@justvanrossum @anthrotype the setuptools 29.0.0 wheel doesn’t install cli-32.exe anymore. I guess we should we stick with 28.8.0 untill setuptools 29 is fixed.
@justvanrossum log.warning("...")

Thanks Just for looking into this.
Don't worry about the appveyor build failing, it seems like a setuptools issue. I'll take care of that later.
log.warning(...) is a shortcut for that, but it's fine like that as well.

@moyogo beat me to it ;)

Thanks guys, I'll use log.warning(...) instead.

on the setuptools Appveyor issue:
https://github.com/pypa/setuptools/issues/861
Just, if you debase your branch on master, the build should now pass. Denis temporarily fixed that with https://github.com/fonttools/fonttools/commit/ea92dd75f1fe9cc2a274f687e831c8a989adb8e9

Ugh, getting caught in areas of git-land that I know nothing about. I did git rebase origin/master but now pushing gives an error.

git push -f

I've disabled force-push on master.  Feel free to enable it, do a forced push, and then disable again, as generally it's a bad idea.

It worked anyway, probably because I force-pushed to a non-master branch (the PR branch).

Thanks for dealing with this - I was out for several days. I certainly agree that existing fonts with non-essential FontDict keys should not be rejected. As a background, I made the original change because of complaints from Morisawa and FontWorks about non-essential keys appearing the the CID font FontDicts.

@readroberts, I would love it if you could review my total changes. I think I left all the intentions in tact and just made sure that valid fonts are not rejected. Warnings are issued for redundant keys.

@justvanrossum I would like to tag 3.3.0 with STAT table support. Is this PR ready to be merged?

It is ready, I was hoping for a review from @readroberts, but I think it's good to go. Thanks.

Thanks Just. We can always improve/fix things later.


Badges not working for Ansible Galaxy

Since a few weeks ago the ansible galaxy badges appear all as errored:

It seems to have been fixed. If you disagree, don't hesitate to reopen the issue.

Looks like it got resolved again. O_o


Add merge mode for update vocabularies

Now we have 2 update vocab options:
-update_vocab replace (ADD,REMAP,DELETE)
-update_vocab merge (ADD,REMAP)

Codecov Report

Merging #423 into master will increase coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #423      +/-   ##
==========================================
+ Coverage   69.53%   69.54%   +0.01%
==========================================
  Files          75       75
  Lines        6381     6384       +3
==========================================
+ Hits         4437     4440       +3
  Misses       1944     1944

Impacted Files
Coverage Δ

onmt/train/Trainer.lua
18.93% <100%> (+0.93%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2a1e320...b7f93db. Read the comment docs.


Train 233

#1994 Bump DC/OS Test Utils to the latest version to include a fix for _wait_for_dcos_history_data

@mesosphere-mergebot label Ready For Review

mergebot/enterprise/build-status/aggregate is not passing because of a known failure with teamcity/dcos/test/upgrade/disabled -> permissive. Manually adding a ship-it.


Reduce the information allocator need in createBlockMeta

In each createBlockMetaInternal, Space allocator will get a updated BlockMetadataManagerView which will do a full copy of pinList. Copy the pinList is slow when the list is huge (especially in async through) and copy is down in a synchronized lock which reduce the concurrency.  And actually, the allocator in the createBlockMetaInternal do not need the pin list info. It only needs the storage tier and directory information.
This PR is to reduce the information given to allocator and thus reduce the contention and improve performance.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/4330/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/4331/
Test FAILed.

@madanadit I added the StorageMetadataView, StorageTierView, and StorageDirView to only contains the information needed for allocator.
I renamed the original StorageTierView and StorageDirView to StorageTierEvictableView and StorageDirEvictableView to distinguish from the new ones.
In Allocator, there are many similar codes. I think of modifying the StorageTierEvictableView to extend StorageTierView, StorageDirEvictableView to extend StorageDirView to reduce the duplicate code, but then there will be many cast in our evictor. The evictor logics are much more complicated then allocator. I want to limit the impact of my changes as small as possible.
PTAL, thanks!

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/4335/
Test PASSed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/4343/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/4345/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/4346/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/4349/
Test FAILed.

Thanks a lot @LuQQiu! Looks good after the comment / variable name fixes. @ggezer PTAL

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/4350/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/4351/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/4352/
Test PASSed.

Mistakenly push one more commit, force push back to the commit that has been reviewed and approved

alluxio-bot, merge this please.

alluxio-bot, cherry-pick this to branch-2.0 please.


add vpn ip to prompt

move from master to next branch

@bhilburn What do you think of this? :)

@golgoth31 - Nice work! This is a really great PR, and a fantastic first contribution to the project. Welcome to P9k =)
@shibumi @onaforeignshore - Thanks for providing review & feedback in #538!

This prompt is Linux only at the moment and does not support macOS, which does not have ip. There is a way to get the info on macOS, as described in #538

@onaforeignshore - Ah, this needs to be fixed, then, or at the very least tracked. Just opened #540 for this. Thanks!


support node v6 in 1.3 branch

This is needed for TileMill, since TileMill does not yet use bones 2.x.
This commit (https://github.com/developmentseed/bones/commit/3e3a0b6fdd2720ab92cd0eb6b2d771ab69627c9b) was all that was needed since the latest express 2.x supports both node v4 and node v6.
Tests pass before and after this change with node v0.4.12 and after with node v.0.6.11.
Although we have a new warning with node v0.6.11:

(node) warning: possible EventEmitter memory leak detected. 11 listeners added. Use emitter.setMaxListeners() to increase limit.
Trace:
    at [object Object].<anonymous> (events.js:139:15)
    at [object Object].once (events.js:160:8)
    at [object Object].listen (net.js:775:10)
    at Function.response (/usr/local/lib/node_modules/expresso/bin/expresso:424:16)
    at Test.fn (/Users/dane/projects/tilemill/node_modules/bones/test/collections.test.js:15:12)
    at Test.runParallel (/usr/local/lib/node_modules/expresso/bin/expresso:959:10)
    at Test.run (/usr/local/lib/node_modules/expresso/bin/expresso:924:18)
    at next (/usr/local/lib/node_modules/expresso/bin/expresso:867:22)
    at runSuite (/usr/local/lib/node_modules/expresso/bin/expresso:875:6)
    at check (/usr/local/lib/node_modules/expresso/bin/expresso:814:12)

This looks good to go. I will confirm with @willwhite tomorrow -- then all that's needed is for this to be rolled into a tag?

this is now tagged in v1.3.24. closing.


Add RemoteCalc API to Science & Math

[X] My submission is formatted according to the guidelines in the contributing guide
[X] My addition is ordered alphabetically
[X] My submission has a useful description
[X] The description does not have more than 100 characters
[X] The description does not end with punctuation
[X] Each table column is padded with one space on either side
[X] I have searched the repository for any relevant issues or pull requests
[X] Any category I am creating has the minimum requirement of 3 items
[X] All changes have been squashed into a single commit

I just fixed it @matheusfelipeog thanks


All contributors/add pankajcoder1

Update readme file for previous pull request.
Please accept it.

Why did you delete the file?

That was not best appraoch so I deleted but i am looking for best approach for that algorithm.

Add this naive approach and add the efficient one also.

@Pankajcoder1 rebase to previous commits and do not tweak other files. So, I'm gonna close this one. Create a PR with your implementations.


bevy_pbr: Optimize assign_lights_to_clusters

Objective

Optimize assign_lights_to_clusters

Solution

Avoid inserting entities into hash sets in inner loops when it is known they will be inserted in at least one iteration of the loop.
Use a Vec instead of a hash set where the set is not needed
Avoid explicit calculation of the cluster_index from x,y,z coordinates, instead using row and column offsets and just adding z in the inner loop
These changes cut the time spent in the system roughly in half

LGTM!

bors r+


Tooltip: Add z index

Test Plan

@edzmh96 Thanks for creating this PR! Couple of comments:

Could you add a PR description (why we're adding this) + test plan of how to test this?
Could you ensure that the builds pass? Looks like right now there is an issue of the files not running through prettier. In VSCode, make sure you've installed the recommended extensions.

@christianvuerings updated the description!


Fix preview branch build and enable automation

This PR adds new automation to the preview branch. It depends on https://github.com/snapcrafters/ci/pull/28, and should not be merged before that PR.
These files are very similar to those on the candidate branch, but specify a different channel.
@merlijn-sebrechts before this can be merged, we need to create an environment named "Preview Candidate Branch" which has all the necessary secrets in it, noting the name of ${{ secrets.SNAP_STORE_PREVIEW_STABLE }}.
Let me know what you think!
Edit*: This grew arms and legs - it turns out the preview build was actually quite broken, and took some significant wrangling to get it to build and launch on Ubuntu. I've tried to break down into small commits where possible, but let me know if there is any further context required!
Closes #244
Fixes #126
Fixes #219

Oh I didn't realise the build was broken, I'll take a look at that soon.

I'd like you to fix the build first, either in this PR or in a separate one, so we can immediately test everything after merging.

Latest commit grabs the changes from #202 and integrates them here.

This last commit builds fine on my box.
I also noticed that gmic is missing here - I'm 95% through porting that for core22 / gimp 2.99 but not quite finished - will leave on a separate branch unless I finish before this merges :)

@merlijn-sebrechts this is now actually "working", but the job takes too long and is killed by Github Actions (we can't make it timeout after a longer time without paying).
Are you happy to merge this?

Latest commit tested with remote-build - resulting snap enabled this screenshot to happen :)

This is now good to go from my side -- both architectures build fine on Launchpad as of today!

@merlijn-sebrechts success! We have a green CI run for the pull request flow! (see here).
I've removed the commit that enabled it to trigger, because I had to loosen the branch constraint on the pull request flow, but I think I can fix that in a later request when we try to converge the two branches?


goseg: pack schdule doesnt roll over to next month

oops, will get this fixed

@mopfel-winrux couldn't reproduce this bug. Can you see if it's still happening?

looks like its fixed. closing


OCP 4.5 - v1beta1 CRDs

https://docs.openshift.com/container-platform/4.5/release_notes/ocp-4-5-release-notes.html#ocp-4-5-deprecated-v1beta1-crds

Looking into

Covered by the below, which we pull in via CI:

https://github.com/swade1987/deprek8ion/blob/master/policies/kubernetes-1.19.rego#L28-L32


Invalid Domain Name

cat main.tf
provider "aws" {
region = "ap-southeast-1"
version = "~> 2.15"
}
provider "null" {
version = "~> 2.1"
}
module "elasticsearch" {
source                  = "git::https://github.com/cloudposse/terraform-aws-elasticsearch.git?ref=master"
namespace               = "eg"
stage                   = "dev"
name                    = "es"
dns_zone_id             = "Z25Y69PJJVXXXX"
security_groups         = ["sg-0bebedf9d19xxxxxx"]
vpc_id                  = "vpc-0f9576da430xxxxxx"
subnet_ids              = ["subnet-0c3a9905724xxxxxx"]
zone_awareness_enabled  = "true"
elasticsearch_version   = "5.6.16"
instance_type           = "m4.large.elasticsearch"
instance_count          = 4
iam_role_arns           = ["arn:aws:iam::12364176xxxx:role/es"]
iam_actions             = ["es:ESHttpGet", "es:ESHttpPut", "es:ESHttpPost"]
encrypt_at_rest_enabled = "true"
kibana_subdomain_name   = "kibana-es"
advanced_options {
"rest.action.multi.allow_explicit_index" = "true"
}
}
terraform init
Initializing modules...

module.elasticsearch
module.elasticsearch.label
module.elasticsearch.user_label
module.elasticsearch.domain_hostname
module.elasticsearch.kibana_hostname

Initializing provider plugins...
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
terraform apply
Error: module.elasticsearch.aws_elasticsearch_domain.default: "domain_name" must start with a lowercase alphabet and be at least 3 and no more than 28 characters long. Valid characters are a-z (lowercase letters), 0-9, and - (hyphen).

We just add new variable allow developers to config domain name
https://github.com/cloudposse/terraform-aws-elasticsearch/blob/0.13.0/variables.tf#L227

This issue still persists. Even with defining a variable "elasticsearch_subdomain_name" (variable name: test-domain-001), terraform plan is still failing with picking up
Error: invalid value for domain_name (must start with a lowercase alphabet and be at least 3 and no more than 28 characters long. Valid characters are a-z (lowercase letters), 0-9, and - (hyphen).) with module.elasticsearch.aws_elasticsearch_domain.default[0], on .terraform/modules/elasticsearch/main.tf line 102, in resource "aws_elasticsearch_domain" "default": 102:   domain_name           = module.this.id
Somehow its not picking up the variable defined.

@vikram-yerneni @nnt this issue is due to some confusing terminology I believe... The domain_name value is set as module.this.id which needs to satisfy the constraints that have been mentioned. module.this.id is from https://github.com/cloudposse/terraform-null-label and the collection of namespace, stage, environment, name and similar parts. You'll need to adjust those to correctly get the domain_name value to be valid.
Please try that out and post back with your actual code if you're still experiencing issues.
I'm going to close this issue as I believe it's just a miscommunication which I've hopefully just rectified. Thanks folks!

@Gowiem Instead of all four variables namespace, stage, environment, name (based on my colleagues advice), I went with name variable and it worked.
Thanks for the inputs here. Appreciate it.


fixs warnings from upgrading to Julia 0.4-rc2

Hi Kyle/Mike -- This PR fixes all the warnings from Julia 0.4-rc2.
Best, Jeff

Hi @kbarbary , I removed the unnecessary calls to @compat.

Looks good to me!

HI @kbarbary , Would you tag a new version of WCSLIB.jl with this revision? Thanks!

will do.


Seller signed psbt validation fixes

Corrected seller signed psbt validation.

In which case does a seller PSBT have multiple inputs?

I believe the intention of this code was to validate that there was 1 input and 1 output.  However, the code was performing the same check twice in the inputs, rather than both the intputs and outputs.


Update protobuf

Protobuf 3.16 is no longer being updated by that community. This upgrades to the closest protobuf version to 3.16 that is still supported which includes current security updates.

@cjvolzka think the ppc jenkins failed for other PRs, so it is probably good if it succeeds with s390

It would be nice to add a comment in the commit message or in the scripts about how/why this particular version was chosen, but looks good otherwise.

Updated description.

Jenkins Linux amd64 Build #7903 [push] Update protobuf (#1747) ... started at 13:05

Jenkins Linux s390x Build #7921 [push] Update protobuf (#1747) ... started at 14:06

Jenkins Linux ppc64le Build #6968 [push] Update protobuf (#1747) ... started at 14:09

Jenkins Linux amd64 Build #7903 [push] Update protobuf (#1747) ... passed after 2 hr 37 min

Jenkins Linux ppc64le Build #6968 [push] Update protobuf (#1747) ... passed after 2 hr 40 min


Icon Translation/Localization

Hi,
Could you make a variation of the "bank-transfer" icon.
Translating the "transfer" into "overboeking".
So it will say: Bank Overboeking.
You could also add:
Invoice icon. Translate into: "Factuur".
Thanks.

It would make sense to translate it to the most popular languages right away, or get rid of the text altogether so that we can add it manually.

@jgautheron I basically agree that in some cases it would make sense to translate them right away. Nonetheless we're talking online shopping, and in that case I'm not so sure, that translating the bank transfer icon (as well as cash on delivery and cash on pickup) is absolutly necessary.
In my experience people who are shopping online actually do understand those three icons. On the other hand I chose to translate the invoice icon, as people didn't understand its meaning in German and Italian. But of course I'm open to other opinions and in case will add such translations as requested.
Anyway, what do mean with "get rid of the text altogether so that we can add it manually"? You mean something like a custom build via Fontello or via CSS?

Yes usually they do understand them. But for an optimal shopping experience, translating the parts where you use text is best.
One quick fix though: I've said to add a bank-tranfser icon saying: bank overboeking, but the official correct Dutch translation would be: "Bankoverschrijving". So not: "Bank overboeking".
Wil you be adding these icons? Thanks
I really love them!

@AlexanderPoellmann They are not necessarily understood. Also there is a difference between end-users and the customers for who you're working on the website: usually they'll want everything translated.
A custom build that excludes text could be really helpful, since you cannot possibly cover all languages.

@Feggel As mentioned before, I'm definitly considering adding translated icons depending on this discussion here!
@jgautheron Don't misunderstand me, with end-users I always mean customers. People who are working on the website are backend-users for me ;)
Nonetheless I agree with you guys, that translated icons will be best in terms of UX/shopping experience. On the other hand we'll have to define which languages to be added and which not, so that we don't blow PaymentFont with tons of duplicate icons.

Picking the languages will be hard. In order to be able to use the font on every eCommerce store my company handles, it would require about 30 languages.
It might be better to split the languages by regions, ex. "west-eu", "east-eu", "africa", "america", "asia", create a build for each, plus one which contains "all".

Perhaps, it is worth in this context to use the content attribute for internationalisation.
As I needed German translations in a project, I changed the pf style like this (incl. a border to make it more clear):
.pf {
            font-size: 30px;
            border: 1px solid #eee;
            padding: 5px;
            border-radius: 5px;
            color: #eee;
            margin: 2px 0;
            position: relative;
            height: 42px;
            width: 72px;
            display: inline-block;
            &:before {
                position: absolute;
            }
        }

Than, I was able to introduce own classes, e.g. pf-vorkasse:
.pf-vorkasse:before {
  content: "Vorkasse";
}

The position absolute is necessary as otherwise the own payment classes and the pf classes will not have the same top position (tested in Chrome an FF)

Hi Alex,
"Barzahlen" is a payment provider, not a translation. Please see here: https://www.barzahlen.de/de/

For the translations.... isn't it possible to have the onlineshop "Write" into the button? .... say you provide the Icon as for Invoice  but with a text attachement "invoice" that the shop can, depending on the choosen language translate?


Fix merge commit title and message

Fixes #1264
testing: PASSED: manual, visually inspecting in chrome

Closing as per discussion in #1264.


nixos/nginx: add recommendedZstdSettings

Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

blocked by https://github.com/tokers/zstd-nginx-module/pull/15

Can we have a NixOS test for this?

I am not really motivated to write one but I updated the source with the fix.

This option is not even documented in the release notes.

You you asked if I could write a NixOS test and I said no because I lack the motivation and time to write one for that. Since this is a none breaking chance and does not change anything for people not enabling the option there is no risk of breaking things.

This option is not even documented in the release notes.

I don't think it is note worthy for people reading the changelog. Browsers don't have any support for zstd, yet, and adding this was mainly motivated to have compression support for nix-archives.

These might be valid arguments but maybe you should have brought them up instead of just going ahead.


Create implicit convertation between ChatId and Update

I would like to request/propose a feature
So usually i do stuff like that to send response
 public async Task Reply(TelegramBotClient client, Update hook)
 {
        await client.SendTextMessageAsync(
                hook.Message.Chat.Id, "Hello World");
 }

But I think it will be good to send message without hardcode chat id position because sometimes it can vary by message type, so i create extension method to retrieve ChatId from hook, but i think its better to have implicit convertion between Update and ChatId so i can use method like this:
 public async Task Reply(TelegramBotClient client, Update hook)
 {
        await client.SendTextMessageAsync(
                hook, "Hello World");
 }

UPDT: or extension method
That's great. Let us know what's on on your mind and please try to be specific.
You also can chat with us on Telegram

Main issue what you wanna do in case where update doesn`t have any chatId/FromId.
My suggestion - add extension methods:
User? GetUser(this Update update);
Chat? GetChat(this Update update);

Which will return corresponding objects if they exist (or sender != anonymous user) after that you can easily implement methods that you need with Reply if chat exists and handle situation when there is none.
Additionally those methods useful while logging/user authorization, etc.

await client.SendTextMessageAsync(hook.Message.Chat.Id, "Hello World");

As a shortcut, you can already use hook.Message.Chat
I don't plan to make further such shortcuts because an Update object may reference many Chats (ex: forwarding from one chat to another, replying to a message from another group)


feat: compute inner products with multiple GPUs  (PROOF- 716)

Rationale for this change
Use multiple GPUs to compute inner products.
This is part of improving the performance of the inner product proof.
What changes are included in this PR?

Reworked  the inner_product to use multiple GPUs
Changed s25t::element's constructor to be constexpr

Are these changes tested?
Yes.

:tada: This PR is included in version 1.10.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


_IncompatibleKeys error when loading a contrastive pre-trained model

Hi,
I followed the steps in the README.md and ran the suggested command to do contrastive pretraining:
cd src/contrastors
torchrun --nproc-per-node=8 train.py --config=configs/train/contrastive_pretrain.yaml --dtype=bf16

I changed was the output_dir variable in configs/train/contrastive_pretrain.yaml so that it would store the model on local disk ,e.g., I set it to output_dir: "nomic-embed-text-v1-unsupervised-1st-try". I also changed the data config configs/data/contrastive_pretrain.yaml so that I only used a subset of the data (to test it out).
However, when then I then went to run the contrastive fine-tuning step, this time changing the model_name in configs/train/contrastive_finetune.yaml from "nomic-ai/nomic-embed-text-v1-unsupervised" to "nomic-embed-text-v1-unsupervised-1st-try/final_model" so that I could try my own contrastive pretrained model, and changing the data config configs/data/finetune_triplets.yaml so that I only used a subset of the data, I got an _IncompatibleKeys error.
I believe this comes from the load_state_dict function. The missing keys and unexpected keys were of the form (for all 112 keys I think):
missing_keys: [‘emb_ln.weight', 'emb_ln.bias’,…]
unexpected_keys: [‘trunk.emb_ln.bias', 'trunk.emb_ln.weight’,…]

i.e., I think the unexpected_keys had an extra "trunk." prefix added to it causing the error (they both had exactly 112 keys).
I tried removing the "trunk." prefix (like a simplified version of what the remap_bert_state_dict function does in contrastors/models/encoder/bert.py) and reran, but then got the following Tensor size mismatch error:
RuntimeError: Error(s) in loading state_dict for NomicBertModel:
	size mismatch for embeddings.word_embeddings.weight: copying a param with shape torch.Size([30528, 768]) from checkpoint, the shape in current model is torch.Size([50257, 768]).

I'm not sure why there would be such a mismatch difference. Could someone please advise?
Many thanks.

thanks for filing this! i am able to reproduce your bug. let me think a little bit about how to make this more clear.
basically what's happening is here i check if pretrained is none and then try to load a NomicBertModel model from the model_name path, however the model is saved as a BiEncoder model.
the quick fix is to replace the following in your yaml
model_name: nomic-ai/nomic-bert-2048
pretrained: <path to checkpoint>

i'll think a little bit about how to make this cleaner. the reason i save the BiEncoder model vs. the underlying trunk object is that there are scenarios where there are learnable layers after the trunk in the BiEncoder model.
let me know if you still face any issues with this!

Hi, thank you very much for this! I sort of understand your thinking now. What you suggested worked without any issues. Thank you for being so responsive and helpful!


Added web interfaces getById and getByName for pipeline and strategy

@ajordens This is the front50 fix for the issue I raised last week.
I dont know where the consumers for this would be. I'd like deck to ask for pipelines similar to applications so that it hits the storage service to check cache freshness. I assume orca would want to do something similar as well.

So, it turns out that there is no static/runtime ambiguity raised between {id}/history and instance/{id}.
So I can just add an ID getter (and PUT) in, or I can add deprecations and alternate APIs to make consistent naming.

Going to close this PR ... it's hung around for a few months and afaik isn't a particular priority for either of us right now.


Uncaught AttributeError in downloader.py line 1068

Aider version: 0.63.2
Python version: 3.11.7
Platform: macOS-10.16-x86_64-i386-64bit
Python implementation: CPython
Virtual environment: No
OS: Darwin 23.4.0 (64bit)
Git version: git version 2.39.3 (Apple Git-146)
An uncaught exception occurred:
Traceback (most recent call last):
  File "aider", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "main.py", line 841, in main
    coder.run()
  File "base_coder.py", line 741, in run
    self.run_one(user_message, preproc)
  File "base_coder.py", line 778, in run_one
    message = self.preproc_user_input(user_message)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "base_coder.py", line 767, in preproc_user_input
    return self.commands.run(inp)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "commands.py", line 241, in run
    return self.do_run(command, rest_inp)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "commands.py", line 213, in do_run
    return cmd_method(args)
           ^^^^^^^^^^^^^^^^
  File "commands.py", line 984, in cmd_help
    res = install_help_extra(self.io)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "help.py", line 24, in install_help_extra
    res = utils.check_pip_install_extra(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "utils.py", line 334, in check_pip_install_extra
    __import__(module)
  File "__init__.py", line 1, in <module>
    from llama_index.embeddings.huggingface.base import (
  File "base.py", line 12, in <module>
    from llama_index.core.base.embeddings.base import (
  File "base.py", line 14, in <module>
    from llama_index.core.callbacks.base import CallbackManager
  File "__init__.py", line 10, in <module>
    from llama_index.core.base.response.schema import Response
  File "schema.py", line 9, in <module>
    from llama_index.core.schema import NodeWithScore
  File "schema.py", line 27, in <module>
    from llama_index.core.utils import SAMPLE_TEXT, truncate_text
  File "utils.py", line 89, in <module>
    globals_helper = GlobalsHelper()
                     ^^^^^^^^^^^^^^^
  File "utils.py", line 45, in __init__
    import nltk
  File "__init__.py", line 180, in <module>
    from nltk.downloader import download, download_shell
  File "downloader.py", line 2475, in <module>
    _downloader = Downloader()
                  ^^^^^^^^^^^^
  File "downloader.py", line 515, in __init__
    self._download_dir = self.default_download_dir()
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "downloader.py", line 1068, in default_download_dir
    for nltkdir in nltk.data.path:
                   ^^^^^^^^^
AttributeError: partially initialized module 'nltk' has no attribute 'data' (most likely due to a circular import)

Thanks for trying aider and filing this issue.
This looks like a duplicate of #2063. Please see the comments there for more information, and feel free to continue the discussion within that issue.
I'm going to close this issue for now. But please let me know if you think this is actually a distinct issue and I will reopen this issue.Note: A bot script made these updates to the issue.


Is there a way to add a non stdin/stdout formatter?

Hi,
So I tried to get flutter format working in apheleia. The problem is, this command
directly writes to the file. AFAIK, there is no option to write to stdout.
My question is, is there a way to add a formatter that only writes directly
to the file?

Is it okay if the file is copied to a temporary directory before the formatter is run on it?

Yeah, that seems fine.

I pushed to a temporary branch (linked above) my work-in-progress implementation, which should support the interface necessary for this type of formatter. Unfortunately, the implementation still has a bug that needs to be fixed for it to work properly. At the moment I don't have the time to work further on it, but I figured I would publish the code so somebody else can take a look if they are so inclined.

Implemented with the latest commit.


Compiler Optimizing Memory Access in Bad Ways, Triggered by Seemingly Unrelated Change

This is an issue with a number of facets that I am going to try and lay out.
I want to optimize a piece of code and run into a number of problems with compiler optimization behaviour that I struggle to rein in.
This is my initial state: https://github.com/949f45ac/xmrig-HIP/blob/f5a71d80ef7f48761a9f1b496bafc19f9a42747d/src/nvidia/phase2.cu#L592
There is fast_div_v2 taking three arguments. RCP is a __shared__ lookup table that gets initialized at the start of the kernel, here: https://github.com/949f45ac/xmrig-HIP/blob/f5a71d80ef7f48761a9f1b496bafc19f9a42747d/src/nvidia/phase2.cu#L486
Now I replace fast_div_v2 with a version that works without a lookup table and is slightly faster. All good, I get improved performance. Call now simply looks like this:
fast_div_v2(reinterpret_cast<ulonglong2*>(&c)->y, din);
But when I remove RCP (already unused inside the loop!) from the kernel completely, that is, remove the initialization at the top, I suffer a ~30% loss in performance.
Very weird!
I looked at the genereted ISA and the problem basically comes down to the following. If we take a look at this whole chunk of code:
		// long_state is 2 MB of memory paced throughout 512 MB
		// j1 is random adress inside
		ulonglong2 y2 = long_state[j1];
		uint64_t t1_64 = c.x | (((uint64_t) c.y) << 32);

		// Reads memory adjacent to y2, ie. long_state[j1^1], [j1^2], [j1^3]
		LOAD_CHUNK(chunk1, j1, 1);
		LOAD_CHUNK(chunk2, j1, 2);
		LOAD_CHUNK(chunk3, j1, 3);

		PRIO(3) // This does s_setprio 0x3.

		const uint din = ( (c.x) + (sqrt_result << 1)) | 0x80000001UL;
		uint64_t n_division_result = fast_div_v2(reinterpret_cast<ulonglong2*>(&c)->y, din);
		uint32_t n_sqrt_result = fast_sqrt_v2(t1_64 + n_division_result);

		y2.x ^= division_result ^ (((uint64_t) sqrt_result) << 32);

		division_result = n_division_result;
		sqrt_result = n_sqrt_result;

		uint64_t hi = UMUL64HI(t1_64, y2.x);
		uint64_t lo = t1_64 * y2.x;

		ulonglong2 result_mul = make_ulonglong2(hi, lo);

		chunk1 = v_xor(chunk1, result_mul);
		result_mul = v_xor(result_mul, chunk2);

		// Write back to long_state[j1^1]
		STORE_CHUNK(j1, v_add(chunk3, d_old), 1);

Loads are going to be very slow, since they are from a random adress.
Hence to reach maximum performance:

Loads should be scheduled ASAP.
Blocking on loads via s_waitcnt should be delayed as much as possible by the compiler ->
Everything up to the value of n_sqrt_result does not depend on the loads, so all of that should be executed before the first s_waitcnt

And so when removing the initialisation of RCP we can see that all of this goes wrong:
        global_load_dwordx4 v[28:31], v[54:55], off
        v_rcp_f32_e32 v38, v37
        ;;#ASMSTART
        V_CVT_F32_UBYTE0 v36, v3
        ;;#ASMEND
        global_load_dwordx4 v[32:35], v[52:53], off
        v_sub_u32_e32 v42, 0, v3
        v_fma_f32 v37, v37, v38, -1.0
        v_add_u32_e32 v39, s7, v38
        v_fma_f32 v36, v36, v38, v37
        v_mul_f32_e32 v36, v36, v39
        v_rndne_f32_e32 v36, v36
        v_cvt_i32_f32_e32 v36, v36
        v_lshlrev_b32_e32 v43, 9, v38
        v_mov_b32_e32 v46, v4
        v_mov_b32_e32 v48, v4
        v_sub_u32_e32 v36, v43, v36
        v_mul_hi_u32 v38, v26, v36
        v_mov_b32_e32 v49, v4
        v_cmp_eq_u32_e32 vcc, 0, v2
        s_and_b64 vcc, exec, vcc
        s_waitcnt vmcnt(1)
        v_add_co_u32_e64 v11, s[0:1], v30, v11
        v_addc_co_u32_e64 v12, s[0:1], v31, v12, s[0:1]
        v_add_co_u32_e64 v9, s[0:1], v28, v9
        v_addc_co_u32_e64 v10, s[0:1], v29, v10, s[0:1]
        v_mad_u64_u32 v[36:37], s[0:1], v27, v36, v[26:27]
        global_load_dwordx4 v[28:31], v[40:41], off
	;; And so on

Instead of doing all loads at once, they are far spread out, and the sqrt calculation is pushed a long way down.
 All loads are done at once and quite early, and awaited only when sqrt calculations have been executed to the last bit.
Yet even with this case there are some problems, mostly that loads aren’t scheduled completely ASAP yet (some instructions are needlessly placed before), and the whole thing crumbles if #pragma unroll 4 is used over #pragma unroll 2.
Additionally, of course, I would like to have some control over this and not have it magically work because there is some unused __shared__ memory declared somewhere else in the kernel.
So I have been trying to somehow force the compiler into doing what I want.
I am using this macro:
#define FENCE32(x) asm volatile("v_mov_b32 %0, %1" : "=v" (x) : "v" (x) : "memory");
The idea is that at least memory operations cannot be reordered with respect to the state of the input variable x at the given point.
So for example I can do
		// ==== LOAD 2 : chunks ====
		LOAD_CHUNK(chunk3, j1, 3);
		ulonglong2 y2 = long_state[j1];
		LOAD_CHUNK(chunk1, j1, 1);
		LOAD_CHUNK(chunk2, j1, 2);

		PRIO(3)
		FENCE32(c.x);
		uint64_t t1_64 = c.x | (((uint64_t) c.y) << 32);

And the compiler starts doing the loads ASAP and not spread out, since I have just disallowed it from using c.x (which is needed by some parts of the following code) before it’s scheduled all the loads.
In the same vain I am trying to do
uint32_t n_sqrt_result = fast_sqrt_v2(t1_64 + n_division_result);

FENCE32(n_sqrt_result);

STORE_CHUNK(j1, v_add(chunk3, d_old), 1);

but here my method starts failing already, since the compiler will INSIST on calculating v_add(chunk3, d_old) (and hence waiting for the chunk3 load to complete) before calculating some parts of fast_sqrt_v2, even though the store instruction must wait for the sqrt calculation to be complete.
Finally I have been able to achieve the desired result (more or less) by implementing v_add in volatile asm – so that it cannot be reordered with respect to FENCE32(n_sqrt_result):
		uint32_t n_sqrt_result = fast_sqrt_v2(t1_64 + n_division_result);
		FENCE32(n_sqrt_result);

		uint4 dl = make_uint4(d_old.x, d_old.x >> 32, d_old.y, d_old.y >> 32);
		asm volatile(
			"v_add_co_u32_e32  %0, vcc, %8, %4 \n\t"
			"v_addc_co_u32_e32 %1, vcc, %9, %5, vcc \n\t"
			"v_add_co_u32_e32  %2, vcc, %10, %6 \n\t"
			"v_addc_co_u32_e32 %3, vcc, %11, %7, vcc \n\t"
			: "=v" (dl.x), "=v" (dl.y), "=v" (dl.z), "=v" (dl.w)
			: "v" ((uint32_t)chunk3.x), "v" ((uint32_t)(chunk3.x >> 32)), "v" ((uint32_t)chunk3.y), "v" ((uint32_t)(chunk3.y>>32)),
			  "v" (dl.x), "v" (dl.y), "v" (dl.z), "v" (dl.w) : "vcc", "memory");

		reinterpret_cast<uint4*>(long_state)[j1^1] = dl;

		y2.x ^= division_result ^ (((uint64_t) sqrt_result) << 32);

It is the fastest version, but still there are problems:

NOT PORTABLE. ISA for addition changed between gfx803 and gfx900, so I cannot build a single binary that works on both. Cf. this issue: https://github.com/ROCm-Developer-Tools/HIP/issues/754
Actually it is only fastest when using #pragma unroll 4 which finally stops creating incorrect code. Lower unrolls are slower than the silly version with the unused RCP array.

I have created a branch documenting the variants I’ve discussed, and a few inbetween, here: https://github.com/949f45ac/xmrig-HIP/commits/bug_doc0
In the configs/ folder there are two example configs the program can be tested with. They will mine on the Monero testnet. For every commit I have put a comment inside these configs that tells how it performed for me.
I understand this is a complex issue, but maybe someone’d interested in looking into it, since it’s quite baffling.
Cheers!
ps. Since creating the examples branch I have noticed that fast_div_v2 contains some aliasing code that may be UB and tried to fix it – it didn’t really help the problem at hand, though.

Is this still an issue? If not, can we please close it?

Original ticket is more than a year old and the person that opened ticket originally has not responded to the latest request.  If this is still an issue, please file a new ticket and we will be more than happy to investigate it.  Thanks!


Calculation Issues

These are real prices that my island has, but my result is always minimum: 999 and maximum: 0 no matter what pattern I put as the previous one. No time traveling has happened.
https://turnipprophet.io?prices=<IP_ADDRESS>.<IP_ADDRESS>.180....

I will have to do a deeper dive into the calculations a bit later on, but my initial quick review indicates that the 51 you got on Wed AM is impossible. I will break down the calculations deeper later today.

I just got a similar result and have since this morning: https://turnipprophet.io/?prices=<IP_ADDRESS>.<IP_ADDRESS>.211, no time travel on my end either. Thanks for putting this tool together!

I will have to do a deeper dive into the calculations a bit later on, but my initial quick review indicates that the 51 you got on Wed AM is impossible. I will break down the calculations deeper later today.

It may be impossible, but it happened. So not sure what exactly is going on.

I just got a similar result and have since this morning: https://turnipprophet.io/?prices=<IP_ADDRESS>.<IP_ADDRESS>.211, no time travel on my end either. Thanks for putting this tool together!

@taromsn  is this the first week you've bought Turnips on your island? Your pattern is perfectly possible for a buy price that is between 103-110, but not possible for a buy price of 102 or lower.

Ah, I see, this is the first time I bought turnips on my island, but not the first time I ever bought turnips.  My mistake.

I have a similar issue with different prices: https://turnipprophet.io?prices=.<IP_ADDRESS>.<IP_ADDRESS>.171...&first=true
Everything was fine until I entered this morning's price of 171

Thank you for the example data! I will include it in my on-going investigation of this.

Another example. I'm fairly confident in my data (though I don't have screenshots of the prices showing in-game, so I suppose a mistake somewhere is possible). https://turnipprophet.io?prices=.<IP_ADDRESS>.<IP_ADDRESS>.57.53..&first=true

@seanharding you've got the "first time buyer" feature enabled - are you certain this week is the first week you've ever bought on your island? Based on your numbers, that doesn't seem right.
First time buyer locks you into a Fluctuating pattern, but your numbers are very clearly showing a Decreasing pattern. If you disable the first time buyer feature, you'll get predictions.
https://turnipprophet.io?prices=.<IP_ADDRESS>.<IP_ADDRESS>.57.53..

Another example here: https://turnipprophet.io?prices=<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.81
I think that would be a fluctuating pattern? Or is the 240 on Tuesday enough for a small spike?

Another example here: https://turnipprophet.io?prices=<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.81
I think that would be a fluctuating pattern? Or is the 240 on Tuesday enough for a small spike?

This looks like a large spike pattern (Can be seen from deleting the buy price).
This case is interesting and looks like floating point problem (Since 88/110 is exactly 0.8 which is the minimum possible rate for first large spike price), would investigate this more.

thanks!

Another example here: https://turnipprophet.io?prices=<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.81
I think that would be a fluctuating pattern? Or is the 240 on Tuesday enough for a large spike?

Sorry I remembered the numbers incorrectly... The minimum possible rate for first large spike price is 0.85, not 0.8. So since 88/110 < 0.85, this does not match the pattern, but the rest of the data perfectly match the large spike pattern.
Is the buy price 110 the accurate buy price from your own island on last Sunday (and there's no time travel involved in this week)? The maximum possible buy price from the pattern algorithm would be 103, so this is too far to be a floating point problem iiuc.

It's not actually mine, I'm just the guy in the friend group with a github account. ;) I've asked again and there has been no time travel but the buy price might have been 101. In that case everything seems to work out fine.

@seanharding you've got the "first time buyer" feature enabled - are you certain this week is the first week you've ever bought on your island? Based on your numbers, that doesn't seem right.

It’s the first time I have bought in the game at all. However I did buy on another island, which may have screwed it up. If that’s a factor it might be helpful if the UI mentions it, even if it can’t predict in that case.

It’s the first time I have bought in the game at all. However I did buy on another island, which may have screwed it up. If that’s a factor it might be helpful if the UI mentions it, even if it can’t predict in that case.

So you're saying that you didn't buy from your island this week?
If that's the case, then you should not have that feature turned on. For what it's worth, the help text above the buttons does already make the "your island" distinction.

I had this same problem, it started when the last two prices (Fri PM and Sat AM) were the same. When I switched Sat AM price down by one bell, the chart normalized. https://turnipprophet.io?prices=108..<IP_ADDRESS>.<IP_ADDRESS>.183.183 vs https://turnipprophet.io?prices=108..<IP_ADDRESS>.<IP_ADDRESS>.183.182.

I'm also getting this problem. this is my second time buying turnips, and I missed them one week. It's also my first week tracking with this app. I've only ever bought and sold on my island.
https://turnipprophet.io?prices=<IP_ADDRESS>.<IP_ADDRESS>.<IP_ADDRESS>.105

I have moved the discussion from @jdelgallatin to issue #182 and the discussion from @mavdnoise to issue #184.


[website] Remove file import JS limitation

Why
Right now, users can only upload JS files through the "Import files" modal. I don't think we should limit it to JS only, as users should be able to upload images and maybe JSON/txt files too.
How
Removed the accept="js" from the file import modal.
Test Plan

See if we can upload more types of files after this change, e.g. assets and code.

Verified locally and works great. Yeah that limitation didn't make any sense 👍


Support left mouse click

does the context-menu support left mouse click? It would be great for such a use case

Sorry for the delayed response. Yes, the library supports left click, in fact it's not tied to either left or right click. To use left click you'd just map it to the "click" event instead of the "contextmenu" event. In the example/index.htm file just change "contextmenu" to "click" and it'll work with left click.