Support the Open Meteo custom data format

OM is a custom format created by and for open-meteo.com. It’s a Zarr-like format that I bet would be easy to Kerchunkify.
There seems to be a considerable catalog of weather data curated by Open Meteo on AWS: https://github.com/open-meteo/open-data
Here’s some more info about the format:

https://github.com/open-meteo/open-data?tab=readme-ov-file#file-format
Reference implementation: https://github.com/open-meteo/open-meteo/blob/main/Sources/SwiftPFor2D/SwiftPFor2D.swift

Some potential sharp edges:

“Each individual chunk varies in size”
“As of now, the Open-Meteo API does not directly support partial reads from S3, but this functionality will be implemented in the future.”

xref: https://github.com/open-meteo/open-data/issues/11

Sounds like a fun little project for someone with spare time or funding. Let's see if open-meteo is interested in your issue. Of course, I know nothing about the format, but "zarr-like" is good.

Each individual chunk varies in size

So maybe it needs ZEP0003, yet another reason why that should be adopted and released. Feel free to add another point in the zep discussion.

Thanks Martin! I added to the discussion.

Over the past week I have been thinking about a portable implementation to access Open-Meteo files from different platforms. Currently, it is only available in Swift and tailored to a specific use-case. To make it more generic, a dedicated library in C/Swift/Rust could be implemented with bindings to different programming languages.
The most basic interface could look like:
let file = OmFile(path: "era5_temperature_2m_2020.om")
/// One year of era5 data for a given location
let data = file.read(offset=[340, 450, 0], count=[1,1,8760])

Of course this kind of interface is useless to support cloud-native operations. The interface should support:

Chunked operations: Read only a small portion of a file. E.g. Read a small area
Sans IO: Ideally the library does not do any IO, but delegates IO to higher levels. Therefore async and concurrent IO can be realised independent from the library.
Concurrent reads: Chunks must be decompressed in parallel.
No allocations: Ideally, the library does not allocate any heap memory. Some buffers must be provided by the caller.
No dependencies: Once IO and concurrency can be ignored, it should be easy to stay free of any dependencies.

Typically such an interface is realised by providing callbacks for IO and implement threading into the library. In combination with async this makes it very hard to make a portable library.
The Open-Meteo file format is relatively simple consisting only of (more details here):

Header with attributes like version, data and chunk dimensions, compression type and scalefactor
Lookup table to resolve chunks addresses
Data block with chunks

A read operation needs 3 IO operations:

Read meta data with dimension information
Calculate which chunks are required and read data offsets in the lookup table
Read compressed chunks, decompress them and return a data array to the user

Sans IO example using read/decompress functions
To eliminate any threading and IO, a library can provide functions that return the IO read offsets and separately accept data in a second function call. In a very simplified way, this could look like the following dummy code (Note: open_meteo_* function calls are kept C-style to make them as portable as possible)
let file = await FileHandle(path: "era5_temperature_2m_2020.om")

let params = ReadOperation(
  // Attributes from the file meta header
  dimensions = [3600, 1800, 8760],
  chunk_dimensions = [3, 3, 20],
  compression = .pfor,
  scalefactor = 20,

  // What to read
  read_offset = [340, 450, 0],
  read_length = [1, 1, 8760]
)

var output = [Float]()

// Which indices need to be read from the lookup table? Attributes `offset` and `length`
let indexRead = open_meteo_read_index(params)
let indexData = await file.read(indexRead.offset, indexRead.length)

// Which chunks need to be read from the data block? Attributes `offset` and `length`
let chunkRead = open_meteo_read_data(params, indexData)
let chunkData = await file.read(chunkRead.offset, chunkRead.length)

// Once data has been read, decompress chunks and place data into the output array
open_meteo_decompress_chunk(params, chunkData, &output)

print(output)

Using functions that return IO operations with offset and length makes it trivial to implement any kind of IO. The await file.read(offset, length) can be replaced by a HTTP library to fetch data from S3. A cache layer could be added as well.
Example realistic IO operations, no concurrency
As multiple data blocks must be read from file, it needs a couple of loops.
var output = [Float]()
var buffer = [Uint8](size: open_meteo_buffer_size(params))

while (let indexRead = open_meteo_read_index(params)) {
  let indexData = await file.read(indexRead.offset, indexRead.length)

  while (let chunkRead = open_meteo_read_data(params, indexData)) {
    let chunkData = await file.read(chunkRead.offset, chunkRead.length)
    open_meteo_decompress_chunk(params, chunkData, &buffer, &output)
  }
}

print(output)

Example full concurrency
Alternatively, reads could be batched, read concurrently and processed concurrently show-casting the full potential of this approach. (Note: Pure dummy code. This is not intended to be valid code)
// Automatically merge and break up IO to ideal sizes
// Merging is important to reduce the number of IO operations for the lookup table
// A maximum size will break up chunk reads. Otherwise a full file read could result in a single 20GB read.
let params = ReadOperation(
  [...]
  io_size_max = 65536
  io_size_merge = 512
)

// Submit all index reads concurrently (small number, 1-2 MB)
var indexDataFutures: [Future<T>]
while (let indexRead = open_meteo_read_index(params)) {
  let indexData = file.read(indexRead.offset, indexRead.length)
  indexDataFutures.append(indexData)
}
let indexDatas = await indexDataFutures

// Once we know the index, collect all data read concurrently (potentially big, up to 20GB)
var chunkDataFutures: [Future<T>]
for indexData in indexDatas {
  let chunkRead = open_meteo_read_data(params, indexData)
  let chunkData = file.read(chunkRead.offset, chunkRead.length)
  chunkDataFutures.append(chunkData)
}
let chunksDatas = await chunkDataFutures

// Once all data arrived, decompress it concurrently
// This could also be done while downloading data, to save memory
concurrent(cores=8) for chunkData in chunksDatas {
  // Each thread needs its own buffer
  var buffer = [Uint8](size: open_meteo_buffer_size(params))
  open_meteo_decompress_chunk(params, chunkData, &buffer, &output)
}

print(output)

The final interface could be as simple as providing the following functions:

open_meteo_read_index(params, &indexInternalOffset) -> Optional<(index, offset)>: Returns all index positions that need to be read
open_meteo_read_data(params, indexData, indexInternalOffset, &chunkInternalOffset) -> Optional<(index, offset)>: Provided the index data, read all chunks data positions that need to be read
open_meteo_decompress_chunk(params, chunkData, &buffer, &output, chunkInternalOffset)
open_meteo_buffer_size(params) -> Int64: Get the size of the buffer that is required to decompress
indexInternalOffset and chunkInternalOffset we be required internally to associate data to the right index/chunk. I was ignoring them in the example above.

In terms of IO read dependencies, a single index read will enable multiple chunk reads. This flow can be used to further optimise concurrent operations. E.g. while a part of an index is still being read, other data chunks can already be decompressed
flowchart LR
    ReadMeta --> ReadIndex45
    ReadMeta --> ReadIndex67
    ReadIndex45 --> ReadChunk42
    ReadIndex45 --> ReadChunk342
    ReadIndex45 --> ReadChunk372
    ReadIndex67 --> ReadChunk62
    ReadIndex67 --> ReadChunk32
    ReadChunk42 --> Decompress42
    ReadChunk342 --> Decompress342
    ReadChunk372 --> Decompress372
    ReadChunk62 --> Decompress62
    ReadChunk32 --> Decompress32

Separating IO and concurrent processing from the library, should make the library relatively straight forward. As long as all functions are strictly concurrency safe, the caller can chose freely how to do IO, async and concurrency.
The drawback is, that the library will be hard to implement in this fashion. However, I am using a similar approach already to write files as streams.
Another advantage is that this approach is very flexible. E.g. additional compression methods or filters can be added easily as callbacks. The library will be effectively reduced to handle how chunks are indexed, compressed and stored. It is therefore more an encoding than a file format and could be integrated as a binary blob into other systems.
Revise Format
The Open-Meteo format needs some smaller updates to be more generic:

Most importantly it needs support for more than 2 dimensions. I might limit the number of dimensions to 8. Are there any applications that need more than 8? I initially chose 2 dimensions to keep all chunk-operations easier. The code gets fairly complex to work with higher order hyper chunks.
Improve index: Each chunk is addressed by a 64-bit integer. For a large number of chunks, this may require a couple of MB just for the lookup table. This could be improved by using delta coding. E.g. Only for every 100 chunks the full 64 bit address is stored and afterwards only 32 bit offsets are stored. This would cut index size in half with only minimal added complexity to resolve addresses. Other file formats like HDF5 use trees, but this would required significantly more IO operations.

I considered adding support for attributes or the ability to add multiple arrays in a single file, but this will increase complexity. If features like this will ever be required, the format could be wrapped inside a container providing more complex features.
Next steps

I am not familiar with kerchunk and how it handles IO / concurrency. It would be great to receive feedback if the presented approach would fit kerchunk.

I (or a contributor) could implement a reference implementation. I would start in Swift which could later be ported easily to C. The underlaying compression is based on a modified version of TurboPFor in C with SIMD. C is therefore somehow required. I would also like to implement some data filters in C with better SIMD support. Ideally, I would also like to reimplement the entire compression layer.

Bindings for Python can be created based on the underlaying C implementation. The first version will only export the C-style functions as presented above. In later version a more user-friendly interface could be provided.

Expose metadata for all models in https://github.com/open-meteo/open-data. To read all data provided, you will need some metadata information like the length of each time-chunk. This can be added as a simple JSON file.

I haven't yet read the detail of what you have written, but I have a couple of thoughts from a quick scan.
First, the pattern where the metadata and chunk offsets for a given read operation are done, followed by reading a set of binary chunks for decompression: this is exactly the flow that kerchunk is made for. If those chunk offset/sizes can be extracted, there is no need for compiled custom code to be called at runtime - it can all be handled by kerchunk, and will come with the side benefits of distributable parallelism and concurrency for the fetches. Kerchunk supports JSON and parquet storage of the chunk information, the later being necessary where there are very large number of chunks. Indeed, this also gets around the "data is 2D problem", since the reference sets can index as many files as you like, to make the overall dataset many-dimensional.
The core question to be asked for kerchunking will be: what's the fundamental chunk size in the data, where "chunk" is a indivisible binary blob, usually a single compressed buffer. Of course, this is a factor of the data, but if the size is very small (~kbytes), then it becomes less practical to index potentially thousands of files, and perhaps it's better to regard each whole file as a chunk. Still may be worth trying, though!
Also, a quick glance at https://github.com/open-meteo/open-data?tab=readme-ov-file#file-format says that the data format is actually really simple. Do we need more than some numpy for finding all the offsets?

Thanks for the quick reply!

Of course, this is a factor of the data, but if the size is very small (~kbytes), then it becomes less practical to index potentially thousands of files

Unfortunately, this is exactly the case. Chunks are very small (~ kbyte). Chunks are stored sequentially and can be indexed through a lookup table (Int64 Array). Only one Int64 address is stored per chunk. The length of each chunk can be calculated with the address of the next chunk.
The chunk index lookup table can contain millions of chunks and requires a couple of MB. Making the index table more efficient to require less than 64 bits per chunk is actually one of the improvements I would like to do for the next major version.

Do we need more than some numpy for finding all the offsets?

It would be possible to get all chunk offsets just by reading the index table as Int64, but I think it will be less practical store all chunks again in Parquet. This is not only to size, but updates to open-data repository get more complicated. Most models update every 1-3 hours and rewrite a good amount of data, that would need to be reindexed again. You also would have to implement filtering and compression which are non-standard and not available as Python bindings.
What would require indexing is to combine multiple "om"-files into a longer time-series. Files are split either by:

Year. E.g. temperature_2m/year_2020.om
Time Chunk: temperature_2m/chunk_649723.om. Time chunks numbers are just unix timestamps divided by a length of 7*24*3600 = 604800 seconds. Lengths vary a bit from weather model to weather model.

Each file then contains one year or 7*24 hourly time-steps which need to be reassembled into a larger array. This steps would require some logic in kerchunk.

The chunk index lookup table can contain millions of chunks and requires a couple of MB. Making the index table more efficient to require less than 64 bits per chunk is actually one of the improvements I would like to do for the next major version.

Although kerchunk stores both offset and size for each chunk, the arrays are compressed and themselves chunked (size not connected to the number of chunks per file), so it's not inconceivable that parquet does better than the flat index in each file. Avoiding having to preemptively touch every file we will need at read time could be a big bonus.

You also would have to implement filtering and compression which are non-standard and not available as Python bindings.

Absolutely for the compression - I understand this is a totally custom algorithm.

Each file then contains one year or 7*24 hourly time-steps which need to be reassembled into a larger array. This steps would require some logic in kerchunk.

kerchunk can probably already do this, although whether you can simply use MultiZarrToZarr, I'm not yet certain.

Absolutely for the compression - I understand this is a totally custom algorithm.

I will do some tests starting in September to provide the necessary building blocks to interact with OM-files. I have received more and more interest from different parties to provide Python bindings and I think with some work, the OM format could be an interesting option to store more environmental data and make it accessible as cloud-native formats.

the OM format could be an interesting option to store more environmental data and make it accessible as cloud-native formats.

Sure, you are the expert!
I'd also be happy to talk in general about the direction.


Add property to make .Net 3.5 install optional

Signed-off-by: John Snow<EMAIL_ADDRESS>The .Net 3.5 install is not required for SQL 2016 but is required for all previous versions of SQL. Additional this PR fixes an issue when passing in an Integer for the version which is being validated in the helper library.
[Describe what this change achieves]
#115
[List any existing issues this PR resolves]
Check List

[ ] All tests pass. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/TESTING.MD
[x] New functionality includes testing.
[x] New functionality has been documented in the README if applicable
[x] All commits have been signed for the Developer Certificate of Origin. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/CONTRIBUTING.MD

nice


[Bug] DBT clone when partition or cluster has changed throws error

Is this a new bug in dbt-bigquery?

[X] I believe this is a new bug in dbt-bigquery
[X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior
Say I have a model that has partition at the day granularity in production. I update the model to use month for partition granularity, and successfully build the model in staging. In my production step, I proceed to clone the model to production using dbt clone. Bigquery will throw
Cannot replace a table with a different partitioning spec. Instead, DROP the table,
and then recreate it. New partitioning spec is interval(type:month,field:timestamp)
and existing spec is interval(type:day,field:timestamp)

Expected Behavior
When there's a partition or cluster change, we should drop the existing model first then clone the model with new partition / clustering.
Steps To Reproduce
Create a simple model
--- sample_model.sql
{{
    config(
        cluster_by = ['col_1', 'col_2'],
        partition_by = {
	        'field': 'timestamp',
	        'data_type': 'timestamp',
	        'granularity': 'day'
        },
    )
}}

select
    'col_1' as col_1,
    'col_2' as col_2,
    CURRENT_TIMESTAMP() as timestamp

run dbt run -s sample_model -t staging or some kind of dev / staging environment you have
and run dbt clone -s sample_model -t prod --full-refresh so the same model exists in prod
--- sample_model.sql
{{
    config(
        cluster_by = ['col_1'],
        partition_by = {
	        'field': 'timestamp',
	        'data_type': 'timestamp',
	        'granularity': 'month'
        },
    )
}}

select
    'col_1' as col_1,
    'col_2' as col_2,
    CURRENT_TIMESTAMP() as timestamp

now change the cluster and partition, then run
run dbt run -s sample_model -t staging or some kind of dev / staging environment you have
and run dbt clone -s sample_model -t prod so the same model exists in prod
you should see an error throw by bigquery telling you to drop the model first
Relevant log output
No response
Environment
- OS: Mac
- Python: 3.8.12
- dbt-core: 1.6.6
- dbt-bigquery: 1.6.7

Additional Context
similar to the behavior in building incremental models when there's a partition / clustering change
https://github.com/dbt-labs/dbt-core/blob/40839c79fcf1854942f8db44d1a2648bffe0a2ab/plugins/bigquery/dbt/include/bigquery/macros/materializations/incremental.sql#L23-L29
https://github.com/dbt-labs/dbt-core/issues/2000

Thanks for opening @SPTKL --- I'm glad you're finding the clone command useful.
My concern here is predominately user-experience-related, namely

when is --full-refresh required, and when isn't it?

Allow me to explain why I pick that as the framing question.
You propose the following as expected behavior:

When there's a partition or cluster change, we should drop the existing model first then clone the model with new partition / clustering.

I agree, but with an important caveat: only if the --full-refresh flag (or perhaps a new flag) is passed.
My rationale is that partitioning can have large a cost impact-- especially on large tables. If a user accidentally clones a dev version of a model with new partitioning to prod and it errors because the partitioning doesn't match -- this is a good thing!
If you look at the corresponding incremental code you shared, the full DROP in lieu of REPLACE will only happen in the context of --full-refresh
In some ways, the --full-refresh flag is a synonym for

I don't care about what's already in the destination, delete if you need!

What do you think? Let me know if I'm off base here

Thank you for the conversion here! I think we will need more information to formulate the right solution at this time. @dataders brings up a very good point: the cost of partitioning a new model because of an accidental rebuild is not a great experience. I'm also wary of how to ensure a seamless user experience with the full-refresh. Due to this, I am going to push this into something we will not be tackling on our roadmap.


How do I add scss to this structure?

Although I did a lot of research, I was unable to add scss. Can you help me?

@yunusunver
Sure.
Could you try the latest master?

thanks its worked , I want to build a structure like this;

But import './app.scss'  I don't want to use it like this. I want to see the scss file in the same folder automatically. Is this possible?


The logic for 'in' is incorrect

version: growthbook-c-sharp 1.0.0
When I was dealing with conditions involving 'in', I found that the results were not as expected.
I suspect that during the check for whether an element is contained in an array, the ToString method converted the array into a string type, and then it was processed using string methods.
see different:
https://github.com/growthbook/growthbook-c-sharp/blob/57b6b0b8e7acf842b390072198f6e124d39c8b7f/GrowthBook/Providers/ConditionEvaluationProvider.cs#L391
https://github.com/growthbook/growthbook-c-sharp/blob/70ea9f47a30334d8ea7a1ccd24fe8521abbfd89e/GrowthBook/Utilities.cs#L388
test code:
[TestMethod]
        public void TestFeature()
        {
            var featureString = """
                                {
                                  "defaultValue": false,
                                  "rules": [
                                    {
                                      "condition": {
                                        "userId": {
                                          "$in": [
                                            "ac1",
                                            "ac2",
                                            "ac3"
                                          ]
                                        }
                                      },
                                      "force": true
                                    }
                                  ]
                                }
                                """;
            var feature = JsonConvert.DeserializeObject<Feature>(featureString);
            var staticFeatures = new Dictionary<string, Feature>
            {
                {
                    "test-in-op", feature
                }
            };
            var context = new Context
            {
                Enabled = true,
                Url = "",
                Features = staticFeatures
            };

            var growthBook = new GrowthBook.GrowthBook(context);
            var flag = growthBook.IsOn("test-in-op");
            Assert.IsFalse(flag);

            // set userId
            context.Attributes.Add("userId", "");
            var errorFlag = growthBook.IsOn("test-in-op");
            Assert.IsFalse(errorFlag); // assert failed, it should be false
        }

@fivesmallq Good catch, thanks! It was correctly identifying both cases as not having an actual array value, but only did a null check on the value prior to comparison which meant that an empty string value would pass and would then be found within the subsequent string comparison. Should be fixed with the PR above.


Bump cryptography to 43.0.0

What and why?
Bumps cryptography to 43.0.0 to address a vulnerability in a lower version.
Bumps sdc-cryptography to 1.2.1 to address the vulnerability.
How to test?
Check everything still works after the update.
Jira
https://jira.ons.gov.uk/secure/RapidBoard.jspa?rapidView=1493&projectKey=RAS&view=detail&selectedIssue=RAS-1206

/deploy wilkia


🚀 [Feature] :  Update beta version in readme

Is your feature request related to a problem? Please describe.
I would prefer to see exact version from the get-go in the readme for efficiency without having to go deeper just to ensure the actual version.
Describe the solution you'd like
Currently the default is a general version with this format: Beta phase v0.X.x. I volunteer to keep this up-to-date accordingly for the readme i.e. Beta phase v0.5.17.
Describe alternatives you've considered
There are automation tools for this purpose. One such tool is Renovatebot, which is a universal dependency update tool that can be configured to update README file as part of its automation process. Another option is readme-ai, an automated README file generator that can create and update README files using data extraction and generative AI. Though I haven't had the chance to use them yet. Will research more on them in the future.
Additional context
This serves for better clarity in documentation.
If you want speed up process of this issue, please send 🚀 as a reaction.

@behics can you add the label "documentation" for this and assign me for it? I'll do it asap. Cheers 🤝

[ ] Mention it in Table Of Contents if you create new Title (header) inside of README

@behics can we close this now as the PR is already completed? Thanks 👍🏻

@behics can we close this now as the PR is already completed? Thanks 👍🏻

Thank you ! :)


Non-symetric topics

There isn't currently an exposed interface differentiating the owner of a topic who has the needed keys to publish to that topic, and subscribers who can read new values but aren't able to publish. Is that a distinction that is desired?

If i remember correctly, one proposal for this is that the topic ID is an ecc public key, and writes must be signed by that key to be accepted.
The trick is that the server doesn't know the topic ID in order to verify that a write is signed by the right key.
Is this a zero-knowledge proof situation?

Seems reasonable. Just standard public key encryption stuff, I don't think we need zkp

the worry is that a client can launch DOS attacks by writing into the 'next' hash value of a topic at each era. Other clients know they haven't seen a real value, but the cost without the client having to do computation to show that it needs to perform those evictions seems potentially low.
Starting with standard PKI seems fine, but we should think about how we plan to mitigate that sort of attack.

57f39b9798dacc84c13fd04c38cbf6ac971b23af differentiates a topic writer from the subscription reader.
There isn't yet a way to prove to the server that a given write should be allowed. Ideally you'd like to prove the previous work to somehow convince the server that writing at the two buckets you ask for is legit, and not someone trying to ddos given cells

I think the observation is correct, a malicious subscriber with a topic handle can currently:

Publish to the next bucket on the log trail (DoS buckets)
Submit false flag interest vectors (DoS Interest vectors)

As of now, subscribers must be trusted to not do these things. That's part of the threat model in the paper. It would be nice to be able to come up with a more general solution that protects us from malicious subscribers and malicious servers colluding. Is worth adding complexity to just address malicious subscribers that don't collude with the servers?

Thinking more about this, i think the argument here is that creation of a new log should be computationally hard, in that the initial seed / buckets should come as the result of a hash problem with sufficient difficulty. This, with an online component, prevents easy brute-force of a new topic write clobbering a targeted bucket.
That doesn't work for advancing the seed, but for that case you'd instead like to have a proof that this is a subsequent write that uses the same write key as has been used before.
Maybe there's a zero-knowledge circuit or verifiable computing technique that can be leverage for this. Maybe with a PIS -style anonymization the write can point to a previous bucket as its successor without linking them to the server.


feat: custom serialization

Serialization using serde is suboptimal. Depending on the encoding used, serialization may not be canonical, and is likely to be inefficient due to length encoding.
This PR adds custom serialization and deserialization that is efficient and canonical.
Because this is easy to mess up, it also adds a basic fuzzer. The fuzzer attempts to deserialize byte slices; deserialization should either fail or subsequently serialize canonically.
Note that serde-based serialization is still supported.
Closes #43.

We can probably remove the n - 1 encoding and infer it from the proof to save four bytes.


Update 02-getting-started.md

Fixing typos and some cleanup.

[x] Changes have been thoroughly tested to not break existing functionality.
[x] New functionality has been documented or existing documentation has been updated to reflect changes.
[x] Visual changes are explained in the PR description using a screenshot/recording of before and after.

Thanks Rich!


Research feather for data frame storage

https://github.com/wesm/feather

About feather:

Feather was designed with the main purpose of providing a fast and efficient way to store and exchange data between different programming languages, particularly Python and R. It aims to minimize serialization and deserialization overhead, making data interchange between these languages more seamless and performant. Feather is optimized for data frames and is particularly useful in scenarios where you need to share large datasets between different data analysis or processing tools.

Example usage:
import pandas as pd
import pyarrow.feather as feather

df = pd.read_csv("./input/very_big_file.csv")
feather.write_feather(df, "./output/very_big_file.feather")
read_df = feather.read_feather("./output/very_big_file.feather")
print(read_df)

My notes:

the .feather file is not readable by users because it's in binary format
it does decrease size of the file (in our case from 4082KB to 2746KB)
it might be considered as a way to save storage
in cashflower context, there is no need to exchange data with R, the .csv file are sufficient
the users of cashflower may want to save the output to .feather - if they want to - which is possible for them


Set up Microsoft 365 for Business

Definition of done:

Purchase is made
Outlook Calendar is set up
Email is set up in Outlook using new (Google Domain)
Confirmed access to shared OneDrive folder and OneNote document(s)

Once basic set-up is complete, will need to organise it as a team.

Dependency: company name and Google Domain
Pushed to Sprint 2

Using Zelle for shared purchases.

Already set up. Need to change name & email to include sunbird web domain. Set up Outlook on our personal devices.
Moved to Sprint 3


fix(scripts_test): use /var/lib/otelcol-sumo as home directory

Signed-off-by: Dominik Rosiek<EMAIL_ADDRESS>

-m is for creating directory


"Key :source not found" error from 0.14.3 to 0.14.5 version

We are now getting this error:

key :source not found in: [engine: Phoenix.LiveView.HTMLEngine, caller: #Macro.Env<aliases: [{LiveViewEngine, Phoenix.LiveView.Engine}, {HTMLEngine, Phoenix.LiveView.HTMLEngine}, {RenderingContext, PhxLiveStorybook.Rendering.RenderingContext}, {RenderingVariation, PhxLiveStorybook.Rendering.RenderingVariation}, {TemplateHelpers, PhxLiveStorybook.TemplateHelpers}], context: nil, context_modules: [PhxLiveStorybook.Rendering.ComponentRenderer], file: "/Users/.../Projects/.../elixir/deps/phx_live_storybook/lib/phx_live_storybook/rendering/component_renderer.ex", function: {:render_component_heex, 3}, functions: [{Kernel, [!=: 2, !==: 2, *: 2, **: 2, +: 1, +: 2, ++: 2, -: 1, -: 2, --: 2, /: 2, <: 2, <=: 2, ==: 2, ===: 2, =~: 2, >: 2, >=: 2, abs: 1, apply: 2, apply: 3, binary_part: 3, binary_slice: 2, binary_slice: 3, bit_size: 1, byte_size: 1, ceil: 1, div: 2, elem: 2, exit: 1, floor: 1, function_exported?: 3, get_and_update_in: 3, get_in: 2, hd: 1, inspect: 1, inspect: 2, is_atom: 1, is_binary: 1, ...]}], lexical_tracker: nil, line: 137, macro_aliases: [], macros: [{Kernel, [!: 1, &&: 2, ..: 0, ..: 2, ..//: 3, <>: 2, @: 1, alias!: 1, and: 2, binding: 0, binding: 1, dbg: 0, dbg: 1, dbg: 2, def: 1, def: 2, defdelegate: 2, defexception: 1, defguard: 1, defguardp: 1, defimpl: 2, defimpl: 3, defmacro: 1, defmacro: 2, defmacrop: 1, defmacrop: 2, defmodule: 2, defoverridable: 1, defp: 1, defp: 2, defprotocol: 2, defstruct: 1, destructure: 2, get_and_update_in: 2, if: 2, ...]}], module: PhxLiveStorybook.Rendering.ComponentRenderer, requires: [Application, Kernel, Kernel.Typespec], ...>]

Elixir version:  1.14.3-otp-25
Erlang version: 25.2.1
Node version: v18.11.0
phx_live_storybook versions: We tried the 0.14.3 to 0.14.5 versions but had no luck
Actions taken:

Remove _build and _deps folder
Re-running mix deps.get and mix phx.server

I'm having this same issue on version 0.4.4

This has been fixed in #221 (you have to pick the main branch though)
0.5.0 should be released really soon, one last ticket to finish


Undefined symbol _ruby_abi_version regression with Xcode 13 -> 14

What version of gRPC and what language are you using?
grpc: 1.48.0
language: ruby 2.7.6
What operating system (Linux, Windows,...) and version?
MacOS 12.6
Apple M1 chip
What runtime / compiler are you using (e.g. python version or version of gcc)
Xcode 14
What did you do?
gem install grpc -v 1.48.0
What did you expect to see?
Successfully install
What did you see instead?
Undefined symbols for architecture arm64:
  "_ruby_abi_version", referenced from:
     -exported_symbol[s_list] command line option
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [grpc_c.bundle] Error 1

Anything else we should know about your project / environment?
Worked correctly a few days ago with Xcode 13

That constant was added in https://github.com/grpc/grpc/pull/28976 . Is there a way to make this conditional on the ruby version if it's only required for Ruby 3.2+?

@ethankhall found a workaround for this:
export DLDFLAGS=-Wl,-undefined,dynamic_lookup
# however you are installing... gem directly
gem install grpc
# or ...
bundle install

cc @ngan @noizwaves

@technicalpickles I have the same issue and it is blocking me to do my work. The provided workaround is not working for me as well. Could we give this issue a higher priority as the project cannot build and I do not know what to do?

Same issue with

ruby 2.7.6p219 (2022-04-12 revision c9c2245c0a) [arm64-darwin21]
google-protobuf-3.21.6
grpc-1.48.0

Log
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: warning same member name (versioning.upb.o) in output file used for input files: /Users/user/.rvm/gems/ruby-2.7.6/gems/grpc-1.48.0/src/ruby/ext/grpc/objs/opt/src/core/ext/upb-generated/xds/annotations/v3/versioning.upb.o and: /Users/user/.rvm/gems/ruby-2.7.6/gems/grpc-1.48.0/src/ruby/ext/grpc/objs/opt/src/core/ext/upb-generated/udpa/annotations/versioning.upb.o (due to use of basename, truncation, blank padding or duplicate input files)
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: warning same member name (versioning.upbdefs.o) in output file used for input files: /Users/user/.rvm/gems/ruby-2.7.6/gems/grpc-1.48.0/src/ruby/ext/grpc/objs/opt/src/core/ext/upbdefs-generated/udpa/annotations/versioning.upbdefs.o and: /Users/user/.rvm/gems/ruby-2.7.6/gems/grpc-1.48.0/src/ruby/ext/grpc/objs/opt/src/core/ext/upbdefs-generated/xds/annotations/v3/versioning.upbdefs.o (due to use of basename, truncation, blank padding or duplicate input files)
:  /Users/user/.rvm/gems/ruby-2.7.6/gems/grpc-1.48.0/src/ruby/ext/grpc/libs/opt/libgrpc.a
Generating Makefile for grpc/grpc_c
creating Makefile

current directory: /Users/user/.rvm/gems/ruby-2.7.6/gems/grpc-1.48.0/src/ruby/ext/grpc
make "DESTDIR=" clean

current directory: /Users/user/.rvm/gems/ruby-2.7.6/gems/grpc-1.48.0/src/ruby/ext/grpc
make "DESTDIR="
compiling rb_byte_buffer.c
In file included from rb_byte_buffer.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2523:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)<0,"bad scan arg format","error")))
                   ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2524:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)!=varc,"variable argument length doesn't match","error")))
                   ^
2 warnings generated.
compiling rb_call.c
In file included from rb_call.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2523:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)<0,"bad scan arg format","error")))
                   ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2524:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)!=varc,"variable argument length doesn't match","error")))
                   ^
rb_call.c:111:13: warning: excess elements in scalar initializer [-Wexcess-initializers]
     {NULL, NULL}},
            ^~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/14.0.0/include/stddef.h:89:16: note: expanded from macro 'NULL'
#  define NULL ((void*)0)
               ^~~~~~~~~~
rb_call.c:111:18: warning: missing field 'reserved' initializer [-Wmissing-field-initializers]
     {NULL, NULL}},
                 ^
rb_call.c:129:60: warning: excess elements in scalar initializer [-Wexcess-initializers]
                                                    {NULL, NULL}},
                                                           ^~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/14.0.0/include/stddef.h:89:16: note: expanded from macro 'NULL'
#  define NULL ((void*)0)
               ^~~~~~~~~~
rb_call.c:129:65: warning: missing field 'reserved' initializer [-Wmissing-field-initializers]
                                                    {NULL, NULL}},
                                                                ^
6 warnings generated.
compiling rb_call_credentials.c
In file included from rb_call_credentials.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2523:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)<0,"bad scan arg format","error")))
                   ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2524:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)!=varc,"variable argument length doesn't match","error")))
                   ^
rb_call_credentials.c:63:77: warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
    VALUE callback_func_str = rb_funcall(callback_func, rb_intern("to_s"), 0);
                                                                            ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2685:10: note: macro 'rb_funcall' defined here
# define rb_funcall(recv, mid, argc, ...) \
         ^
rb_call_credentials.c:64:77: warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
    VALUE callback_args_str = rb_funcall(callback_args, rb_intern("to_s"), 0);
                                                                            ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2685:10: note: macro 'rb_funcall' defined here
# define rb_funcall(recv, mid, argc, ...) \
         ^
rb_call_credentials.c:66:66: warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
        rb_funcall(callback_func, rb_intern("source_location"), 0);
                                                                 ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2685:10: note: macro 'rb_funcall' defined here
# define rb_funcall(recv, mid, argc, ...) \
         ^
rb_call_credentials.c:70:70: warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
          rb_ary_entry(callback_source_info, 1), rb_intern("to_s"), 0);
                                                                     ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2685:10: note: macro 'rb_funcall' defined here
# define rb_funcall(recv, mid, argc, ...) \
         ^
rb_call_credentials.c:101:75: warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
  VALUE backtrace = rb_funcall(exception_object, rb_intern("backtrace"), 0);
                                                                          ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2685:10: note: macro 'rb_funcall' defined here
# define rb_funcall(recv, mid, argc, ...) \
         ^
rb_call_credentials.c:112:59: warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
      rb_funcall(exception_object, rb_intern("inspect"), 0);
                                                          ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2685:10: note: macro 'rb_funcall' defined here
# define rb_funcall(recv, mid, argc, ...) \
         ^
rb_call_credentials.c:163:19: warning: unused parameter 'creds_md' [-Wunused-parameter]
    grpc_metadata creds_md[GRPC_METADATA_CREDENTIALS_PLUGIN_SYNC_MAX],
                  ^
rb_call_credentials.c:164:13: warning: unused parameter 'num_creds_md' [-Wunused-parameter]
    size_t* num_creds_md, grpc_status_code* status,
            ^
rb_call_credentials.c:164:45: warning: unused parameter 'status' [-Wunused-parameter]
    size_t* num_creds_md, grpc_status_code* status,
                                            ^
rb_call_credentials.c:165:18: warning: unused parameter 'error_details' [-Wunused-parameter]
    const char** error_details) {
                 ^
rb_call_credentials.c:216:13: warning: excess elements in scalar initializer [-Wexcess-initializers]
     {NULL, NULL}},
            ^~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/14.0.0/include/stddef.h:89:16: note: expanded from macro 'NULL'
#  define NULL ((void*)0)
               ^~~~~~~~~~
rb_call_credentials.c:216:18: warning: missing field 'reserved' initializer [-Wmissing-field-initializers]
     {NULL, NULL}},
                 ^
14 warnings generated.
compiling rb_channel.c
In file included from rb_channel.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2523:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)<0,"bad scan arg format","error")))
                   ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2524:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)!=varc,"variable argument length doesn't match","error")))
                   ^
rb_channel.c:188:57: warning: excess elements in scalar initializer [-Wexcess-initializers]
                                                 {NULL, NULL}},
                                                        ^~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/14.0.0/include/stddef.h:89:16: note: expanded from macro 'NULL'
#  define NULL ((void*)0)
               ^~~~~~~~~~
rb_channel.c:188:62: warning: missing field 'reserved' initializer [-Wmissing-field-initializers]
                                                 {NULL, NULL}},
                                                             ^
In file included from rb_channel.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2345:37: warning: token pasting of ',' and __VA_ARGS__ is a GNU extension [-Wgnu-zero-variadic-macro-arguments]
        rb_scan_args(argc,argvp,fmt,##__VA_ARGS__))
                                    ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2345:37: warning: token pasting of ',' and __VA_ARGS__ is a GNU extension [-Wgnu-zero-variadic-macro-arguments]
rb_channel.c:764:40: warning: incompatible function pointer types passing 'VALUE (VALUE)' (aka 'unsigned long (unsigned long)') to parameter of type 'VALUE (*)(void *)' (aka 'unsigned long (*)(void *)') [-Wincompatible-function-pointer-types]
  background_thread = rb_thread_create(run_poll_channels_loop, NULL);
                                       ^~~~~~~~~~~~~~~~~~~~~~
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/intern.h:501:32: note: passing argument to parameter here
VALUE rb_thread_create(VALUE (*)(void *), void*);
                               ^
7 warnings generated.
compiling rb_channel_args.c
In file included from rb_channel_args.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2523:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)<0,"bad scan arg format","error")))
                   ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2524:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)!=varc,"variable argument length doesn't match","error")))
                   ^
rb_channel_args.c:33:13: warning: excess elements in scalar initializer [-Wexcess-initializers]
     {NULL, NULL}},
            ^~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/14.0.0/include/stddef.h:89:16: note: expanded from macro 'NULL'
#  define NULL ((void*)0)
               ^~~~~~~~~~
rb_channel_args.c:33:18: warning: missing field 'reserved' initializer [-Wmissing-field-initializers]
     {NULL, NULL}},
                 ^
rb_channel_args.c:124:63: warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
    num_args = NUM2INT(rb_funcall(params->src_hash, id_size, 0));
                                                              ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2685:10: note: macro 'rb_funcall' defined here
# define rb_funcall(recv, mid, argc, ...) \
         ^
5 warnings generated.
compiling rb_channel_credentials.c
In file included from rb_channel_credentials.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2523:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)<0,"bad scan arg format","error")))
                   ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2524:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)!=varc,"variable argument length doesn't match","error")))
                   ^
rb_channel_credentials.c:87:13: warning: excess elements in scalar initializer [-Wexcess-initializers]
     {NULL, NULL}},
            ^~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/14.0.0/include/stddef.h:89:16: note: expanded from macro 'NULL'
#  define NULL ((void*)0)
               ^~~~~~~~~~
rb_channel_credentials.c:87:18: warning: missing field 'reserved' initializer [-Wmissing-field-initializers]
     {NULL, NULL}},
                 ^
In file included from rb_channel_credentials.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2345:37: warning: token pasting of ',' and __VA_ARGS__ is a GNU extension [-Wgnu-zero-variadic-macro-arguments]
        rb_scan_args(argc,argvp,fmt,##__VA_ARGS__))
                                    ^
5 warnings generated.
compiling rb_completion_queue.c
In file included from rb_completion_queue.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2523:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)<0,"bad scan arg format","error")))
                   ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2524:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)!=varc,"variable argument length doesn't match","error")))
                   ^
2 warnings generated.
compiling rb_compression_options.c
In file included from rb_compression_options.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2523:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)<0,"bad scan arg format","error")))
                   ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2524:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)!=varc,"variable argument length doesn't match","error")))
                   ^
rb_compression_options.c:82:13: warning: excess elements in scalar initializer [-Wexcess-initializers]
     {NULL, NULL}},
            ^~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/14.0.0/include/stddef.h:89:16: note: expanded from macro 'NULL'
#  define NULL ((void*)0)
               ^~~~~~~~~~
rb_compression_options.c:82:18: warning: missing field 'reserved' initializer [-Wmissing-field-initializers]
     {NULL, NULL}},
                 ^
rb_compression_options.c:172:77: warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
  algorithm_name_as_string = rb_funcall(algorithm_name, rb_intern("to_s"), 0);
                                                                            ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2685:10: note: macro 'rb_funcall' defined here
# define rb_funcall(recv, mid, argc, ...) \
         ^
rb_compression_options.c:302:56: warning: passing 'char **' to parameter of type 'const char **' discards qualifiers in nested pointer types [-Wincompatible-pointer-types-discards-qualifiers]
  if (!grpc_compression_algorithm_name(internal_value, &algorithm_name)) {
                                                       ^~~~~~~~~~~~~~~
/Users/user/.rvm/gems/ruby-2.7.6/gems/grpc-1.48.0/include/grpc/compression.h:50:56: note: passing argument to parameter 'name' here
    grpc_compression_algorithm algorithm, const char** name);
                                                       ^
In file included from rb_compression_options.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2345:37: warning: token pasting of ',' and __VA_ARGS__ is a GNU extension [-Wgnu-zero-variadic-macro-arguments]
        rb_scan_args(argc,argvp,fmt,##__VA_ARGS__))
                                    ^
7 warnings generated.
compiling rb_enable_cpp.cc
compiling rb_event_thread.c
In file included from rb_event_thread.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2523:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)<0,"bad scan arg format","error")))
                   ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2524:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)!=varc,"variable argument length doesn't match","error")))
                   ^
rb_event_thread.c:144:20: warning: incompatible function pointer types passing 'VALUE (VALUE)' (aka 'unsigned long (unsigned long)') to parameter of type 'VALUE (*)(void *)' (aka 'unsigned long (*)(void *)') [-Wincompatible-function-pointer-types]
  rb_thread_create(grpc_rb_event_thread, NULL);
                   ^~~~~~~~~~~~~~~~~~~~
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/intern.h:501:32: note: passing argument to parameter here
VALUE rb_thread_create(VALUE (*)(void *), void*);
                               ^
3 warnings generated.
compiling rb_grpc.c
In file included from rb_grpc.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2523:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)<0,"bad scan arg format","error")))
                   ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2524:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)!=varc,"variable argument length doesn't match","error")))
                   ^
rb_grpc.c:54:13: warning: excess elements in scalar initializer [-Wexcess-initializers]
     {NULL, NULL}},
            ^~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/14.0.0/include/stddef.h:89:16: note: expanded from macro 'NULL'
#  define NULL ((void*)0)
               ^~~~~~~~~~
rb_grpc.c:54:18: warning: missing field 'reserved' initializer [-Wmissing-field-initializers]
     {NULL, NULL}},
                 ^
rb_grpc.c:112:57: warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
        t.tv_sec = NUM2INT(rb_funcall(time, id_tv_sec, 0));
                                                        ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2685:10: note: macro 'rb_funcall' defined here
# define rb_funcall(recv, mid, argc, ...) \
         ^
rb_grpc.c:113:59: warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
        t.tv_nsec = NUM2INT(rb_funcall(time, id_tv_nsec, 0));
                                                          ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2685:10: note: macro 'rb_funcall' defined here
# define rb_funcall(recv, mid, argc, ...) \
         ^
rb_grpc.c:183:66: warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
  return rb_funcall(grpc_rb_time_val_to_time(self), id_inspect, 0);
                                                                 ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2685:10: note: macro 'rb_funcall' defined here
# define rb_funcall(recv, mid, argc, ...) \
         ^
rb_grpc.c:188:63: warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
  return rb_funcall(grpc_rb_time_val_to_time(self), id_to_s, 0);
                                                              ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2685:10: note: macro 'rb_funcall' defined here
# define rb_funcall(recv, mid, argc, ...) \
         ^
8 warnings generated.
compiling rb_grpc_imports.generated.c
compiling rb_loader.c
compiling rb_server.c
In file included from rb_server.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2523:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)<0,"bad scan arg format","error")))
                   ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2524:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)!=varc,"variable argument length doesn't match","error")))
                   ^
rb_server.c:119:13: warning: excess elements in scalar initializer [-Wexcess-initializers]
     {NULL, NULL}},
            ^~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/14.0.0/include/stddef.h:89:16: note: expanded from macro 'NULL'
#  define NULL ((void*)0)
               ^~~~~~~~~~
rb_server.c:119:18: warning: missing field 'reserved' initializer [-Wmissing-field-initializers]
     {NULL, NULL}},
                 ^
4 warnings generated.
compiling rb_server_credentials.c
In file included from rb_server_credentials.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2523:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)<0,"bad scan arg format","error")))
                   ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2524:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)!=varc,"variable argument length doesn't match","error")))
                   ^
rb_server_credentials.c:87:13: warning: excess elements in scalar initializer [-Wexcess-initializers]
     {NULL, NULL}},
            ^~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/14.0.0/include/stddef.h:89:16: note: expanded from macro 'NULL'
#  define NULL ((void*)0)
               ^~~~~~~~~~
rb_server_credentials.c:87:18: warning: missing field 'reserved' initializer [-Wmissing-field-initializers]
     {NULL, NULL}},
                 ^
4 warnings generated.
compiling rb_xds_channel_credentials.c
In file included from rb_xds_channel_credentials.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2523:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)<0,"bad scan arg format","error")))
                   ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2524:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)!=varc,"variable argument length doesn't match","error")))
                   ^
rb_xds_channel_credentials.c:84:39: warning: missing field 'reserved' initializer [-Wmissing-field-initializers]
     GRPC_RB_MEMSIZE_UNAVAILABLE, NULL},
                                      ^
3 warnings generated.
compiling rb_xds_server_credentials.c
In file included from rb_xds_server_credentials.c:19:
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2523:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)<0,"bad scan arg format","error")))
                   ^
/Users/user/.rvm/rubies/ruby-2.7.6/include/ruby-2.7.0/ruby/ruby.h:2524:20: warning: 'diagnose_if' is a clang extension [-Wgcc-compat]
    __attribute__((diagnose_if(rb_scan_args_count(fmt)!=varc,"variable argument length doesn't match","error")))
                   ^
rb_xds_server_credentials.c:85:39: warning: missing field 'reserved' initializer [-Wmissing-field-initializers]
     GRPC_RB_MEMSIZE_UNAVAILABLE, NULL},
                                      ^
3 warnings generated.
linking shared-object grpc/grpc_c.bundle
Undefined symbols for architecture arm64:
  "_ruby_abi_version", referenced from:
     -exported_symbol[s_list] command line option
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [grpc_c.bundle] Error 1

make failed, exit code 2

Gem files will remain installed in /Users/user/.rvm/gems/ruby-2.7.6/gems/grpc-1.48.0 for inspection.
Results logged to /Users/user/.rvm/gems/ruby-2.7.6/extensions/arm64-darwin-21/2.7.0/grpc-1.48.0/gem_make.out

I had the same issue but with Ruby 3.0.4. After trying everything else, I fixed it by reinstalling Ruby:
rbenv uninstall 3.0.4
rbenv install 3.0.4

Just to follow up, the workaround didn’t work for us and we ended up downgrading grpc to 1.470

Having same issue.

Same issue with M1 Pro Apple Silicon (arm64) but on ruby 3.1.0 . Uninstalling and installing again didnt solve

Just to follow up, the workaround didn’t work for us and we ended up downgrading grpc to 1.470

Downgrading grpc to 1.470 was the fix for me too!
Thanks

Downgrading grpc to 1.470 was the fix for me too!
Thanks

Unfortunately, this is not a viable solution as another dependency is relying on 1.48.0
Has anyone had any luck?

https://github.com/grpc/grpc/pull/31151 fixes this (not merged yet).

This also fails for me on a M2 chip (which makes sense).
The lines from the bugfix above are present in the extconf.rb file for v. 1.48.0, here: https://github.com/grpc/grpc/blob/d2054ec6c6e8abcecf0e24b0b4ee75035d80c3cc/src/ruby/ext/grpc/extconf.rb#L114-L117
so not sure that that is the fix.

#31151 fixes this (not merged yet).

This also fails for me on a M2 chip (which makes sense).
The lines from the bugfix above are present in the extconf.rb file for v. 1.48.0, here:
grpc/src/ruby/ext/grpc/extconf.rb
Lines 114 to 117 in d2054ec
ext_export_file = File.join(grpc_root, 'src', 'ruby', 'ext', 'grpc', 'ext-export')
ext_export_file += '-truffleruby' if RUBY_ENGINE == 'truffleruby'
$LDFLAGS << ' -Wl,--version-script="' + ext_export_file + '.gcc"' if linux
$LDFLAGS << ' -Wl,-exported_symbols_list,"' + ext_export_file + '.clang"' if apple_toolchain
so not sure that that is the fix.

The lines from the bugfix above are present in the extconf.rb file for v. 1.48.0

@weyus that's because the bugfix is removing these lines (not adding them).

@eduardojunio Oh geez. Sorry. 🤦

I'm still getting the same make error after running:
export DLDFLAGS=-Wl,-undefined,dynamic_lookup
gem install grpc -v 1.48.0

I have been trying this workaround and it helped, I just needed to run it in the following sequence, so the grpc installation runs on clean environment I guess.
rbenv uninstall 3.0.4
rbenv install 3.0.4
export DLDFLAGS=-Wl,-undefined,dynamic_lookup
gem install grpc -v 1.50.0
bundle install

Are there any configuration options for grpc to pass in these flags? If so, we'd be able to do something like this locally to make sure it always installs with the flag:
bundle config build.grpc --some-flag-here

I haven't found a way to set environment variables this way though.

Related upstream Ruby bug report: https://bugs.ruby-lang.org/issues/19005

As mentioned in the Ruby issue, there are several possible workarounds until Ruby releases the patched versions for this:

Recompile Ruby with DLDFLAGS="-Wl,-undefined,dynamic_lookup" set in the enviornment. RbConfig::CONFIG['DLDFLAGS'] should show these flags:

irb(main):001:0> RbConfig::CONFIG['DLDFLAGS']
=> "-L/Users/stanhu/.asdf/installs/ruby/3.0.2/lib<EMAIL_ADDRESS>-Wl,-undefined,dynamic_lookup -Wl,-multiply_defined,suppress"

With gem install:

gem install grpc -v 1.48.0 -- --with-ldflags="-Wl,-undefined,dynamic_lookup"

Set bundle config:

bundle config build.grpc --with-ldflags="-Wl,-undefined,dynamic_lookup"

Explanation
As I mentioned in the Ruby bug report, XCode 14.0 now warns with the message when -Wl,undefined,dynamic_lookup is used:
% gcc -o test -Wl,-undefined,dynamic_lookup test.c
ld: warning: -undefined dynamic_lookup may not work with chained fixups

This warning was enough to make the Ruby configure script believe that those flags were no longer available on the system. -Wl,undefined,dynamic_lookup has long been the default everywhere, and most gems were not tested without those flags.
I believe the patches in https://github.com/ruby/ruby/pull/6441/files fix the configure script in Ruby to enable this linker option despite this warning message, which is why the latest 3.2.0 preview release works fine. The patches have been backported to 2.7 and up, but there is no official released version yet.

https://github.com/grpc/grpc/pull/31970 looks like it should fix the underlying symbol issue in gRPC. I should mention that Ruby 2.7.7, 3.0.5, and 3.1.3 should work without any special build flags.

3.1.3 still threw the error for me without the flags.

Yeah, 3.1.3, 3.2.0, and other versions still need the flags. Apparently the fix accidentally worked in 2.7.7 and 3.0.5. See my note in https://bugs.ruby-lang.org/issues/19005#note-25.
I proposed an upstream fix in https://github.com/ruby/ruby/pull/7086.


Add an option to select a specific collectible reward level

Some items have a maximum reward level and will not give any better reward with more work. This could lead to shorter macros.
Example: https://ffxivteamcraft.com/db/en/item/31946/Grade-4-Skybuilders'-Oven
This one caps the reward at 1000 collectability (or 10000 quality) when the item has almost 12000 quality points max. The solver should try to solve for 10000 quality instead to optimize for speed.
The quality thresholds are relatively easy to get from xivapi datasets, let me know if you need any help with this.
An option could allow to try to find macro for a specific reward (useful when leveling and you know you can't reach the max reward anyway), max reward or max quality on collectibles.

If you could point me to the table/columns to look at, that would be great!
This has been on my todo list for a while and I should already have most of the groundwork done in the frontend + simulator to accept "target quality" values, I just need to update the craftingway_data repo to pipe the thresholds in.

So, 2 ways:

Check HWDCrafterSupply table and check if you find the item in the first part of the table, save the column index. Then you will have columns Base/Mid/HighCollectableRating with the value you need at the matching index in the same table.
If it's not from Ishgard, then it's in the CollectablesShopItem table, you'll have the object ID in there and the column CollectablesShopRefine pointing to an index in the table of the same name. That table has a Low/Mid/HighCollectability column for this. 0 in the last columns means there are less ranks (for example all the new crafting relic steps that have only low and mid for best reward).
Let me know if you have more questions!


Fixes #4.  Fix DST start/end month.  Calculate DST from ordinal week number.

Is that docstring intended for day_from_week_and_weekday() or for
weekday_in_month() ?
The examples in the docstring imply the former, but that function is not my
code.
On Sat, Sep 30, 2023 at 10:48 AM ekondayan @.***> wrote:

@.**** requested changes on this pull request.
Formatting: on lines 877 and 878 would you put spaces around the + and -
operators
Docstring: would you change the docstring to this:
"""Calculate and return the day of the month for the Nth ordinal week
occurrence of the specified weekday
within a given month and year. If there are fewer occurrences of the
specified weekday in the month,
the function returns the day of the last occurrence of the specified
weekday. For instance, if you are
looking for the second Tuesday of a month, "second" is the ordinal week
representing the occurrence of
the weekday "Tuesday," and you would use 2 as the value for the ordinal
parameter in the function.
Example:
day_from_week_and_weekday(2021, Ntp.MONTH_MAR, Ntp.WEEK_LAST,
Ntp.WEEKDAY_SUN)
day_from_week_and_weekday(2021, Ntp.MONTH_OCT, Ntp.WEEK_LAST,
Ntp.WEEKDAY_SUN)
Args:
    year (int): The year for which the calculation is to be made, must be an integer greater than 1.
    month (int): The month for which the calculation is to be made, must be an integer in the range 1(Jan) - 12(Dec).
    ordinal_week (int): Represents the ordinal week in the specified month, must be an integer in the range 1-6.
    weekday (int): Represents the specific weekday, must be an integer in the range 0(Mon)-6(Sun).

Returns:
    int: The day of the month of the Nth ordinal week occurrence of the specified weekday. If the ordinal specified is greater
         than the total occurrences of the weekday in that month, it returns the day of the last occurrence of the specified weekday.

Raises:
    ValueError: If any of the parameters are of incorrect type or out of the valid range.
"""

—
Reply to this email directly, view it on GitHub
https://github.com/ekondayan/micropython-ntp/pull/6#pullrequestreview-1651730043,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABHMNQRD4JA67MDLYDZE3VTX5BLPBANCNFSM6AAAAAA4RAFXK4
.
You are receiving this because you authored the thread.Message ID:
@.***>

Sorry for the oversight, I took the example section from the "day_from_week_and_weekday" and forgot to change the name.  Everything else is related to your function("weekday_in_month"). If you see something inaccurate feel free to edit it as you wish.
weekday_in_month.txt
One more thing, would you rename the "ordinal" argument to "ordinal_week"
Thank you.

"ordinal_week" isn't the correct meaning for that parameter, this was part
of the problem in the day_from_week_and_weekday() code that I was fixing.
It's the Nth weekday in that month, so maybe ordinal_weekday?
On Sat, Oct 7, 2023 at 10:33 PM ekondayan @.***> wrote:

Sorry for the oversight, I took the example section from the
"day_from_week_and_weekday" and forgot to change the name. Everything else
is related to your function("weekday_in_month"). If you see something
inaccurate feel free to edit it as you wish.
weekday_in_month.txt
https://github.com/ekondayan/micropython-ntp/files/12839849/weekday_in_month.txt
One more thing, would you rename the "ordinal" argument to "ordinal_week"
Thank you.
—
Reply to this email directly, view it on GitHub
https://github.com/ekondayan/micropython-ntp/pull/6#issuecomment-1751927136,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABHMNQVXSZAHD4KOLD7VY7TX6I3JLAVCNFSM6AAAAAA4RAFXK6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJRHEZDOMJTGY
.
You are receiving this because you authored the thread.Message ID:
@.***>

Yes, it is better

Should WEEK_FORTH be WEEK_FOURTH?
On Sun, Oct 8, 2023 at 7:09 AM ekondayan @.***> wrote:

Yes, it is better
—
Reply to this email directly, view it on GitHub
https://github.com/ekondayan/micropython-ntp/pull/6#issuecomment-1752037676,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABHMNQRMWI4E7IDAE3BIWDTX6KXZJAVCNFSM6AAAAAA4RAFXK6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJSGAZTONRXGY
.
You are receiving this because you authored the thread.Message ID:
@.***>

Yes

Amended and pushed.
On Fri, Oct 13, 2023 at 8:50 AM ekondayan @.***> wrote:

Yes
—
Reply to this email directly, view it on GitHub
https://github.com/ekondayan/micropython-ntp/pull/6#issuecomment-1761737132,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABHMNQTLEND4X5HLUIEIK7DX7FPMHANCNFSM6AAAAAA4RAFXK4
.
You are receiving this because you authored the thread.Message ID:
@.***>

Thanks for the contribution. One last thing before I merge the commit.
Would you explain in the commit description why "Calculate DST from ordinal week number" is introduced and why is it needed.

Not sure I understand what you're asking me to explain.  "Calculate DST
from ordinal week number" is what day_from_week_and_weekday() is already
doing.
It's doing that wrong in some cases.  But, more importantly, it's solving
the wrong problem:  Correct me if I'm wrong, but no country I've found so
far defines DST transition days in those terms.
On Sun, Oct 15, 2023 at 8:21 AM ekondayan @.***> wrote:

Thanks for the contribution. One last thing before I merge the commit.
Would you explain in the commit description why "Calculate DST from
ordinal week number" is introduced and why is it needed.
—
Reply to this email directly, view it on GitHub
https://github.com/ekondayan/micropython-ntp/pull/6#issuecomment-1763422568,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABHMNQSM6TBIIGJUTKZETCLX7P5Q7ANCNFSM6AAAAAA4RAFXK4
.
You are receiving this because you authored the thread.Message ID:
@.***>

Not sure I understand what you're asking me to explain.  "Calculate DST
from ordinal week number" is what day_from_week_and_weekday() is already
doing.

day_from_week_and_weekday() is perfectly fit to find the last weekday of the month, which is applicable to half the countries that use DST. The shortcoming is that it does not work for the countries that change the DST on a day other than the last weekday of the month. Take a look at DST by country
The correct way is to find the ordinal weekday of the month, which weekday_in_month() is doing.
I am asking you to explain that in the commit message, so when people pull that commit, they can see the reason behind switching from day_from_week_and_weekday() to weekday_in_month().
I believe that for a project to be successful I must be very well commented and explained. That is why I am asking you to verbosely explain your PR in the docstring of the function and the commit message.
P.S.
Now I see one country that the library does not support - Israel. They change the time on the Friday before the last Sunday :)

You can confirm that both functions find the last weekday correctly
import unittest
import ntp

class TestDayFromWeekAndWeekdayWithImport(unittest.TestCase):

    def test_compare_both(self):
        errors = []
        for year in range(1, 3000):
            for month in range(ntp.Ntp.MONTH_JAN, ntp.Ntp.MONTH_DEC):
                for weekday in range(ntp.Ntp.WEEKDAY_THU, ntp.Ntp.WEEKDAY_SUN + 1):
                    try:
                        result1 = ntp.Ntp.day_from_week_and_weekday(year, month, ntp.Ntp.WEEK_LAST, weekday)
                        result2 = ntp.Ntp.weekday_in_month(year, month, ntp.Ntp.WEEK_LAST, weekday)
                    except:
                        continue

                    if result1 != result2:
                        errors.append(f"For {year}/{month}, day_from_week_and_weekday=>{result1} and weekday_in_month=>{result2}")

        # Raise an assertion with all collected error messages
        self.assertTrue(not errors, "\n".join(errors))

# Run the tests
unittest.TextTestRunner().run(unittest.TestLoader().loadTestsFromTestCase(TestDayFromWeekAndWeekdayWithImport))

Looking at the code, it looked like it would always work for the last
weekday, and always work for Sunday.  But non-Sunday and non-last could
cause errors depending on the month starting day.
On Mon, Oct 16, 2023 at 9:41 PM ekondayan @.***> wrote:

You can confirm that both functions find the last weekday correctly
import unittestimport ntp
class TestDayFromWeekAndWeekdayWithImport(unittest.TestCase):
def test_compare_both(self):
    errors = []
    for year in range(1, 3000):
        for month in range(ntp.Ntp.MONTH_JAN, ntp.Ntp.MONTH_DEC):
            for weekday in range(ntp.Ntp.WEEKDAY_THU, ntp.Ntp.WEEKDAY_SUN + 1):
                try:
                    result1 = ntp.Ntp.day_from_week_and_weekday(year, month, ntp.Ntp.WEEK_LAST, weekday)
                    result2 = ntp.Ntp.weekday_in_month(year, month, ntp.Ntp.WEEK_LAST, weekday)
                except:
                    continue

                if result1 != result2:
                    errors.append(f"For {year}/{month}, day_from_week_and_weekday=>{result1} and weekday_in_month=>{result2}")

    # Raise an assertion with all collected error messages
    self.assertTrue(not errors, "\n".join(errors))

Run the testsunittest.TextTestRunner().run(unittest.TestLoader().loadTestsFromTestCase(TestDayFromWeekAndWeekdayWithImport))
—
Reply to this email directly, view it on GitHub
https://github.com/ekondayan/micropython-ntp/pull/6#issuecomment-1765655701,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABHMNQWUF7Z2EJJNRIBBJN3X7YD6FAVCNFSM6AAAAAA4RAFXK6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRVGY2TKNZQGE
.
You are receiving this because you authored the thread.Message ID:
@.***>

Not only the last Sundays, but also Thursdays, Fridays and Saturdays. It also works for first weekdays, basically it works for all first and last weekdays. It doesn't work for countries that start DST on the second Sunday, like US, Canada, Greenland, Cuba, etc.
After switching to weekday_in_month() maybe I will mark day_from_week_and_weekday() for deprecation.
P.S.
I found a bug in day_from_week_and_weekday()  that returns wrong first weekdays for certain months, which I fixed in the last commit.

I had run into problems with first weekdays, but I made my fork before you
put in your fix.
Yes, deprecating weekday_in_month looks appropriate to me but I didn't want
to make that call.
On Mon, Oct 16, 2023 at 10:21 PM ekondayan @.***> wrote:

Not only the last Sundays, but also Thursdays, Fridays and Saturdays. It
also works for first weekdays, basically it works for all first and last
weekdays. It doesn't work for countries that start DST on the second
Sunday, like US, Canada, Greenland, Cuba, etc.
After switching to weekday_in_month() maybe I will mark
day_from_week_and_weekday() for deprecation.
P.S.
I found a bug in day_from_week_and_weekday() that returns wrong first
weekdays for certain months, which I fixed in the last commit.
—
Reply to this email directly, view it on GitHub
https://github.com/ekondayan/micropython-ntp/pull/6#issuecomment-1765687861,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABHMNQXNVITJIFSSITD5YVDX7YIW7AVCNFSM6AAAAAA4RAFXK6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRVGY4DOOBWGE
.
You are receiving this because you authored the thread.Message ID:
@.***>


Cordova build fails for Android 12

Specifying the telerik imagepicker using this entry in config.xml
  <plugin name="cordova-plugin-telerik-imagepicker" version="2.3.6" source="npm" />
causes the build to fail with this series of VoltBuilder log entries.
Discovered plugin "cordova-plugin-telerik-imagepicker". Adding it to the project
Installing "cordova-plugin-telerik-imagepicker" for android
Failed to install 'cordova-plugin-telerik-imagepicker': Error: cordova-plugin-telerik-imagepicker cannot be added. <edit-config> changes in this plugin conflicts with <edit-config> changes in config.xml. Conflicts must be resolved before plugin can be added.
    at PlatformMunger.add_plugin_changes (/node_modules/cordova-common/src/ConfigChanges/ConfigChanges.js:109:19)
    at /node_modules/cordova-common/src/PluginManager.js:121:33
    at _fulfilled (/node_modules/q/q.js:854:54)
    at /node_modules/q/q.js:883:30
    at Promise.promise.promiseDispatch (/node_modules/q/q.js:816:13)
    at /node_modules/q/q.js:877:14
    at runSingle (/node_modules/q/q.js:137:13)
    at flush (/node_modules/q/q.js:125:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
Failed to restore plugin "cordova-plugin-telerik-imagepicker". You might need to try adding it again. Error: Error: cordova-plugin-telerik-imagepicker cannot be added. <edit-config> changes in this plugin conflicts with <edit-config> changes in config.xml. Conflicts must be resolved before plugin can be added.
Cannot read properties of null (reading 'find')

2023-06-15 10:05:49,712 -
Failed to install 'cordova-plugin-telerik-imagepicker': Error: cordova-plugin-telerik-imagepicker cannot be added.
<edit-config> changes in this plugin conflicts with <edit-config> changes in config.xml. Conflicts must be resolved before plugin can be added.
2023-06-15 10:05:49,712 - Build failed 7cc946d7-5aa9-4d86-ab26-8fefaf6cd426
2023-06-15 10:05:52,020 - Output: Package          Current  Wanted  Latest  Location                      Depended by
cordova-android   11.0.0  11.0.0  12.0.0  node_modules/cordova-android  7cc946d7-5aa9-4d86-ab26-8fefaf6cd426

Did you get it working? Used another plugin?


adding in extra row for total served+unserved pop

This closes #206
Example outputs can be found in #206

Hey I had a look at #206.
The outputs look quite good but I noticed the Scottish data doesn't have all the columns. I guess you probably already know this
The Milton Keynes data has:
Year | Sex | Age | Disability Status | Local Authority | Urban/Rural | Series | Observation Status | Unit Multiplier | Unit Measure | Value
While the Scotland data has:
Local Authority | Series | Observation Status | Unit Multiplier | Unit Measure | Value | Sex | Year
I ran the code for Eng/Wales...which randomly selected Ealing.
This worked great, here's the csv output in my spreadsheet app:

I do not currently have the data to run this for Scotland so I can't test it. But I have looked over the code which looks great and it certainly works for England/Wales so the logic is good.


add test for redaction

Codecov Report
Base: 84.85% // Head: 85.28% // Increases project coverage by +0.43% :tada:

Coverage data is based on head (aea9cf2) compared to base (781077e).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #75      +/-   ##
==========================================
+ Coverage   84.85%   85.28%   +0.43%
==========================================
  Files          42       42
  Lines        4847     4847
==========================================
+ Hits         4113     4134      +21
+ Misses        642      623      -19
+ Partials       92       90       -2

Flag
Coverage Δ

go_tests
85.28% <ø> (+0.43%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

sub.go
92.15% <ø> (+2.35%)
:arrow_up:

xopjson/jsonlogger.go
76.20% <0.00%> (-0.40%)
:arrow_down:

line.go
76.47% <0.00%> (+16.66%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


⚠ Bofrost.be has degraded performance

In 9cb9517, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1279 ms

Resolved: Bofrost.be performance has improved in ba19b59.


[21159] Adjust for removal of include/fastrtps

Description

This PR adjust for the include/fastrtps removal from:

eProsima/Fast-DDS#4930

It depends from the documentation changes from (docs PR not opened yet):

eProsima/Fast-DDS#4898

Contributor Checklist

[x] Commit messages follow the project guidelines.
N/A: Code snippets related to the added documentation have been provided.
[x] Documentation tests pass locally.
N/A; Applicable backports have been included in the description.

Reviewer Checklist

[x] The PR has a milestone assigned.
[x] The title and description correctly express the PR's purpose.
[x] Check contributor checklist is correct.
[ ] CI passes without warnings or errors.

@richiprosima please test_3 this


Added _relatesToOrganization to docmaps api

https://github.com/elifesciences/data-hub-issues/issues/622

I guess you are still working on the query?


Access to wiki and documentation suggestion

I wanted to add an example with config to wiki, but it is not enabled for editing. Is it intentional?
Could you please add to Getting Started documentation an example of using ComparisonConfig?
It also will be good to have full help on searchable github site.
Please also add a link to  CodeProject Tutorial

MNF,
While I appreciate your suggestions, I prefer the CHM file which is included with the source code.  The CHM file has a config example.  The CodeProject tutorial is dated and uses a previous version which is obsolete.  I am still unsure if I am going to stick with GitHub.  GitHub might be a fad like CodePlex which is where I hosted before.
Greg


Doesn't onnxruntime-cpu have onnxruntime.quantization.preprocess module?

I got an error that cannot import name 'preprocess' from onnxruntime.quantization.

I got an error that cannot import name 'preprocess' from onnxruntime.quantization.

The python package doesn't have it, but you can find it in the source code. As far as I understand, there is currently no way to install the python package directly from the source (e.g. python setup.py install won't work). A workaround is creating a symlink that points to the source files. For that:

Download the source code from github

git clone https://github.com/microsoft/onnxruntime.git

Make sure that onnxruntime is already installed in your python environment. Locate the quantization package inside the site-packages folder and rename it. Let's say you are using a virtual env, then it should look like something like this

venv\Lib\site-packages\onnxruntime\quantization -> venv\Lib\site-packages\onnxruntime\quantization_

Now, create a symlink that points to the newly downloaded onnxruntime source files (WARNING: use absolute paths)

import os

os.symlink(
  '<path-to-onnxruntime>/onnxruntime/python/tools/quantization',  # use absolute path
  '<your-to-venv>/Lib/site-packages/onnxruntime/quantization'  # use absolute path
)

Test to see if it is working

python -m onnxruntime.quantization.preprocess --help

@yufenglee Do you have an answer for this question?

Maybe onnxruntime.quantization should not raise the following warning until preprocess is part of the Python package?
Please consider to run pre-processing before quantization. Refer to example: https://github.com/microsoft/onnxruntime-inference-examples/blob/main/quantization/image_classification/cpu/ReadMe.md


Results Page Should Auto-Select Results Step

When loading the results page, the first step (email) will be selected.  Instead, the results step should be selected by default in that case.

https://github.com/Autodesk/molecular-design-applications/pull/178


Add optional low-footprint printf float replacement

Disabling float support in the firmware via Kconfig and using this replacement instead saved about 4.5 kB of flash during my tests.
Please double-check if the approach makes sense and if I may have missed any important edge cases.

Zephyr now supports picolibc, which should provide a reliable solution for lower-footprint float printf.


CMake: Respect libdir for config files

Fixes installation path for RPM distros that use /usr/lib64 on x86_64.

CMake Error: INSTALL(EXPORT "tinyxml2Targets") given absolute DESTINATION "/usr/local/lib/cmake/tinyxml2" but the export references an installation of target "tinyxml2" which has relative DESTINATION "lib".

Hm, I'll investigate over the week-end or beginning of next week.

Closing pending a fix.


Update Required in Header

I've noticed that starting from fall 2023, the existing code is no longer functional. According to my research, the initial line in the header script should be updated to the following as of now:
<script async src="https://maps.googleapis.com/maps/api/js?v=3.53&key={API KEY}"></script>

However, it's important to note that this adjustment may become obsolete after April 2024. For more details and updates, please refer to the following resources:

Community Qualtrics Forum
Stack Overflow Discussion

Hey! I am facing the same issue. My map does not display, and my API works perfectly. By using your solution the map displays, but the location of the PIN is not recorded (I also tried single-line text and so on...). Also just one pin location, instead of two showing... How can we solve this?
Thanks!

Hi there,
I'm happy to hear that your issue is now resolved! I must point out that it is not my solution, I just found it in the links I provided.
I've made a slight adjustment to the HTML script. Please use this updated code and paste it as the first line within the header script. This should definitively address the problem and allow you to collect data accurately.
@pablomartinezalcaraz

Wow! Thanks a lot, it works perfectly 👍

Is this still working for anyone? I cannot get the map to display. I have tried changing it to 3.57 also. No dice.


The problem of decomposing the prompt into the sum of smaller prompts and low-rank products.

Hi, I see that the code for decomposing the prompt into the sum of smaller prompts and low-rank products in your paper is in DePT/peft/tuners/prompt_tuning_lora.py? If so, the code does not seem to reflect the decomposition of the prompt into the sum of smaller prompts and low-rank products, and it is still the code for prompt-tuning. Can you answer my doubts?

We have self.lora_embedding_A = nn.Parameter(torch.zeros((config.max_length, config.r)), requires_grad=True) and self.lora_embedding_B = nn.Parameter(torch.zeros((config.r, config.token_dim)), requires_grad=True).  Please check how they are used in the code.

OK, I've seen that. Thanks a lot for your answer!


Requires explicit discriminator value for every variant

The current macro only works for enum FooBar { Foo = 0, Bar = 1 } and not enum FooBar { Foo, Bar } or enum FooBar { Foo = 0, Bar }.

Hmm. Even if the macro was somehow adjusted to match variants without explicit discriminators, rust-lang/rust#23898 prevents them from being used in the match statement we generate.

Fixed by e12402cd39fca5c5df4ba676d75f0d9be7b5fa16, in a way that I didn’t think was going to be possible!


When creating a line chart the background is always black

This is the code that I write:
`var chartNode = new ChartjsNode(600, 600);
return chartNode.drawChart({
type: 'line',
data: {
labels: ["14:00", "14:10", "14:20", "14:30", "14:40", "14:50", "15:00", "15:10", "15:20"],
datasets: [{
label: '# of Viewers',
data: [1000, 1150, 900, 700, 2000, 2200, 2200, 3000, 2500],
backgroundColor: [
'rgba(255, 159, 64, 0.2)'
],
borderColor: [
'rgba(75, 192, 192, 1)'
],
borderWidth: 1,
fill: true
}]
},
options: {
  responsive: false,
  animation:true
}

})`
but the result is always with black background, why?
Thanks

Don't assign backgroundColor and borderColor as an array of rgba color strings. Assign it directly to a color string
example:
datasets: [   {      ...      backgroundColor: "rgba(75, 192, 192, 1)",      borderColor: "rgba(75, 192, 192, 1)"      ...   } ]
This should solve your issue.


Your projects needs a .gitignore in order for it to be open source.

If you don't want it to be open source don't use GitHub.
You should pay if you want to keep it closed-source !!!
Plz add this .gitignore

No idea why I did this. Probably trolling for whatever reason


GH-673: Revert "Remove chmod on project dir"

Describe in detail the problem you had and how this PR fixes it
Due to the decision to remove the mode change, this inherently
reintroduced a bug that prevents the system to write to the bind
mounted folder. the chmod is intended to fix it.
Is there an open issue you can link to?
this fixes GH-673

How does this fix #676? In fact, if you don't have read perms for the folder, how would the chmod even succeed?

@nhooyr if I got the writing part for this right, you still have ownership in the folder because you're inside coder. I've used this before in my Coder images and they seem to work. We can still change the modes since we have ownership on it.

@nhooyr, for what it is worth I would like to see this approved or denied as I am experiencing the same issue:
https://github.com/cdr/code-server/issues/454#issuecomment-502461873

you still have ownership in the folder because you're inside coder.

Then why do you need to make it group editable if you have ownership?

Apparently this does not mandate the ownership well for bind mounts for some reason so explicitly setting them as RW is the only way to make sure you can write changes from container to host @nhooyr.

We need to know what the reason is before merging this. The only reason I see is if the directory outside of the container has a different owner than uid 1000 at which point you should be changing the directory outside of the container or run code-server as a different user instead of uid 1000. See #640

We need to know what the reason is before merging this. The only reason I see is if the directory outside of the container has a different owner than uid 1000 at which point you should be changing the directory outside of the container or run code-server as a different user instead of uid 1000. See #640

it will "Permission denied" on docker

We need to know what the reason is before merging this. The only reason I see is if the directory outside of the container has a different owner than uid 1000 at which point you should be changing the directory outside of the container or run code-server as a different user instead of uid 1000. See #640

it will "Permission denied" on docker

Even if we set it to UID1000, the permissions in the bind mounted folder is not the same, therefore, we should force writes and reads on the bind folder so we can write to the folder.

There is a inherent reason we do this as well, permissions is irrelevant to the host because its a container-specific permission. There will be no impact on security since it only applies to the container.

There is a inherent reason we do this as well, permissions is irrelevant to the host because its a container-specific permission. There will be no impact on security since it only applies to the container.

I don't think this is true, if its mounted in, then any changes to it will affect the folder outside the container. Docker doesn't have user namespacing enabled by default, so the UIDs/GIDs inside the container are the same as the UIDs/GIDs outside.
Can you show the perms on the folder you're having trouble with and then show the UID the container is running as? Do a ls -la and post it here.

In fact can you open a new issue? A issue is better suited for debugging than this PR.

Revisiting this PR due to GH-992. Impact is considered major.

There is a inherent reason we do this as well, permissions is irrelevant to the host because its a container-specific permission. There will be no impact on security since it only applies to the container.

I don't think this is true, if its mounted in, then any changes to it will affect the folder outside the container. Docker doesn't have user namespacing enabled by default, so the UIDs/GIDs inside the container are the same as the UIDs/GIDs outside.
Can you show the perms on the folder you're having trouble with and then show the UID the container is running as? Do a ls -la and post it here.

@marcdumais-work Provides a explanation over at our friends at Theia, which you can find here.

The permission issues you have are expected, and a generic Docker issue when sharing files between a host and container using a volume. What happens is that inside the container, the theia user is different than your user on the host. You can use the id -u and id -g commands to confirm (respectively user id and current main group id) - the user name/group names do not matter, just the ids, when it comes to permissions on Linux.

If you are looking for a solution specifically for yourself, one option would be to modify the image you use and have the theia user created in the image using your host uid/gid, and then you could access the same files from both without these issues.The permission issues you have are expected, and a generic Docker issue when sharing files between a host and container using a volume. What happens is that inside the container, the theia user is different than your user on the host. You can use the id -u and id -g commands to confirm (respectively user id and current main group id) - the user name/group names do not matter, just the ids, when it comes to permissions on Linux.

So essentially speaking in the terms of Unix permission model we do not have the ability to write to that folder even if we own /projects because the file contents isn't ours. Overriding permissions in the container isn't a real security issue since we're only mounting a specific folder. Giving write access on that folder explicitly is the only safe workaround. Another workaround is to have the user write it under the root group. We shouldn't worry about the container writing where its not supposed to because we are supposed to write there, but the workaround is intended so they can write normally to a mounted folder or a PV.

This wouldn't work regardless. If we do something like this, it has to be done in the entrypoint.sh, not just for the top level project directory which will have the real project directory mounted in as even if the user owns the top level project directory, they still won't be able to access files in a subdirectory.