ValueError: Blink type tag not present

Hi,
Thanks for your efforts in developing this code! Much appreciated.
I have a problem with the code stopping during the iterate_records function within ccl_chromium_indexeddb.py.
The code stops here:
               if blink_type_tag != 0xff:
                    # TODO: probably don't want to fail hard here long term...
                    raise ValueError("Blink type tag not present")

I'm finding that for some records in my Skype IndexedDB, have a blink_type_tag of 16, causing the code to stop at the above location.
I've implemented a temporary workaround to allow the code to print a message and continue rather than stopping at this point:
               if blink_type_tag != 0xff:
                    # TODO: probably don't want to fail hard here long term...
                    #raise ValueError("Blink type tag not present")
                    print("***** Skipping record with unknown blink_type_tag: " + str(blink_type_tag) + " *****")
                    continue

After running the above modified code, I found that for my Skype IndexedDB, I get approximately 5,000 affected records with a blink_type_tag of 16, out of a total of 20,000 records.
On a potentially related matter, I'm also seeing 'ccl_v8_value_deserializer._Undefined object at 0xXXX' errors when printing out the values: (yes, this might be a direct consequence of my code modification above, but thought you should know).
print(f"key: {record.value}")

key: {'_serverMessages': [{'id': '1525335581949', 'originalarrivaltime': '2018-05-03T08:19:38.846Z', 'messagetype': 'RichText', 'version': '1525335581949', 'composetime': '2018-05-03T08:19:38.846Z', 'clientmessageid': '10187620813155729536', 'conversationLink'<EMAIL_ADDRESS>'content': 'OK. I'll provide you the photos that we could do for the embossed texture. ', 'type': 'Message', 'conversationid'<EMAIL_ADDRESS>'from': 'https://hk2-2-client-s.gateway.messenger.live.com/v1/users/ME/contacts/8:removed'}], 'cuid': '10187620813155729536', 'conversationId'<EMAIL_ADDRESS>'createdTime':<PHONE_NUMBER>949.0, 'creator': '8:removed', 'content': 'OK. I'll provide you the photos that we could do for the embossed texture. ', 'messagetype': 'RichText', 'contenttype': <ccl_v8_value_deserializer._Undefined object at 0x7eff208b68b0>, 'properties': <ccl_v8_value_deserializer._Undefined object at 0x7eff208b68b0>, '_isEphemeral': False, '_fileEncryptionKeys': <ccl_v8_value_deserializer._Undefined object at 0x7eff208b68b0>, '_countsType': 1, '_isMyMessage': 0}

Any assistance would be greatly appreciated!

Hi there, thanks for the report.
I'll do the easy one first: the undefined thing you're seeing isn't an error - it's representing the JavaScript "undefined" value (which is different from null (or None in Python) - the problem there is that there's not a nice str or repr defined for that class - that's an easy fix practically, and that'll be easy to do (and I'll get it done shortly).

So the version tag issue: I'll go back into the Chrome source and have a look at what is permissible there - I vaguely recall reading something in there about the version tag being able to deviate, but I wasn't able to generate any test data which did so. If it's not sensitive, are you able to share one of the blobs from the leveldb which exhibits this so I can cross reference it with the source?

Thanks for the reply. I assume you are referring to the "Blink Type tag" issue, rather than "version tag issue"? Or are these related?
I can try and find a non-sensitive blob for you. Can you provide guidance on how I could extract one for you from the leveldb? And what format to supply it in? (Or could I achieve this by inserting a print 'something' command as part of my workaround code above?)

Hi first - I've pushed an update that'll stop making the Undefined values look weird!
To export the blob, maybe temporarily alter your update to:
if blink_type_tag != 0xff:
    # TODO: probably don't want to fail hard here long term...
    #raise ValueError("Blink type tag not present")
    with open(f"example_data_blink_tag_{record.seq}.bin", "wb") as temp_out:
        temp_out.write(record.value)
    continue

That should dump the contents of each nonconforming record to a file in your working directory... that is going to create 5000ish files of course based on your example above, so you may want to add a counter or just break the script part way through!

Hi,
Thanks for the update. The new "undefined" message definitely looks better in the output now.
I've had a look at the contents of the nonconforming records by printing out record.value. I can see that they definitely contain Skype messages, which are of interest to us.
I can send you a few sample dump files, as per your earlier suggestion, but due to their potentially sensitive nature, I will need to send these to you over a separate/private channel. (I don't think that GitHub supports such a mechanism). Would it be OK if I reach out to you directly via the CCL website contact page? Or do you have a better suggestion?
Thanks in advance.

Hi there, yes, please do; if you mark it foa of Alex in R&D then it should get to me. I look forward to hearing from you.

Thanks Alex. I just sent you a message via the CCL website.

Thank you for that. I'm having a rotten time trying to find a rationale for the data however...
I had written up my findings for the header where the problem resided, but I've just noticed something else peculiar about the data that I have no idea how I could have missed. In the files you sent after the initial version varint (or possibly even including it, but that makes one of the versions look incorrect), every 2 bytes are swapped. That's really weird isn't it? That isn't the case with the example message (with the Undefined values) you provided above.
Anyway - if you swap each pair of bytes starting at offset 3 in all of the examples you kindly provided to me, the data looks right... so now the mystery is...why the byte swapping? If it's a bug it is happening earlier in the process, but I can't see where or why. I'll have to think around this some more.
Alongside the mystery is a big API issue which is: the Exception shouldn't be happening without a way to handle it when you're iterating records in a for-loop. That's something I can address, probably with a switch to log errors rather than throw them, or with a callback function. Or both options maybe.

Commit 24b97c5c029dd09ffcd587a1d056bd40108c662d will make it easier to skip the bad data. Readme has been updated to show you how.

Thanks for the update. Yes, I did notice that the affected messages had every 2 bytes swapped!
Let me know if I can assist with this. Feel free to supply code with debug statements that I can run against my data if you wish.

Any updates on the cause of the byte swapping in some of the messages?
As a temporary workaround, I'm thinking of trying to write some code to do the byte swapping for us, so that we can successfully extract these messages. I'm ok with Python, but this particular task could take me a while to work out on my own.. Any chance you can suggest some suitable temporary code to do this?
I have tried running the script on a Skype IndexedDB from a different system and it worked just fine. The only difference between the two IndexedDBs that I can think of, is that the failing datastore will most likely contain some messages with non-latin characters.. (specifically Chinese characters). I'm not sure if this is related to the root cause of the issue, but I thought it might be worth mentioning.
Hope this helps. Thanks again.

Any updates on the cause of the byte swapping in some of the messages?
As a temporary workaround, I'm thinking of trying to write some code to do the byte swapping for us, so that we can successfully extract these messages. I'm ok with Python, but this particular task could take me a while to work out on my own.. Any chance you can suggest some suitable temporary code to do this?
I have tried running the script on a Skype IndexedDB from a different system and it worked just fine. The only difference between the two IndexedDBs that I can think of, is that the failing datastore will most likely contain some messages with non-latin characters.. (specifically Chinese characters). I'm not sure if this is related to the root cause of the issue, but I thought it might be worth mentioning.
Hope this helps. Thanks again.

Hi there, sorry I've been on leave so I haven't had a lot of time to dig. I can't see any obvious reason why it's happening at the moment regardless. If it was only in the text/in fixed length numerical values there may be some logic, but it's literally everything. I wonder if it's some kind of semi-secure delete attempt?
As for byte swapping - it looks like it's everything after offset 3 so something like (untested)
python

swappable_data = record.data[3:]
swapped_data = bytes(swappable_data[i - 1] if i % 2 == 1 else swappable_data[i + 1] for i in range(len(swappable_data) - 1))
if len(swappable_data) % 2 == 1:
    swapped_data += swappable_data[-1:]

(the condition at the end there makes assumptions about how the end of the swapped data is handled).
you'll need to trap the bad data and run it through the deserializer "manually", something like (obj_raw is your record data), the code, roughly can be found around line 450 of ccl_chromium_indexeddb something like:
blink_deserializer = ccl_blink_value_deserializer.BlinkV8Deserializer()
deserializer = ccl_v8_value_deserializer.Deserializer(obj_raw, host_object_delegate=blink_deserializer.read)
try:
    value = deserializer.read()  # value is the processed object
 except Exception:
    if bad_deserializer_data_handler is not None:
        bad_deserializer_data_handler(key, record.value)
    else:
        raise

Hi there, sorry I've been on leave so I haven't had a lot of time to dig. I can't see any obvious reason why it's happening at the moment regardless. If it was only in the text/in fixed length numerical values there may be some logic, but it's literally everything. I wonder if it's some kind of semi-secure delete attempt?
As for byte swapping - it looks like it's everything after offset 3 so something like (untested)
python

swappable_data = record.data[3:]
swapped_data = bytes(swappable_data[i - 1] if i % 2 == 1 else swappable_data[i + 1] for i in range(len(swappable_data) - 1))
if len(swappable_data) % 2 == 1:
    swapped_data += swappable_data[-1:]

(the condition at the end there makes assumptions about how the end of the swapped data is handled).
you'll need to trap the bad data and run it through the deserializer "manually", something like (obj_raw is your record data), the code, roughly can be found around line 450 of ccl_chromium_indexeddb something like:
blink_deserializer = ccl_blink_value_deserializer.BlinkV8Deserializer()
deserializer = ccl_v8_value_deserializer.Deserializer(obj_raw, host_object_delegate=blink_deserializer.read)
try:
    value = deserializer.read()  # value is the processed object
 except Exception:
    if bad_deserializer_data_handler is not None:
        bad_deserializer_data_handler(key, record.value)
    else:
        raise

Closing as inactive (also the problem was specific in this data it seems)


Mermaid ELK integration not working in draw.io

Preflight Checklist

[x] I agree to follow the Code of Conduct that this project adheres to.
[x] I have searched the issue tracker for a feature request that matches the one I want to file, without success.

Describe the bug
The ELK integration in draw.io appears to not function properly. When trying to use ELK for layouts, no layout is applied, and the diagrams do not adjust as expected.
To Reproduce
Steps to reproduce the behavior:

Open draw.io.
Attempt to use the ELK layout engine for diagram organization.
No layout is applied, and no error message is shown.
The diagram remains in its original state without adjustment.

Expected behavior
The diagrams should automatically adjust their layout according to the ELK engine when selected. This was functioning previously but now appears broken.
Screenshots
N/A
draw.io version (In the Help->About menu of the draw.io editor):

draw.io version: v24.7.17

Desktop (please complete the following information):

OS: Linux Mint 22
Browser: Firefox
Browser Version: 130.0.1

Smartphone (please complete the following information if applicable):

Device: N/A
OS: N/A
Browser: N/A
Version: N/A

I tested the problem in incognito/private mode with all browser extensions switched off, write "yes" below:

yes

Additional context
This issue has been reported before in issue #3634 but seems to have resurfaced.
Can be checked with this

Mermaid ELK 集成在 draw.io 中不起作用

It's broken at the moment. Mermaid isn't a core part of draw.io, it's support very much "as-is".


GUVNOR-2321: Guided Decision Table: Wrong 'label' for operator selector

See https://issues.jboss.org/browse/GUVNOR-2321

+1

Thanks.
master: 2b1c743c3897f9f17b5cac2a9c0505fc4b710f76


Orchard.Tags - Manage Tags Admin Page Not Alpha Sorted

Not a big deal for a few but wouldn't be manageable with a big site.
New admin shown but valid for old admin as well.

@Skrypt this isn't admin tag either

Are you planning making a lot of pull requests this week-end ?

Yep haha, im going to bed now for a fresh start tomorrow to clear some of my backlog :)


Implement Databricks-compatible dataset versioning

Notice

[x] I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":

I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.

I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorised to submit this contribution on behalf of the original creator(s) or their licensees.

I certify that the use of this contribution as authorised by the Apache 2.0 license does not violate the intellectual property rights of anyone else.

Motivation and Context
Why was this PR created?
Copied from internal Slack channel:
Hi Dmitrii. Thanks for following up. Unfortunately, I’m still facing issues…
To recap, my data is Azure Blob Storage, and we’re running on Databricks. As a result, I’m accessing via mnt paths.
Unfortunately, /mnt is not accessible by local file APIs. @prasad.gaikwad essentially pointed this out on my other question (use /dbfs/mnt/... as the filepath for Pandas-based datasets, instead). I see that echoed in https://forums.databricks.com/answers/14051/view.html -> https://docs.databricks.com/data/databricks-file-system.html#local-file-apis
Therefore, a filepath like /mnt/... for a versioned SparkDataSet fails, because it uses iglob--a local file API, trying to access something starting with /mnt--to find versions. https://github.com/quantumblacklabs/kedro/blob/develop/kedro/io/core.py#L486 <-- this line returns []
If you specify the filepath for a versioned SparkDataSet starting with /dbfs/mnt instead, it actually finds the versions! However… the _load method comes back with: org.apache.spark.sql.AnalysisException: Path does not exist: dbfs:/dbfs/mnt/... T_T
I assume Databricks is throwing that extra dbfs:/ up front, even if it’s prefixed by /dbfs already. What would you suggest? Standardize on specifying paths with /dbfs for Databricks, but special case drop the /dbfs prefix up front in the _load or something?
How has this been tested?
What testing strategies have you used?
Manually tested on my current project (when no versions exist, and with existing versions).
TODO: Add test coverage for new lines.
Checklist

[x] Read the contributing guidelines
[x] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
[ ] Updated the documentation to reflect the code changes
[ ] Added a description of this change and added my name to the list of supporting contributions in the RELEASE.md file
[x] Added tests to cover my changes
[ ] Assigned myself to the PR

Pushed fbadd0c to demonstrate that the tests cover the negative if branch for all 3 cases: save, load, and exists. Otherwise, build passes without additional tests (or just one additional test).

Apart from tests, could you please also document your change, so that the users know that it solves a specific issue on Databricks.

@DmitriiDeriabinQB Where would you suggest documenting this change--in pyspark/README.md or in the docs for working with Databricks? I tried explaining it in the release notes, but it seemed kinda long (I can add a parenthesized reference to see comments elsewhere).

@deepyaman I would put it into pyspark/README.md and probably also into SparkDataSet.__init__ docstring - as a note for the filepath argument, for example.

Had a chat with @DmitriiDeriabinQB and it's much clearer in my head what's going on now. Feel free to ignore my questions, just my comments about test code stand.

@deepyaman I would put it into pyspark/README.md and probably also into SparkDataSet.__init__ docstring - as a note for the filepath argument, for example.

Done!

Thanks @DmitriiDeriabinQB @lorenabalan! :D


Markdown such as backticks and newlines rendered incorrectly on Slack

Version
What package version of the SDK are you using. 4.8.0
Describe the bug
Markdown sent by a bot isn't translated correctly to the Slack rich text format. Backticks do not work for inline code and new lines are not translated.
To Reproduce
Steps to reproduce the behavior:

Write a C# bot that sends the following to a room in a Slack Workspace.

turnContext.SendActivity("A `message` *with some bold text* and _some italicized text_.", cancellationToken: cancellationToken);

Deploy the bot to Azure
Connect the bot to Slack using Azure Bot Registration
Trigger the Bot

Expected behavior
I expected it to render like so:

Actual Behavior

Additional context
Notice that the inline code block isn't rendered as a code block. Also note that the bold text is italicized.
I understand that I can get inline code to work by escaping the backtick, but that seems wrong to me. Markdown doesn't require escaping backticks. Neither does Slack. You can see for yourself using Block Kit Builder and see what I mean.
As for the bold, I do understand that the Markdown format requires double asterisks. I was hoping to write my messages in a format that works for both Slack and Teams. Do you have a pointer to where the translation to rich text for each platform occurs?
[bug]

@haacked - Thank you for reaching out to us
If you would like to bypass the formatting applied by ABS, the easiest thing to do is set the text as channel data:
await turnContext.SendActivityAsync(
    new Activity
    {
        Type = ActivityTypes.Message,
        ChannelData = new
        {
            text = "A `message` *with some bold text* and _some italicized text_.",
        },
    }, cancellationToken);

If you would like to bypass ABS altogether, you can always use the Slack adapter that's now built into the SDK, though that would involve a lot of reworking of your bot.
Does this resolve your issue?

That works, thanks! What is ABS and can you point me to where that code is?

@haacked - ABS is the Azure Bot Service. The ABS channel connectors are how your one bot is able to talk to many channels like Slack. While the Bot Builder SDK code is open source, the service code is proprietary and I cannot show it to you.
I'm glad the solution is working for you and I'm closing this issue now.

While the Bot Builder SDK code is open source, the service code is proprietary and I cannot show it to you.

Understood. @v-kydela where is the proper place to report issues with ABS? And thanks again for your help.

@haacked - Issues can be reported in the BotFramework-Services repo or in the Azure portal. If you're not sure something is a bug, it's a good idea to post a question on Stack Overflow.


[CELEBORN-1726][FOLLOWUP] Avoid NPE when transition worker state

What changes were proposed in this pull request?
Avoid NPE when transition worker state
Why are the changes needed?
Fix test.
Does this PR introduce any user-facing change?
No
How was this patch tested?
No need.

Merged to main(v0.6.0) and branch-0.5(v0.5.2).


Proposal: Replace package manager lookup with facts discovery

For a container base OS there is no indication on what kind of package manager it supports. Ansible has a module that does the discovery of the package manager: https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/facts/system/pkg_mgr.py
Do something similar for container base filesystem.

Resolved in #67


allowing tables_path to be empty

We should allow a user to leave "tables_path" empty unless they are adding custom tables. At the moment this is causing an error, as it expect to set a "PosixPath" when running setup.
What should happen in the code if it's empty?

if we set a different default folder than a user could accidentally have there a copy of a table and that could cause issues.
so possibly we should just set to None and deal with that in the code when we open a table.

Another issue linked to this is when cmor is trying to load the tables for grids and for variable def:
tables.append(cmor.load_table(f"{ctx.obj['tpath']}/{ctx.obj['grids']}"))
tables.append(cmor.load_table(f"{ctx.obj['tpath']}/{ctx.obj['table']}.json"))

In theory this should still work if we're using the tables coming with the package, but it's failing when I set a random existing dir for tables

Minor issue linked to this is I forgot in get_plev() calculations.py to open coordinate file from compiled data, so this fails:
fpath = f"{ctx.obj['tables_path']}/{ctx.obj['_AXIS_ENTRY_FILE']}"
unless it's actually passed in that folder

same in require_bounds

Solved this by setting a default "non-existing-path" when tables_path is  left empty or it's None
if cdict['tables_path'] is None or cdict['tables_path'] == "":
        cdict['tables_path'] = appdir / "non-existing-path"
    else:
        cdict['tables_path'] = appdir / cdict['tables_path']

Also the bug above is also fixed


Fix Clef backend

It looks like somehow the the Clef backend diverged from the the Clef API. This PR adds a fix for that divergence.
Thank you for all of your hard work maintaining this library :)

Thanks!


Strange behavior on event handling in cogs

Describe the bug
I'm having behavior that I believe to be incorrect on creating of cogs. When I create more than one class inheriting from Cog, even though I don't instantiate it, it's being executed by the bot. And when I use the class, it seems to me that the bot is handling the same event more than once. Perhaps with the code below it will be easier to understand.
Are you using TwitchIO within a Discord Bot?
No
What commit of TwitchIO are you using?
Name: twitchio
Version: 2.0.0b7
Summary: A Python IRC and API wrapper for Twitch.
Home-page: https://github.com/TwitchIO/TwitchIO
Author: TwitchIO
Author-email: UNKNOWN
License: MIT
Location: c:\users\renan\desktop\twitch-bot-template\.venv\lib\site-packages
Requires: aiohttp
Required-by:

To Reproduce
Steps to reproduce the behaviour:
from twitchio.ext import commands

class FirstCog(commands.Cog):
    def __init__(self, bot):
        self.bot = bot

    @commands.Cog.event()
    async def event_message(self, message):
        if message.echo:
            return
        print("Hello from first cog")

class SecondCog(commands.Cog):
    def __init__(self, bot):
        self.bot = bot

    @commands.Cog.event()
    async def event_message(self, message):
        if message.echo:
            return
        print("Hello from second cog")

bot = commands.Bot(
    token="...",
    prefix="...",
    initial_channels=["..."],
)

bot.add_cog(FirstCog(bot))
# bot.add_cog(SecondCog(bot))  # do not add the second cog :D

bot.run()

When I type something on Twitch's chat, the console shows me:
Hello from first cog
Hello from second cog

If you uncomment the bot.add_cog(SecondCog(bot)) part and execute:
Hello from first cog
Hello from second cog
Hello from first cog
Hello from second cog

Expected behaviour
Only the events of the added cog should work.
Do not call twice the same event in different cogs
(please complete the following information):

OS:  Windows 10x64
Python: 3.9.5

Split the code in separated cogs also duplicate the event handling:
#bot/first_cog.py

from twitchio.ext import commands

class FirstCog(commands.Cog):
    def __init__(self, bot):
        self.bot = bot

    @commands.Cog.event()
    async def event_message(self, message):
        if message.echo:
            return
        print("Hello from first cog")

def prepare(bot):
    bot.add_cog(FirstCog(bot))

#bot/second_cog.py

from twitchio.ext import commands

class SecondCog(commands.Cog):
    def __init__(self, bot):
        self.bot = bot

    @commands.Cog.event()
    async def event_message(self, message):
        if message.echo:
            return
        print("Hello from second cog")

def prepare(bot):
    bot.add_cog(SecondCog(bot))

# __main__.py

from twitchio.ext import commands

bot = commands.Bot(
    token="...",
    prefix="...",
    initial_channels=["..."],
)

bot.load_module("bot.first_cog")
bot.load_module("bot.second_cog")

bot.run()

After any message on twitch chat:
Hello from first cog
Hello from first cog
Hello from second cog

There is an issue in the meta.py file with _load_methods()
izzy posted a workaround for it till it gets fixed.
this is from line 101
for event, callbacks in events.items():
    for callback in callbacks:
        if callback.__module__ == self.__module__:
            callback = partial(callback, self)
        else:
            continue

Fixed as of version 2.0.4. Thanks for the issue 👍


Releasing v0.8.5 soon

We had a request from the SCICO project to release a new version of PyAbel with the latest features. This seems like a great time for a release anyway, since we don't have any outstanding pull-requests, and we've incorporated lots of new goodies since v0.8.4 was released.
Any objections if @MikhailRyazanov and I release v0.8.5?

None from me. Go for it.

OK:

[x] RDT
[x] PyPI
[x] Zenodo
[ ] Conda?

For Travis, I think that only one of the builds successfully uploads to PyPi, and the rest fail because there is already a pyabel v0.8.5 on PyPi. (Or, at least that's what I think is going on. The error message is unhelpful, but I recall seeing it before.) We are indeed on PyPi: https://pypi.org/project/PyAbel/

I honestly have no idea how the conda-forge thing works. I thought that maybe it automatically took the recent version...

@rth: should the conda-forge repo automatically update after we make a release on PyPi?

Yes, a PR will be made by a bot in https://github.com/conda-forge/pyabel-feedstock but you would need to check and merge it.

Awesome, thanks!
I guess we should add this instruction to our release checklist in the contributing.rst.

@rth, the conda PR has been indeed created and passes all tests! Is it possible to see the resulting files? I can see in the logs that the Cython library is now compiled with the new options, but it would be interesting to examine the actual packages before releasing them.

You should see the log of pip install in the CI logs. It's also possible to add extra arguments to that command in meta.yaml for more verbosity.

I would like to see the conda packages built during the test. Or they are not saved at this stage?

It doesn't look like they are. The setup.py looks good to me, I don't think there would be issues.

The previous version also looked OK, but was crashing on user's systems (#320). ;–) I also hope that now it should work fine but just wanted to make sure.
Anyway, if there is nothing else we can do, could you please merge that conda-forge PR? (I don't have write permissions there.)

I have merge permission there, so I merged it. I don't seem to have admin permission, so I couldn't add @MikhailRyazanov to that repo. @rth, do you know how we can att @MikhailRyazanov?

Thanks, @DanHickstein! Now I see the v0.8.5 conda files: https://anaconda.org/conda-forge/pyabel/files.
I've downloaded the one that matches my system (linux-64...py39), and can confirm that it does work. And no AVX-512 instructions inside this time.

Okay, I think that I figured out how to add you as a maintainer on our confa-forge repo.
https://conda-forge.org/docs/maintainer/updating_pkgs.html#updating-the-maintainer-list
https://github.com/conda-forge/pyabel-feedstock/issues/21
https://github.com/conda-forge/pyabel-feedstock/pull/22

OK, thanks! Let's see what happens when you merge that PR at  conda-forge/pyabel-feedstock...

I merged the PR, so I think that you're a maintainer now. I'm not sure how to check if this is the case or not, save for making a  PR there and seeing if you have a "merge" button.

Hm, I didn't notice any changes immediately, but that bot sent me an invitation to join conda-forge, and after acceptin it, I can see now various editing tools, which didn't work before (they were telling me to fork the repository). So I guess, this indicates that it worked, thanks!

Great!
So, it looks like we now ticked all of the boxes and I'll close this Issue. (Please re-open if there are outstanding things that we didn't address yet.)
Thanks Mikhail!

Also got en email:

You’ve been added to the "pyabel" team for the organization:
"conda-forge"
"pyabel" has 4 members and gives access to 1 "conda-forge" repository.

and can see myself among https://github.com/orgs/conda-forge/teams/pyabel/members.

Excellent! You are now a member of a very prestigious club with many perks. Your shipment of celebratory champagne should be arriving by courier shortly ;).


Can I use Representable10 and Update10 with structures containing lists?

Hi there,
First, congratulations on a really cool package. I’m excited to explore whether it’s a solution to this problem we’ve encountered while building the Semantic framework: we want to generate syntax trees that have shape functors attached to them, but working with these structures in a manner that’s comparably fluent to plain-old-data-types is tricky.
I’ve been experimenting with a JSON grammar, and have run into a problem: I can’t figure out how to specify that one syntax node should contain a list of others. I have these two types:
newtype EscapeSequence (f :: Type -> Type) = EscapeSequence { text :: Ap10 Text f }
  deriving stock (Eq, Ord, Read, Show, Generic, Generic1)
  deriving anyclass (NFData, Hashable)
  deriving (Portray, Diff) via Wrapped Generic (EscapeSequence f)
  deriving (Foldable10, Traversable10, Constrained10 c, Functor10, Applicative10, Representable10, Update10) via Wrapped1 Generic1 EscapeSequence

newtype StringContent (f :: Type -> Type) = StringContents { contents :: EscapeSequence f }
  deriving stock (Eq, Ord, Read, Show, Generic, Generic1)
  deriving anyclass (NFData, Hashable)
  deriving (Portray, Diff) via Wrapped Generic (StringContent f)
  deriving (Foldable10, Traversable10, Constrained10 c, Functor10, Applicative10, Representable10, Update10) via Wrapped1 Generic1 StringContent

and I would like StringContent to contain an [EscapeSequence f], or some sort of structure that allows one StringContent to hold multiple EscapeSequence values. (Note that in this case I could just remove the type parameter from EscapeSequence, but lists etc. appear elsewhere in syntax where this wouldn’t be an option.) I’ve tried using a plain [EscapeSequence f] field, which since [] is not Representable, precludes using Representable10 and Update10. I’ve tried using Cofree, and also tried using Compose and :.: and ListT to glom on an additional list-layer. None of these things seem to work—the only solution I’ve found is to remove the Representable10 and Update10 derivings. While I could dispense with them, it would be a huge win if I could retain that typeclass machinery, as I’d love to use ten-lens. Do you have any suggestions here?

Sorry for the long delay -- I'm struggling to convince GitHub to notify me when things happen in these repos.  I've just squashed another potential reason for it not to notify me, but who knows if it's the last.
It seems to be unavoidably true that a version of StringContent with a list of EscapeSequences can't be Representable10 or Update10: those are about having a fixed, statically-known "shape" w.r.t. the uses of f, and having a list in that position means the type has a variable shape.  So anything that actually requires the full power of Representable10 or Update10 actually won't work on this type.
That said, there could be situations where some functionality was convenient to derive from Representable10 but didn't actually need it, so depending on what in particular you wanted to use it for, it might be possible to come up with a suitable generalization that works for not-quite-Representable10 types.


fix: remove redundant refresh token spec

This does cleanup changes. Verification and review would be typically checking if commands work :D


Tidying up renderActorSheet hook

Made a quick fix, and combined #16

Made a quick fix, and combined #16


thread safety?

Is PyOctree thread safe? I would like to use multiprocessing to have multiple threads use the same pyoctree object to check if lines intersect the same mesh concurrently. Does that sound safe?
If not, is there a way I can copy an already made pyoctree object? Will copy.deepcopy work?

I implemented the serialization functions (needed for multiprocessing and copy) in https://github.com/mhogg/pyoctree/pull/56
You can test it via pip install git+https://github.com/EmJay276/pyoctree.git@serializability
Minimal example see:
https://github.com/EmJay276/pyoctree/blob/serializability/pyoctree/Examples/pickle_test.py
import pickle

import numpy as np
from pyoctree import pyoctree

vertices = np.array(((0, 0, 0), (1, 1, 1), (2, 0, 0)), dtype=float)
polygons = np.array(((0, 1, 2),), dtype=int)

tree = pyoctree.PyOctree(vertices, polygons)

tree_data = pickle.dumps(tree)
tree_new = pickle.loads(tree_data)


HTMX example is not working per v4.4

What version of Hono are you using?
4.4.4
What runtime/platform is your app running on?
Bun
What steps can reproduce the bug?

bun create hono@latest -> x-basic
bun i -D typed-htmx
Add the snippets from https://hono.dev/snippets/htmx#typed-htmx:import "typed-htmx";

declare global {
  namespace Hono {
    interface HTMLAttributes extends HtmxAttributes {
    }
  }
}

What is the expected behavior?
When working with '.tsx' files, hx- attributes should be available.
What do you see instead?
Typescript declaration merging failure.
Additional information
Was working on v4.3.

I found out that this is due to the Hono intrinsic namespace not being declared globally nor exported from the hono/jsx module.
A workaround would be importing the https://github.com/honojs/hono/blob/v4.4.4/src/jsx/intrinsic-elements.ts manually and declare it:
import type {} from "../node_modules/hono/dist/types/jsx/intrinsic-elements";
import type {} from "typed-htmx";

declare module "../node_modules/hono/dist/types/jsx/intrinsic-elements" {
	namespace Hono {
		interface HTMLAttributes extends HtmxAttributes {}
	}
}

Is there a plan to export the intrinsic namespace?

Looks like the scope needs to be reduced to intrinsic-elements, fixed by
130a131,135
>     "./jsx": {
>       "types": "./dist/types/jsx/index.d.ts",
>       "import": "./dist/jsx/index.js",
>       "require": "./dist/cjs/jsx/index.js"
>     },

Usage:
declare module "hono/jsx/intrinsic-elements" {
  namespace Hono {
    interface HTMLAttributes extends HtmxAttributes {}
  }
}

or https://github.com/honojs/hono/pull/2936
declare module "hono/jsx" {
  namespace Hono {
    interface HTMLAttributes extends HtmxAttributes {}
  }
}

Taking back my opinion, it seems there is a namespace error occurring in JSX as well
This usage isn't working: https://hono.dev/guides/jsx#override-type-definitions

It is working by export the intrinsic-elements declaration types, but ideally we should be able to declare it globally

Hi @yvvki, thank you for raising the issue. This should be fixed.
@EdamAme-x @fzn0x I've created the PR which implements the one way of resolving this #2937

Thanks @yusukebe !


Status bar color is not changing

I am using SystemUiController library to update the system bars color but it working only for navigation bar, not for status bar.
Using this code to update color to black
val systemUiController = rememberSystemUiController()
    val useDarkIcons = !isSystemInDarkTheme()

    DisposableEffect(systemUiController, useDarkIcons) {
        systemUiController.setSystemBarsColor(
            color = Color.Black,
            darkIcons = useDarkIcons
        )
        onDispose {}
    }

Status bar color should be black.
Device:
Samsung S10+
Android version 12

Commented default status bar code from Theme.kt. and it's working now.

Yes. I had commented the Android Studio template status bar color code in Theme.kt file. And then SystemUIController was working fine.

Still seeing this. Also the icons are fading away

Commented default status bar code from Theme.kt. and it's working now.

What do you mean by this?
here's my Theme.kt
status_bar.webm

I believe it's how it is in Android S. Seems to be fine in Android R. I'll have to check with some other versions


Run tests using multiple threads/processes

Description
In GA we should run the threads using multiple threads/processes such that the CI would not take too much to test the PR/merge.
NB: Check the Syft repo on how it is done there and copy the behavior.
Are you interested in working on this improvement yourself?

Yes, I am.

Already solved in this


STORM-1885. python script for squashing and merging prs.

I'm a little on the fence in terms of squashing the commits of others vs. asking the contributor to do so. There are a lot of situations where spreading out a big patch over multiple commits makes sense and makes the history more consumable.
A couple of questions:

How does this preserve authorship in a pull request that has commits from multiple authors?
How would this work with our current branch model? Specifically, applying a pull request to multiple branches.

commits can be preserved in contributors branch as they seem fit but it doesn't help having them in the main repo. As a contributor they know what those commits means but everyone else will doesn't have any knowledge of individual commits and why they made them.
How does this preserve authorship in a pull request that has commits from multiple authors?
It will ask for primary authors and the user who is merging this can input more than one author at the time of merge.
How would this work with our current branch model? Specifically, applying a pull request to multiple branches
It will allow you to choose a main branch and allows you to pick the squashed commits onto other branches as well by cherry-picking.

It will ask for primary authors and the user who is merging this can input more than one author at the time of merge.

That means it removes authorship information. If we tag a squashed commit as coming from multiple authors, we still wouldn't be able to differentiate what code was contributed by the individual authors.
So if I merged a pull request with multiple authors, the result would be a single commit from me with a message listing the contributing authors, is that correct?

"That means it removes authorship information. If we tag a squashed commit as coming from multiple authors, we still wouldn't be able to differentiate what code was contributed by the individual authors."
We won't be able capture this in JIRA in either. I am not sure how much of this is important to have all the commits from each contributor for a single JIRA which in itself is rare unless its a big patch. It does have ability to give each contributor credit in the commit log.
"So if I merged a pull request with multiple authors, the result would be a single commit from me with a message listing the contributing authors, is that correct?"
Yes.  If its important we could squash commits into single commit per contributor.

What's the difference between Spark script vs Kafka script?
Spark script is origin of Kafka and Zeppelin, so unless there're specific improvements, I think picking Spark script is more promising. For example, trunk is often not used for git project but Kafka is using that.

We're using branches which doesn't fully represent current version for branch. So our script should determine version more smart. One way to determine is looking at pom.xml.

@HeartSaVioR  Not aware of spark script. I am ok with using either one or make this one work as we needed.

Actually I was the one which claims separated credits from other project. (https://github.com/OpenTSDB/asynchbase/pull/122)
But there was a strong reason to do so, and I think it's not the normal case we can see it often. As I addressed from mailing list, many big Apache projects already used this approach.
If there're cases which squashing really hurts then we can have exceptional case.

@harshach Yeah, I don't know what things Kafka improve from Spark script so I wanted to see the benefit if you know about it. As I commented earlier, just adopting script doesn't work since we use different branch model (master, minor, bugfix) so it should be fully tested (including JIRA integration) before adopting.

@HeartSaVioR already ran a simple tests. I like it because it allows us to tag the reviewers and additional committers in the tag message and can be picked onto other branches as well. It does work with JIRA as well.

@harshach
I skimmed a bit, and guess determining fix version will not work since branch names we use are different from Spark and Kafka and so on. We can still input them by hand so there's no issue on it but if we fix it to fit for Storm that would be great. (optional)

We won't be able capture this in JIRA either. I am not sure how much of this is important to have all the commits from each contributor for a single JIRA which in itself is rare unless its a big patch. It does have ability to give each contributor credit in the commit log.

From a legal perspective it's very important that we be able to track the provenance of all code that lands in an ASF repository and could potentially be released.
For example:
Bob is a committer. Alice and Charles are not. Alice and Charles collaborate on a patch, both making commits. In the process Charles commits some code that he doesn't have the legal rights to (its proprietary, etc.). Later Bob uses this script to merge the pull request, and squash all the commits. Alice and Charles are listed as authors of the patch, but there is no history regarding how the code that the ASF doesn't have rights to get there. Was it Charles or Alice?
That may seem like an edge case, but one that we should absolutely consider.

@harshach The source of the file is referenced here:
https://github.com/apache/storm/pull/1468/files#diff-da45fe3972445a9f82ef768808dd8853R20
I'd like to get clearance that what this script does or enables is okay before proceeding.

@ptgoetz makes sense. We can make explicit case for not merging this PR if the origin PR has commits from multiple authors and also can be integrated into the tool to not to proceed if thats the case.

I'd really like to go forward with automated tools for developers / committers. What I've stated from dev@ mailing list, many projects already use specific tools for merging, and the merge script originated from Spark is well used for Spark, Kafka, Zeppelin (now TLP).

I'll take a deep look and describe what this script actually does.

Here's my understanding regarding this script.

main()

get latest branch (highest version) from github mirror

it assumes that prefix of release branch is release version

get information regarding pull request and events of pull request
prompt user(committer) to input commit title: default value is title of pull request, and user can replace them
standardize commit title (via standardize_jira_ref)
prompt user to use title as modified vs original
check that pull request is already merged (closed by asfgit)

if it is, check merge commit is fetched to local, and cherry-pick to latest branch assuming user wants backport

prompt user to continue if PR seems to resolve conflicts (by seeing flag from PR information)
print information of pull request, and commit title, and so on, and prompt user to go on
merge PR into target branch of PR: get commit hash afterwards
prompt user to see user wants cherry-pick

if yes, cherry-pick to latest branch

prompt user to update associated JIRA

if yes, resolve issue as fixed with leaving comment

merge_pr()

fetch branch which pull request is referring
fetch and checkout branch which pull request targets (from asf-git)
merge pull request branch with squash option

if there're some conflicts, guide user to fix it and mark as resolved (via git add)

prompt user to input main author: default value is who has most of commits (via extracting authors from pull request branch and sort)
prompt user to input reviewers: can be blank
prompt user to see user wants to list all commits into commit message
construct commit message by

commit title
body of pull request if presented
all authors
all reviewers if presented
user if user resolves merge conflict manually
auto close message for pull request
list of commits if user want to

commit with passing main author as author, and commit message
prompt user to push, or stop
push changes to remote repository (to asf-git)
clean temporary branches
get commit hash and return

cherry_pick()

prompt user to input branch name to cherry-pick: default value is latest branch which is passed from main()
fetch and checkout branch which cherry-pick targets (from asf-git)
cherry-pick with commit hash

if there're some conflicts, guide user to fix it and complete cherry-pick manually

prompt user to push, or stop
push changes to remote repository (to asf-git)
clean temporary branches
get commit hash and return

resolve_jira_issue()

prompt user to input JIRA issue ID: default value is extracted value from commit title
get information of the issue
check status and stop if issue is already marked as 'Resolved'
print information of the issue
prompt user to input comma-separated fix versions

default values are unreleased versions matched to target branch for merge() / cherry_pick()

develop branch is treated as default version

mark issue as 'Resolved' with setting fix versions and leaving comment

We may have to modify lots of part of script since...

We don't have develop branch so all about develop branch should be modified. Spark merge script also doesn't have handling with develop branch since they don't have develop branch, too. Maybe adopting spark script would be easier than adopting kafka script.
Branch policy is not compatible with projects which uses this script. They have branches per version but we just maintain version lines (major, minor, bugfix) so we should do something while determining fix versions from merged branches.
We're having master and 1.x / 1.0.x branches heavily diverged, so there're often two or more pull requests submitted per one issue. (We don't cherry-pick between master and 1.x for storm-core since it's easy to see merge conflict.) It should be tested (at least unit test and integration test) individually, and issue should be closed when all of pull requests are checked in. It means that we're having different merging step which other projects don't have.
We should update CHANGELOG while merging step. Personally I don't like updating CHANGELOG so opened thread for discussion but we didn't decide something clearly.
Commit message will contain body of pull request which is free format for now and tends to be meaningless for commit message. We need to guide contributor to write meaningful information. Thanks for Github we can have body template of the pull request which many projects have been using already.

So without arranging our branch policy and merging step, it will be hard to get merge script fit for us.

@HeartSaVioR Thanks for documenting the script.
"Branch policy is not compatible with projects which uses this script. They have branches per version but we just maintain version lines (major, minor, bugfix) so we should do something while determining fix versions from merged branches."
Agree but than again we need modify either script to get this done . So spark or kafka script doesn't matter much.
"We're having master and 1.x / 1.0.x branches heavily diverged, so there're often two or more pull requests submitted per one issue. (Committers don't cherry-pick between master and 1.x for storm-core manually since it's easy to see merge conflict.) It should be tested (at least unit test and integration test) individually, and issue should be closed when all of pull requests are checked in. It means that we're having different merging step which other projects don't have.
Moreover, commit message hook (closing PR) doesn't work if PR is not against master."
I am +1 on adding unit tests, integration tests to be run as part of the script but have a manual validation from the user to say yes/no to go-ahead with merge. As the unit tests can be flaky there can be false negatives.
"We should update CHANGELOG while merging step. Personally I don't like updating CHANGELOG so opened thread for discussion but we didn't decide something clearly."
Agree with you on this. This extra-step adds unnecessary commits to the log. As long as we update the JIRA fixVersions before the release its easy to generate a change log.
"Commit message will contain body of pull request which is free format for now and tends to be meaningless for commit message."
With the above script one can edit the commit title and it will have JIRA number & title that will give us much more meaningful message about the work done as part of th ejira.
"So without arranging our branch policy and merging step, it will be hard to get merge script fit for us."
I disagree with this. We can have minimal modifications to the script to work with our current branches.

-1
I am generally opposed to this. Most PRs only have a small number of commits and aren't a problem. For PRs with a large number of commits, it's simple enough to ask the contributor to squash their own commits.
I'm not sure I see the benefit in adding another script, which we will have to maintain, in order to do something we should rarely be doing.
Also, I worry that having this script will lead to a sharp rise in totally-squashed PR merges, even when there's not really any benefit (and in fact, loss of authorship info) since some people are likely just going to use the script whenever they're doing a merge.

I'll second @knusbaum's -1. Based on points I made earlier. This has the potntial to automatically destroy code provenance, especially if more than one contributor is involved in a pull request. From a legal perspective, the ASF need to be able to determine the origin of all code changes.
I would recommend any project that uses a variant of this script to double-check with ASF legal that it is okay. I could be totally wrong, but I'd rather play it safe.
I'd also argue that a well thought out series of commits cane make reviews easier. For example, separating maven  build changes from code changes. I'd rather we encourage devs to think out the partitioning of commits themselves and squash appropriately if necessary.

I'm totally +1 to this approach, even though I think script should be modified to Storm's project style.
Like I said to dev@ mailing list, I have been doing reviewing and merging pending pull requests for weeks and months, and it was painful enough to merge and port back to each branches, even though I ignored cleaning up commits.  If I want to clean up commits before merging it should be more painful. CHANGELOG is subject to not in sync among branches, but we need to write it manually because it's hard to filter merge commits to see the change list. (We could just rely on JIRA issues for alternative.)
Regarding commits, I don't want to keep commits like 'kicking travis', 'address review comments', etc. which is not helpful at any chances. For my last 2 years of development of Storm, I didn't  utilize individual commit. If something is wrong with recent merge, we rollback the merge, not individual commit. Squashing commits is widely used strategy and already shows success story to many big projects. Even Github provides the squash merge mode (recently rebase mode too) in GUI.
If we want to merge in squashed commit, it should be done in merging process, not reviewing process. For me, ideal review process should be contributor-friendly. While we can't put efforts to only maintain Storm project (by reviewing pull requests, etc.), contributors also can't. Once we create a script which also squashes the commits, we don't need to make contributors bothering with rebasing and squashing commits. If not, all individuals including us should do it just after merger said 'please rebase and squash in order to merge.', which is also bothering for mergers, too. Moreover, there's a chance for contributors to be busy at the moment, and pull request goes stale. Pull requests which need upmerging are the case.
I understand and agree the authorship issue, but we can treat it as exceptional case. Many pull requests are authored by one.
Let's make merging phase as painless, or at least less painful thing.

I'll also point out that the "if other Apache projects do it, it is oaky" stance is particularly dangerous. PMC members must understand ASF policy and not rely on what other projects do. If what another project does is wrong, then doing the same thing in our project just introduces liability.

I'm okay with automating the merge process, just not the way it is implemented here. Perhaps we shouldmove the discussion to the dev list.

@knusbaum maintaining the script shouldn't push us from adopting a better approach. It's another piece of code that we need to maintain just like entire repo that we are maintaining right now.
Lets continue this discussion over mailing list.


[NTR]enteroendocrine cell of appendix

Note - please check that the term does not already exist (check OLS: https://www.ebi.ac.uk/ols/ontologies/cl)
Preferred term label:
enteroendocrine cell of appendix
Synonyms
enteroendocrine cell of vermiform appendix
enteroendocrine cell of appendix vermiformis
Definition (free text, please give PubMed ID in format PMID:XXXXXX)
Enteroendocrine cell that is located in the vermiform appendix. Enteroendocrine cells are scattered throughout the epithelial layer of the gut wall and produce gut hormones. Enteroendocrine cells are in a continuous state of cell turnover, being replaced by a cycle of local stem cell division that gives rise to all the epithelial cell types, including enterocytes, mucus-producing goblet cells, enteroendocrine cells, tuft cells and Paneth cells. Intestinal enteroendocrine cells are embedded within a polarized absorptive epithelium and have an apical membrane facing directly into the gut lumen and a basolateral surface facing the interstitial space, giving rise to the term ‘open-type’ enteroendocrine cell. PMID: 30760847
Parent cell type term (check the hierarchy here https://www.ebi.ac.uk/ols/ontologies/cl):
intestinal enteroendocrine cell http://purl.obolibrary.org/obo/CL_1001516
What is the anatomical structure that the cell is a part of? Please check Uberon: https://www.ebi.ac.uk/ols/ontologies/uberon
vermiform appendix http://purl.obolibrary.org/obo/UBERON_0001154
Your nano-attribution (ORCID)
0000-0003-3440-1876
Any additional notes or concerns

@sfexova
CL_0009030 (enteroendocrine cell of appendix)


Composable, extendable and customisable decks

Hey there, thanks for the update! Looks like you're making good progress. 🚀
I've been thinking about Brain Brew and what it could bring to Ultimate Geography in the long term, so I thought I'd share some long-term ideas. (@aplaice, this might interest you as well.)

The problem
Over the years, users of UG have asked for a number of extra fields to be added to the deck -- most notably IPA and audio pronunciation, but also languages, currencies, flag descriptions (vixillology), links to Wikipedia, etc.
These requests have so far been rejected or put on hold for various reasons, ranging from lack of user interest to maintenance burden (single CSV, long-term support, etc.)
IMO, the key to maintaining a good-quality open source project in the long term is to avoid scope creep - i.e. doing a few things really well rather than doing every possible thing poorly. So I've always been careful about keeping the deck as simple and uncluttered as possible. I prefer to have a vast majority of really happy users rather than to try-but-kinda-fail to satisfy everyone.
Brain Brew will obviously solve the CSV limitation, but the other issues remain... If a field has strong user interest and people are willing to maintain it in the long term, it would make sense to include it in the deck's repository, but then including the field in one or more note templates can still be problematic as it impacts every user. And this still leaves fields that don't have strong user interest or guaranteed long-term support...
If I understand correctly, the Personal Fields feature you've been working on will at least let users add more content to their deck while still allowing them to re-export or re-import it. This is a great step forward, but eventually, we'll need a leap, and I think Brain Brew can make that leap.
What we need
What we need is a way to make decks fully composable, extendable and customisable.
Users need the ability to compose, extend and customise a core deck in whichever way they want without having to actually modify that deck.
Let me start by explaining what I mean by those terms.
Composable
This is about allowing users to create, modify and remove templates from a deck.
A basic example of composability is UG's extended deck (confusing... 😄), which combines fields in two additional templates.
But there are plenty more use cases for composability. Who knows, perhaps somebody would like to have a template with capitals on the front and flags on the back? 🤷 Or perhaps they're sick of seeing Flag similarities and want to remove the field from the flag templates. Or perhaps they'd like to turn the Country info field into a hint field?
Going even further, perhaps a user has extended the deck with another field (see next section) and they'd like to create a template with this field, or modify one of the templates to include the field as additional information?
Extendable
This is about adding new content - i.e. fields, for instance currencies or IPA. Pretty straightforward.
Customisable
This is about customising the styles of the templates (font, color, spacing, etc.) People may find some fonts easier to read, or they may want to add icons to help them understand the cards faster.

A note on translated decks
Translated decks are perfect examples of both extendability and composability: new fields that are combined into new templates. (Technically, the templates in UG are the same for all languages, but that's just because Anki DM doesn't support translating templates.)

How do we get there?
Extension repositories
The way I see it, the first thing we need is a way to store "extended" fields in separate repositories. Yes, perhaps even translated fields!
The advantage of having "extension" repositories separate from the main deck's repository is clear: each "extension" repository would be free to live its life, to have its own maintainers, its own opinions (e.g. stylistic choices, simplified vs traditional Chinese characters, etc.), and so on. If an extension repository were to no longer be maintained or to become too opinionated, people could fork it, create an alternative version of it, or simply stop using it. Either way, the core deck would remain unimpacted.
Deck configuration
Then we need a deck configuration schema that allows:

referencing a core deck's repository,
referencing any number of extension repositories,
adding/removing/replacing the core deck's templates (either with custom templates or with templates provided by an extension repository),
customising the core deck's styles.

Ideally, the configuration schema would be "Brain Brew-agnostic", in the sense that it would not contain configuration specific to Brain Brew. Some sort of high-level format that could be processed by any tool.
People would share their deck configurations in their own repos, in Gists, or whatever.
Brain Brew support
Obviously Brain Brew would need to support this deck configuration format. 😄
Tooling
Users would then need an easy way to generate decks from shared configurations.
Installing Python, setting up Brain Brew, copying a deck configuration file, running the build command, importing in Anki, etc. It's way too much for most users. We need tooling to help, perhaps a desktop app. That's long, long term... 😄

What do you think?
First, what do you think about the general idea/plan/whatever the hell this was...? 🤣 Am I making any sense? Do you think extension repositories are the way forward? It felt good to try to identify which issues extendability, composability and customisability could resolve for us deck maintainers...
Obviously, this whole thing makes no sense if Brain Brew can't support it. It's already built for composability, so I guess the complexity is more around extendability. Do you think referencing and pulling content from other repositories is doable? I mean, the idea would really be for Brain Brew to run potentially outside of the core deck, in a folder without any content but a configuration file. Is this even plausible?

Firstly, @ohare93 I'm extremely sorry that I still haven't had time to have a look at and test Brain-Brew!

@axelboc I think that composability, extensibility and customisability would all be extremely useful.

Do you think extension repositories are the way forward?

I think that they're a really interesting idea and could be great! I'm not sure if they're the way forward, but they seem better than the main alternatives I can think of (having all the "alternatives" in the core deck, as "optional" data/configurations, or of having the alternatives as forks of the core deck, that'd have to be constantly rebased after any update in the core).
Obviously I have no idea whether they'd work with Brain Brew. (I also don't want to pile even more pressure and expectations onto @ohare93...)

Tooling
Users would then need an easy way to generate decks from shared configurations.
Installing Python, setting up Brain Brew, copying a deck configuration file, running the build command, importing in Anki, etc. It's way too much for most users. We need tooling to help, perhaps a desktop app. That's long, long term... smile

In principle, it could mostly work as an Anki add-on, since Anki presumably bundles most of the relevant Qt libraries needed for a functional GUI.
However, as you say, it's very, very long term, and probably out-of-scope here since it'd involve modifying/replacing/merging with CrowdAnki.

Wow, what an extensive and well thought out first issue for my little project here! :wink: :ok_hand: lots to reply to, and all of it good (even the :pie: in the sky parts! :smile:) I am usually not on the receiving end of 1000 word paragraph replies, but I am happy to receive such! (And apologies for the length of my own reply :sob:)
Replies to random parts

IMO, the key to maintaining a good-quality open source project in the long term is to avoid scope creep - i.e. doing a few things really well rather than doing every possible thing poorly. So I've always been careful about keeping the deck as simple and uncluttered as possible. I prefer to have a vast majority of really happy users rather than to try-but-kinda-fail to satisfy everyone.

I fully agree :100:%! Though I have desired more in the past (Edinburgh of the Seven Seas :sob:) and still argue that the extended little islands are relevant, I can concede that having these as part of an offshoot deck as part of this extendable project you propose is a much better solution in the end :+1:

Extension repositories
The way I see it, the first thing we need is a way to store "extended" fields in separate repositories. Yes, perhaps even translated fields!
The advantage of having "extension" repositories separate from the main deck's repository is clear: each "extension" repository would be free to live its life, to have its own maintainers, its own opinions (e.g. stylistic choices, simplified vs traditional Chinese characters, etc.), and so on. If an extension repository were to no longer be maintained or to become too opinionated, people could fork it, create an alternative version of it, or simply stop using it. Either way, the core deck would remain unimpacted.

A great idea, and one that I have been dabbling in myself lately. Have you considered using git submodule to add a git repo to a git repo? I will soon start using this to separate my Danish and Esperanto language learning cards into their own repos, but combine / separate them using Brain Brew in a parent repo. This way the data is silo'd and separate, while still being usable for Brain Brew (it is as long as Brain Brew can find the given filepath to read/write!)
This type of style could work for what you are proposing, but may not be necessary if the future goal of a version of Brain Brew that can read directly from the repos themselves is achieved.
Federation of UG

:smirk:
In this hypothetical future, where there are a small Federation of UG deck translations, extensions, and improvement repositories, would you consider creating a Git Organisation for all of the "official" ones to live under? What to deem "official" is up for debate, though I'd argue that translations of the core deck easily fit. An organisation would have many benefits:

Easier for people to find translations / extensions / improvements (any other categories? :sweat_smile:)
Can assign "owners" of the repos (such as the people doing the translating/updating) and simply "@deckowners" or some such for a breaking change to the core deck, rather than having to have an ever expanding list
Easy permission system for trusted users to be able to fix issues on any/all repo
More community feeling and cohesion. When a change is made to one deck, other organisation members are much more likely to see than if we're a bunch of satellite randos not caring what goes on where. If that change impact their deck, or is a better way to do things, then people can react :+1:
Central place to discuss potential offshoot decks/ideas, or meta changes to all the decks in the Projects panel of an Organisation. Would also keep the issues of the Core deck neat and tidy.

Just a thought I had from you proposing this, not a deal breaker :grin:

If I understand correctly, the Personal Fields feature you've been working on will at least let users add more content to their deck while still allowing them to re-export or re-import it.

It will allow any user of CrowdAnki to select any field in any import as "Personal" so that the value will not overwrite an existing entry. The value of a "Personal" field will still be taken for new cards though.
In this way deck managers can choose to add a default value to a card, by setting that value or not in their data in Brain Brew. Also they can will soon be able to set a field a Personal on the CrowdAnki import file itself, so that it auto defaults to that on import. The user can still change it to not Personal, but this will minimise the accidental incorrect import. (This feature cannot be implemented in Brain Brew until it is pulled into CrowdAnki first)
More info available on my Patreon post here. (I don't shill in your repo, but I sure can shill in mine! :wink:)

Needs
Alright, let me summarize the things you believe are needed to make a great system going forward. Though I will remove the categorisation of "Composable, Extendable, and Customisable" in favour of my preferred way to look at this: Deck Parts.
In order to be able to untangle this mess of scope and create a more Federated approach to Anki deck moderation you state (and I agree) that we will need to give the users full choice over the following items upon Import:
1. Note Models

Card Types (aka "templates" as they contain the front and back template). "Composable" in your system.
Note styling. "Customisable" in your system.
Note Fields. "Extendable" in your system.

For any of the card types users should be able to:

Create
Modify
Remove

:exclamation: Apart from Note Styling! The css here is used for every Card Type on a Note type. Though I do have a general idea about programmatically being able to set styling separately on each card by the use of ".card1" anki css and simply combining all the individual parts to make a whole, but it is ofc much simpler to just use the one css as that is how Anki is designed :sweat_smile:
2. Notes
Users should be able to take extra notes into the deck, plain and simple. If  someone wants to extend the deck with their offshoot repo of American states but still fit into the Ultimate Geography framework Ultimate Geography then that should be possible.
The deck as it currently stands could even be broken down into categories like Independent Countries, Territories, Seas/Oceans, etc :thinking: Users could then have an option for which to include. Though this "chose what you want to import" functionality will be covered in one of my PRs for CrowdAnki to filter the import based on the tags :+1:
3. Media
Notes encompass most the media of the deck. Though media can also be referenced and used in a Note Model (in card template or styling) as long as the file begins with an underscore. Not naming it so will result in a working card, but the media file will not be referenced correctly and thus will not export with a note and may be found in the "Check Media" option as unused.
Regardless of where the media file is referenced, it is worthwhile to mention that it would be good to be able to replace the media of the deck with other media :thinking: This would be an option taken by the guy just today who raised the issue about media format for maps not being SVGs :sweat_smile: and I'm sure there are other use cases.
4. Deck Headers
The stuff that makes a deck itself: the name, config/review options, and description. It would be good for others to be able to translate the deck description and name. However I am currently trying to push for these to be optional default values used only if you do not already have the cards. I improved this already with the "Do Not Move Existing Cards" PR to CrowdAnki, however if new cards are imported then the Deck Headers are used to make a new deck with which to place them. Again, I hope my Import Options CrowdAnki PR can fix this issue by giving the user the option of where to put the cards, if no Deck Headers are desired/available.
I hope that in the future a CrowdAnki deck could be imported without any Deck Headers and that the user would just be prompted on import what the do with the new and existing cards. I am working on that as an option in CrowdAnki just now.

Brain Brew's Current Functionality
I am very happy to report that each of the requirements stated above are already functional in the current version of Brain Brew :party:
Now let me list the caveats :laughing:

Note Models are currently all or nothing: each is packaged with the fields, styling, and templates. Though technically one can achieve the above requirements, but there would be massive duplication of data for small changes :sob: This is a current top priority (see Ongoing Work below)
Csvs are the only non CrowdAnki format currently supported (Yaml on the way).
Media from Note Models is not fully functional yet
Media is not settable per deck generation yet

And of course all that says nothing about the fact that one must still do this from the command line :sweat_smile: though if one were to create a few split up repos of UG, as discussed in the Federation section above, and simply have them locally on your computer then currently you could use Brain Brew to read data from and/or write data to each to build and manage a deck!

Obviously, this whole thing makes no sense if Brain Brew can't support it. It's already built for composability, so I guess the complexity is more around extendability.

Extending generated decks with more fields is already working :+1: See this Yaml Builder Config file here. If one wished to add more notes then simple add another "csv_file_mapping". To add more fields add another "derivative" under that, and make sure each of the fields in your csv are listed in "csv_column_to_fields" under "note_mode_mappings" has a partner matching field (or put it in the "personal_fields" list. See my Brain Brew Starter Project for a working version with comments.
I realise all of this may not have been clear of obvious due to the complexity of the task and my unclear communication about it in the past. I have invested some time into the documentation (though full code documentation will be later as major refactoring is occuring now), so if there is anything else that is unclear do let me know and I will try and make a writeup to explain it :+1:
Ongoing Work

Personal Fields [Waiting for PR in CrowdAnki to be accepted]
Refactoring Deck Parts into Yaml files, and minimising their requirements (no file written if not needed)
Yaml Source file (easier after the above is complete, and more complicated than you would have originally thought!)
Note Model generation and extraction

After these key tasks are done I will most likely invest some time into a proper demo of Brain Brew, as I have not had an official first publication, outside of a disjointed Discord chat and posting on the UG thread :sweat_smile: The above changes will make things muuuch easier for others to understand and contribute their own changes to the project, and I have high hopes that the lofty goals will attract likeminded helpful souls who simply wish for their specific Source type to be implemented and so they do it themselves :grin:

Brain Brew in the future
@aplaice

In principle, it could mostly work as an Anki add-on, since Anki presumably bundles most of the relevant Qt libraries needed for a functional GUI.

Brain Brew could indeed be an Anki add-on, as I have made it quite minimally dependent. I also (from adding the visual config window to CrowdAnki) have some experience with making the UI itself, and could simply generate the current builder config files from a ui options, instead of the user filling them out beforehand just now. This would maintain the use of Brain Brew as a Maintainers command line tool, while allowing normal users to benefit too.

However, as you say, it's very, very long term, and probably out-of-scope here since it'd involve modifying/replacing/merging with CrowdAnki.

I do not foresee a full replacement of CrowdAnki. It just does too much. Though if in the future there comes constant challenge/denial to future PRs going through then I suppose I could see myself forking and building ontop of that :thinking: begrudgingly... :disappointed: CrowdAnki does have an MIT license, so that wouldn't be an issue.
I would much prefer a model where Brain Brew is it's own add-on, but has a dependency on CrowdAnki also being installed. In this system Brain Brew would simply call the functions of CrowdAnki using Anki Hooks. So if a user chose to import a deck in some way using Brain Brew, then it would do all the transformation of Csv/Yaml/etc into a CrowdAnki file in a temporary folder, and behind the scenes CrowdAnki would be called by Brain Brew with that file as the target. On cleanup the temp files would be removed. Similar for export into one of these files, Brain Brew would instruct CrowdAnki to make the json file in a temp repo, and then it would work it's magic to convert to the other location.

Tooling
Users would then need an easy way to generate decks from shared configurations.
Installing Python, setting up Brain Brew, copying a deck configuration file, running the build command, importing in Anki, etc. It's way too much for most users. We need tooling to help, perhaps a desktop app. That's long, long term... smile

If I can implement the above changes, and have Brain Brew native in Anki then such a future is indeed possible! :grin: at that point each deck or offshoot need only provide a working file of any format that Brain Brew can read, and it will combine them all together as stated in the config file :+1: I hope we can get there together!
Random ideas

Translate Note Models by replacing the css styling only, and simply replacing the content of specific containers with ids? :thinking:
Use TiddlyWiki as an interactive source format option? (I am in love with TiddlyWiki now) This would be instead of (or together with) Csv/Yaml.

It is free
Is just a html file, and can he hosted on Github pages
Can be added to or taken from with a REST api
Could allow for easy linking of related countries, and the like

Summary
Good ideas mate @axelboc I am fully onboard! To infinity and beyond! :grin:
@aplaice

Firstly, @ohare93 I'm extremely sorry that I still haven't had time to have a look at and test Brain-Brew!

No worries, mate. I'm taking my time, so feel free to take yours! :grin:
At present I am aiming for 8 hours of Brain Brew work a week, during the weekends, as I have a full time job. I hope by my shitty Patreon page you can see I am indeed committed to the project :grin:
And though I seriously greatly enjoy writing these fucking biblical length replies, as it greatly clarified my own thoughts on these topics, it does sap a few hours of precious time I could be using to code! :sweat_smile: :laugh: so let's all agree to smaller replies going forward? :wink: on that note, feel free to open any further more specific Issues on this repo, even if they are just questions about a specific topic or functionality. There aren't any users yet, so there'll be no pollution of the issues space! :grin:

let's all agree to smaller replies going forward?

I'll do my best... 🤣

Have you considered using git submodule to add a git repo to a git repo?

I've never used submodules, so I'm not hugely familiar with them. The idea is to get the content of all the child repos in subfolders when you clone the parent repo, is that right? I guess it would simplify things for Brain Brew, but I feel like it wouldn't be very flexible.
Perhaps it's a bit too far-fetched, but I picture more something like a package manager: you specify which repos you're interested in and it downloads them in a temp/untracked folder.

Actually, scratch that, the ideal scenario for me would be to no longer need a deck manager in the UG repo at all! I know, crazy 😄 With inspiration from some of your comments, here is what I'm envisioning:

UG and each extension/translation repo includes a YAML file at the root describing the repo in a way that Brain Brew understands (so it knows what the repo contains and where to get things).
Brain Brew comes as an Anki addon that allows importing a Francken-deck from a config file (or GUI). The add-on's import config allows specifying which core repo and which extensions to import and how.
On import, Brain Brew downloads the content of the repos specified in the import config/GUI, does its own thing to generate a CrowdAnki JSON file, and hands over the file to CrowdAnki to perform the actual import (as you've suggested could work).
As the owner for the UG shared deck, my process no longer involves running a command within the repo itself and importing the generated ZIP file in Anki. Instead, I use the Brain Brew add-on with a very simple config to directly import the core UG repo (i.e. the standard, English deck).
I then publish the shared deck on Anki Web as usual, and then export the deck as an APKG and attach it to the release notes on GitHub.
I no longer attach ZIPs for the extended decks and translations to the release notes. It is up to users to import those decks themselves with the Brain Brew add-on (which is much simpler for them too, since they don't have to download/extract/import the ZIP).
To make it easy on users to import popular alternative decks, we publish pre-made import configurations in the core UG repo and list them in the repo's root YAML file. When a user enters the URL of the repo in Brain Brew, the add-on auto-populates a drop-down menu with the pre-made import configs.
The add-on keeps a history of the import configs used, so users can upgrade decks easily. (We could even imagine notifying the user when an update is available.)

In this hypothetical future, where there are a small Federation of UG deck translations, extensions, and improvement repositories, would you consider creating a Git Organisation for all of the "official" ones to live under?

YES, yes, a hundred times yes! :100: It's been in my mind for a while now, and this is what actually triggered me to write this issue.
I want the core deck to remain highly stable and "protected", but I don't want this policy to get people's way! I want UG to become an ecosystem with a life of its own! If the vision I shared above comes to be (at least in part), then a GitHub organisation will be the next logical step, for sure.

The deck as it currently stands could even be broken down into categories like Independent Countries, Territories, Seas/Oceans, etc

Totally, since it wouldn't prevent me from importing all of these "core categories" in order to publish the standard English shared deck.

I'll do my best...

:grin: failure is highly probably, but we can at least try! :laughing:

Actually, scratch that, the ideal scenario for me would be to no longer need a deck manager in the UG repo at all! I know, crazy  With inspiration from some of your comments, here is what I'm envisioning:
...

That is a bold future indeed! :grin: I had not even considered that as a possibility, as Brain Brew is a management tool in my eyes. But if each user has access to it in an easy pre-configurable fashion then individual deck generation as you stated is quite simple :exploding_head: I think you may be onto something here, and that may make me consider an native Anki add-on sooner than I expected :thinking: "Sooner" != soon though :sweat_smile: at least not without help

YES, yes, a hundred times yes!  It's been in my mind for a while now, and this is what actually triggered me to write this issue.
I want the core deck to remain highly stable and "protected", but I don't want this policy to get in people's way! I want UG to become an ecosystem with a life of its own! If the vision I shared above comes to be (at least in part), then a GitHub organisation will be the next logical step, for sure.

Noice :+1: So the work you and @aplaice have been doing to trim down the UG deck will hopefully become the new-UG federation's core deck :grin: Perhaps if this is the goal do you still plan on releasing the trimmed deck as the next version? :thinking: though this eventual goal may take a while to realise, perhaps it is worth the wait just to skip all the issues with people desiring different versions? :sweat_smile:

Perhaps if this is the goal do you still plan on releasing the trimmed deck as the next version?

Yes, I'm hoping to work on the pre-release this week-end. I'm open to suggestions as to what to do with the removed cards, though. I'll continue the discussion in https://github.com/axelboc/anki-ultimate-geography/issues/307

I think you may be onto something here, and that may make me consider an native Anki add-on sooner than I expected

Awesome to hear, by the way. 😊 Looking forward to helping out where I can!

Just thought I'd clarify that this whole ideal future discussion doesn't change the fact that we should most definitely replace Anki DM with Brain Brew in UG as soon as you think Brain Brew is ready for it. IMO, the crucial short term goal is still to make translations more manageable.

Just thought I'd clarify that this whole ideal future discussion doesn't change the fact that we should most definitely replace Anki DM with Brain Brew in UG as soon as you think Brain Brew is ready for it. IMO, the crucial short term goal is still to make translations more manageable.

Sounds great, mate :+1: I assume you are still not particularly keen on split csvs though? :smirk: if so, I will wait until Yaml is working as a source option :+1: if not then I can probably make the PR next weekend. With the caveat that template changes will need to be made in Anki until I implement card template generation.
Perhaps at least an in-progress PR for that you and others could play with for a while to see how it works and try to find if there's anything that's missing or broken? :slightly_smiling_face: I will try and document things as best as possible, but am also open to a call or live chat at that stage to walk whoever through the generation process, and answer any "what if I wanted to change X" questions :sweat_smile:

I think it's worth waiting for YAML, but a proof-of-concept PR with split CSVs still sounds like a good idea, since I'm sure there's plenty to learn and discuss. Looking forward to it!

I was curious about this issue @ohare93 mentioned. and I have to say...
What a great thread!
I really loved the idea of federated decks, it's a game changer.
Although he said it's a little project, I wouldn't say that. :wink:


Edge's zindex is higher than node, when there exists node overlap

https://codesandbox.io/s/jovial-dawn-lwikq

we can set zIndex  attr of cell to solve it


Support for apiCall with mutation rule in kyverno tests

Problem Statement
I want to make a unit test for a policy that does an apiCall and mutates the gid and uid value based on existing namespace annotation. But since I am doing an API call I am getting the below error:
error while applying mutate policy check-group-id-uid -> resource default/Pod/test-pod
error: failed to mutate elements: variable substitution failed: failed to resolve allowedGID at path /patchStrategicMerge/spec/containers/0/securityContext/runAsGroup: JMESPath query failed: Unknown key "allowedGID" in pathI0713 15:50:22.494538   69866 rule.go:233] autogen "msg"="processing rule" "rulename"="mutate-group-id-uid"
Below is the ClusterPolicy:
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: check-group-id-uid
  annotations:
    policies.kyverno.io/title: Check Group ID and UID
    policies.kyverno.io/description: >-
      Ensures that the group ID and UID in the pod's security context match the namespace annotation range. If they don't match, the policy updates the values to the first allowed UID and GID in the allowed list.
    policies.kyverno.io/severity: high
    policies.kyverno.io/subject: Pod
spec:
  background: false
  schemaValidation: false
  rules:
  - name: mutate-group-id-uid
    match:
      any:
      - resources:
          kinds:
          - Pod
    exclude:
      any:
      - resources:
          namespaces:
          - "core-*"
          - "kube-*"
    context:
    - name: allowedUID
      apiCall:
        urlPath: /api/v1/namespaces/{{ request.namespace }}
        jmesPath: abs(metadata.annotations."openshift.io/sa.scc.uid-range" | split(@, '/') | [0].to_number(@))
    - name: allowedGID
      apiCall:
        urlPath: /api/v1/namespaces/{{ request.namespace }}
        jmesPath: abs(metadata.annotations."openshift.io/sa.scc.supplemental-groups" | split(@, '/') | [0].to_number(@))
    mutate:
      foreach:
      - list: "request.object.spec.containers"
        patchStrategicMerge:
          spec:
            containers:
            - name: "{{ element.name }}"
              securityContext:
                runAsUser: "{{ allowedUID }}"
                runAsGroup: "{{ allowedGID }}"

Solution Description
I tested the policy in a test cluster and it does mutate the value. Seems like I am having trouble with Kyverno test failing due to a mutating policy that is fetching values using an API call. Kyverno doesn't mock or simulate these API calls. Is doing this kind of test  actually possible?  please, let me know.
Example "Good" Resource
No response
Example "Bad" Resource
No response
Other Comments
No response
Slack discussion
No response
Troubleshooting

[X] I have read and followed the documentation AND the troubleshooting guide.
[X] I have verified the policy does not exist in the samples library.

Testing these types of policies requires kyverno test using a values file as documented here.


Add storage explorer link for transfers to Azure Storage containers

closes #4
Add storage explorer link to previous transfer list to visualize results:

Button only present for AzureStorage transfers
Storage explorer link in app.config.json of the form storageexplorer://v=1&accountid=/subscriptions/{{subscriptionId}}/resourceGroups/{{resourceGroup}}/providers/Microsoft.Storage/storageAccounts/{{account}}&subscriptionid={{subscriptionId}}&resourcetype=Azure.BlobContainer&resourcename={{container}}

account and container are retrieved for each transfer process
the rest of variables are provided at provisioning time with terraform

Working sample (click on the completed transfer): http://378-company2-mvd.northeurope.azurecontainer.io/transfer-history-viewer

This seems to require Azure Storage Explorer to be installed - is there a way to generate a link that opens in the browser?

There is the possibility of generating a link with a SAS token with a URL leading to the Azure portal, but this would be a security risk as we can not guarantee the person using the app has permissions to see the transferred files. The storage explorer has authentication on its own, so we make sure only authorized users can see them.

@paullatzelsperger implemented an info box at the top to indicate that Azure Storage Explorer needs to be installed.


Removing snapshots before tests.

Fixes #205
Changes proposed in this pull request:

Delete current snapshots if they are there
Generate new snapshots on tests

appveyor is running on a windows shell :thinking: I did not know this @abahmed any suggestions how to target them all then?


Argocd bundle

This commit add argocd to kairos bundle

[x] chart
[x] cli

[x] download binary
[x] handle multiple architecture

[x] doc
[x] test

[x] test chart

[ ] deploy chart from cloud-config file
Somethig likeflux:
  env:
  # Override default $KUBECONFIG of /etc/rancher/k3s/k3s.yaml if needed
# KUBECONFIG: /home/csagan/.kube/config
    GITHUB_TOKEN: abcde1234
github:
  owner: csagan
  repository: fleet-infra
  path: clusters/cosmos
  components-extra: image-reflector-controller,image-automation-controller

[x] CI/CD

related to https://github.com/kairos-io/kairos/issues/2322

Nice addition @Ludea, thanks for the contribution! Left some comments but i haven't worked much with the bundles so they are more questions than me trying to block the PR

Hoping everything is ok

https://github.com/kairos-io/community-bundles/actions/runs/10778514658/job/29890365571?pr=71#step:4:206 what is wrong ?

Installing the bundle with no values results in this error in the helm install pod:
+ echo 'Installing helm_v3 chart'
+ helm_v3 install --version 2.10.2 argocd argocd/argocd --values /config/values-01_HelmChart.yaml
Error: INSTALLATION FAILED: chart "argocd" matching 2.10.2 not found in argocd index. (try 'helm repo update'): no chart name found

I guess it should match the CHART VERSION from here?:
~ $ helm search repo argo
NAME                      	CHART VERSION	APP VERSION  	DESCRIPTION
argo/argo                 	1.0.0        	v2.12.5      	A Helm chart for Argo Workflows
argo/argo-cd              	7.5.2        	v2.12.3      	A Helm chart for Argo CD, a declarative, GitOps...
argo/argo-ci              	1.0.0        	v1.0.0-alpha2	A Helm chart for Argo-CI
argo/argo-events          	2.4.7        	v1.9.2       	A Helm chart for Argo Events, the event-driven ...
argo/argo-lite            	0.1.0        	             	Lighweight workflow engine for Kubernetes
argo/argo-rollouts        	2.37.6       	v1.7.2       	A Helm chart for Argo Rollouts
argo/argo-workflows       	0.42.1       	v3.5.10      	A Helm chart for Argo Workflows
argo/argocd-applicationset	1.12.1       	v0.4.1       	A Helm chart for installing ArgoCD ApplicationSet
argo/argocd-apps          	2.0.0        	             	A Helm chart for managing additional Argo CD Ap...
argo/argocd-image-updater 	0.11.0       	v0.14.0      	A Helm chart for Argo CD Image Updater, a tool ...
argo/argocd-notifications 	1.8.1        	v1.2.1       	A Helm chart for ArgoCD notifications, an add-o...

I think we should implement and e2e test suite in community bundles (no in this PR though). It would automatically catch many of the issues I discovered. It would also make it easier to test locally.

@jimmykarily I adress all your comments

Even with squash commit I get the conflicting file :s

Seems to be good

Looking good to me, thanks @Ludea !


Issue with Presence Light (Athom Presence Sensor 1.0.2)

Hello,
I'm encountering an issue where the presence_light switch turns itself back on after being turned off when motion is detected. This behavior is observed on multiple devices. I have not found any obvious problems in the code that would explain this behavior. Below is the log showing that the MM-Wave sensor is causing the presence_light to turn back on. I am also attaching my configuration file for reference.

Steps to Reproduce:
Turn off the presence_light.
When motion is detected by the MM-Wave sensor, the presence_light turns back on automatically.

Expected Behavior:
The presence_light should remain off when turned off, regardless of motion detection.

Actual Behavior:
The presence_light turns back on by itself after being turned off when motion is detected.

Additional Information:
This issue occurs on multiple devices.
Attached is the log showing the sensor's action.

Any assistance in resolving this issue would be greatly appreciated.