Inconsistencies between JSON and SMDX models

Here are some observations I've made regarding https://github.com/sunspec/models/issues/62#issuecomment-1288095736:

many (not all) fields no longer have Access: R defined. Should we assume that R is the default?
r/rw has become R/RW
len=1/2/4 for standard types is not defined. Only sometimes it is. Should we assume that len should only be defined on strings? Then it should be removed everywhere else!
the usage of model type/name is inconsistent, both in the old as well as in the new models
ID and L fields have been added to the blocks- but not everywhere?

What I see here is that what should be the same definitions is stored and maintained twice (as json and as xml).
The way I look at this project it should be restructured to 1 single source format and then have code generate all other formats during the build/release.
So a release will have all formats and the source will have only one.
If I can find some time I'm willing to have a look at trying to set that up.
Key questions here:

What is the primary format? I see more structural elements in the XML format so I would expect that to be the source.
Are you guys ok with having this code in Java where most of my experience lies and which is well automatable in terms of building releases and thus generating all other desired formats.

CC: @bobfox @shelcrow

The other problem is that long-standing issues- in any format- are simply not fixed. The xml is outdated and the json is inconsistent and both don't match. Community feedback is not turned into corrective actions. See https://github.com/sunspec/models/issues/62, it is 2.5 years old...

To me this starts to feel like an unmaintained standard the entire solar industry is based upon. Not a nice feeling.

...and that's exactly what you notice when working with various vendor implementations (Kostal, anyone?)...

Sorry for the frustration. I think there are a couple of high level points I'll make now and then come back with more specific details shortly after some review.
A point that I guess is not made explicitly enough is that the XML encoding for model definitions is deprecated. The existing definitions will always be published and available but ideally they will never be updated again. The XML modeling does not support multiple repeating groups which is used in many of the 7xx models. It is not possible to model some of 7xx models with the XML encoding. The intention is to use JSON for all models moving forward.
Once a model definition is complete, it is frozen. Exceptions are made for editorial fixes but any change must be functionally compatible with the existing model. The SunSpec model definitions are produced using the SunSpec Dashboard tool for consistency.
The last modeling push involved extending the modeling capabilities and documenting the modeling requirements. At that time all of the XML based models were moved to JSON and the 7xx models were created. A fair number of people participated in the process and many corrections were made in that effort (see the change log in the model reference spreadsheet). We are currently reviewing whether any of the non 7xx models contain inconsistencies in the JSON version.
There are a few corrections needed to the modeling specification. We are reviewing those as well and will report back with specifics as they relate to the issued raised here.
We are reviewing the all of the relevant elements and will come back with more detail shortly to address the issues you have raised
.

Once a model definition is complete, it is frozen. Exceptions are made for editorial fixes but any change must be functionally compatible with the existing model.

It would really help the community if feedback was given as to which of the identified issues can't be fixed (being frozen) and which ones are addressable. It is quite frustrating to re-discover 2+ year old issues :(

I understand your frustration, sorry about that. I do not want to frustrate people, I want a good base to build my code upon. Both the Json and XML are a tad too unspecified in some cases, the json even has a few things that are simply wrong.
I'm fully fine with you having chosen to freeze the XML and continuing with the Json files. Having only 1 format to maintain is easier.

The Json has the Model ID (ID) and Model Length (L) fields defined as part of the models. SunSpec modbus is essentially a chain of models.

2 registers that are a header of the chain 'SunS'
Followed by repeated

1 register: the model ID (or 0xFFFF/0x0000 as a terminator)
1 register: number of registers of the content
N registers: the content
You can only know the size and interpretation of the content AFTER the model ID and size have been read; so these first 2 registers cannot be part of the model definition as is present in the JSon definition. This is fundamentally wrong in the Json variant.

The current XML definition (although frozen, it IS used) has a few places where is leans upon defaults. For something like access and mandatory it is not so much of an issue, but for len an unspecified lookuptable is needed to figure this out on a per type basis. For many types it is reasonable to assume the number of registers based upon the naming. A bitfield32 field is 2 (16 bit) registers and a float64 is 4 bytes. But types like ipaddr, sf and pad a less clear. The type eui48 is even confusing, you would expect 3 registers (3*16=48) but in reality it is 4 (which is NOT specified in model 11!).

The actual encoding of characters in string remains unspecified, I currently assume ASCII and not UTF-8 or something else.

The last few days I have been making some experimental software to spot the differences between json and xml.
With that I have found quite a few descriptions/labels/notes missing in the Json that are present in the XML; all not very important.
Yet I also found a true bug in the json this way that is most likely created during the migration from XML to json: https://github.com/sunspec/models/issues/240

So I put up 2 pull requests for discussion/consideration (still have to sign CLA).

2 genuine bugs in the current json spec I found while doing these xml cleanups: https://github.com/sunspec/models/pull/241

My proposed way of cleaning up/standardizing the XML:  https://github.com/sunspec/models/pull/242
Essentially: Remove useless default attributes that are defined in the schema as the default. Add a len everywhere because there is no explicit definition available and this copies what is done in the json variant. And because of the way I did it all attributes are in the same ordering everywhere.

About the len; I have seen https://github.com/sunspec/models/issues/64 and looking at the json spec I see that there the len has been added everywhere. So adding it here seems like just making it 'the same'.
If you guys say this should not be added (and thus only a len for strings) then a table with all types and sizes must be made available. I checked https://sunspec.org/wp-content/uploads/2015/06/SunSpec-Information-Models-12041.pdf and most are defined but types like count and eui48 are not.
Note that I see 2 things I would change in the json representation of the spec:

The ID and L are part of the list definition of sunspec, not part of the models. So I would remove the ID and L from every model (i.e. make it the same as what the XML did, which is correct).
Add the explicit offset within a model to each point in the spec to simply make the json spec more robust (which would have avoided the bug I found in the json).

An additional thing I just noticed: The groups (both fixed and repeating) in the json representation of the spec do not have the size (number of registers) of each explicitly specified for each group. I think it should have that.


Use empty CollectionType<T> instead of Option<CollectionType<T>>

While ScyllaDB doesn’t distinguish between null and and empty set, it’s not very ergonomic to represent these collections as Option<CollectionType<T>>. For example, an empty set in Scylla should be deserialised into an empty HashSet<T>, not a None::<CollectionType<T>>.
This would be a breaking API change.

I'll include this functionality in the ongoing deserialization refactor, planned for release 0.14.

@617a7aa Please take a look at #1007. Please confirm that the behaviour showcased by the added test is in line with your expectations.

ACK, looks good to me.


Type checking: invalid method override not detected

To reproduce:
PYTHONPATH=site-lib/ python3 src/runYourProgram.py test-data/testTypesSubclassing1.py

blocked by #33


[FEATURE] User creation flow

User story:
As a new user who wants to provide their data for the first time,
when I access the new supplier portal,
then I'm presented with a registration flow that finishes when I access the confirmation link sent via email.
Acceptance criteria:

first step of the new onboarding flow asks for user details: name/username, email, password
upon submitting this data, an email with a magic link is sent to the provided email address,
by accessing this link, the user is logged in and redirected to the landing page of the new portal.

Developer notes:

Similar mechanism exists for registering with the current Django admin. We should make sure that the generated link has the right URL (redirecting to the new portal, not the old admin panel), as well as that it automatically logs the user in. Probably the content of the email should be updated as well.
Ideally introducing email-based confirmation as a prerequisite to providing new data will improve the quality of the submissions. As an example, this should prevent cases where someone is acting on behalf of a company (and their email address) without their knowledge.

 New registration form: self-serviced user creation


Link previews aren't working with Signal

I came here to file a feature request for link previews, but from the other issues, it looks like it's actually supported and working on other platforms. For some reason, the link previews don't seem to be working on Signal (either Android or iOS), even when link previews are available, and even though they work for Twitter links (both text and images).
Because Signal supports link previews for all sites served over SSL/TLS, and because it works for other websites on Signal but consistently not for psky, I think this is an issue on psky's end.
Example of missing preview before message is sent (psky):

Example of preview before message is sent (Twitter):

Example of preview after message is sent (Twitter):

I wonder if the format that Signal expects is different from the format that psky is serving.
The next step would to figure out Signal's user agent, then curl a tweet with Signal's user agent, and post the response 🥺

here's the signal ios link preview fetcher code, for example: https://github.com/signalapp/Signal-iOS/blob/2ee794f3171d1fbb56a9f5331edc9e82a2434e3f/SignalUI/Utils/LinkPreviewFetcher.swift#L4

Blegh, my usual testing flow (that works with Telegram and Discord) doesn't work here because I need https.......
public-server> nc -l 3029
*dm http://ip:3029 to a friend on signal*

iirc there should be a way to listen on https and get the raw HTTP data.... maybe httpbin?

https://requestinspector.com/ does not work with Telegram or Signal embeds
OpenSSL might work but it'll probably be self-signed and Signal will probably not fetch it, but it might.

I think just WhatsApp works:
curl -v -A "WhatsApp" https://twitter.com/TrustedSec/status/1196503244685684738

vxtwitter does a redirect for WhatsApp user agent, finding that vxtwitter code would be useful.....

Okay, success at original technique by using a Caddyfile as follows:
testing.psky.app {
        reverse_proxy localhost:6968
}
# sudo systemctl caddy reload

nc -l 6968

*dm link on signal*

GET / HTTP/1.1
Host: testing.psky.app
User-Agent: WhatsApp/2
Accept: text/html,application/xhtml+xml
Accept-Encoding: gzip,deflate
X-Forwarded-For: <IP_ADDRESS>
X-Forwarded-Host: testing.psky.app
X-Forwarded-Proto: https

So it's WhatsApp/2.
What does vxtwitter do with that.......

I'm not sure what Signal is doing!
It could be that it relies on the oembed.json that I've commented out, but I'm skeptical.
Requests do hit the python process running psky.app, but Signal must not like the response for some reason.
This also doesn't work:
def generate_html_profileonly_signal(full_path):
    html = f"""

      <meta name="description" content="fooooo">

barrrrrr

    """
    return html

My guess is that Signal gets the full http response from e.g. Twitter, and parses out whatever fields it wants in some way. But I'm not sure how.
The code is here: https://github.com/signalapp/Signal-Desktop/blob/9ad9b4da0f4447876490e2dbc462a2b7316dd128/ts/services/LinkPreview.ts#L333
The logs reporting failure are pretty boring:
WARN  2023-05-17T21:18:00.640Z fetchLinkPreviewMetadata: failed to fetch link preview HTML; bailing
INFO  2023-05-17T21:18:00.640Z addLinkPreview: failed to load preview (not necessarily a problem)

This code does in fact parse it with DomParser inside Signal Desktop's TypeScript:
https://github.com/signalapp/Signal-Desktop/blob/9ad9b4da0f4447876490e2dbc462a2b7316dd128/ts/linkPreviews/linkPreviewFetch.ts#L419-L435
This is probably returning null because my HTML is malformed somewhere?

Yes, here, fetchLinkPreviewMetadata is returning null
https://github.com/signalapp/Signal-Desktop/blob/9ad9b4da0f4447876490e2dbc462a2b7316dd128/ts/services/LinkPreview.ts#LL312C1-L313C1
Because parseMetadata is failing:
https://github.com/signalapp/Signal-Desktop/blob/9ad9b4da0f4447876490e2dbc462a2b7316dd128/ts/linkPreviews/linkPreviewFetch.ts#L366

Found it! Signal expects an og:title. Fixed!
Link previews should work across the board in Signal now. I'll commit the code another time since it's late.
Thanks for the issue and help, @ChimeraCoder and @jasikpark !

yayyyy


Deferring index_view until init_app

If you don't specify the index_view when you init the app straight away you are forced to use the default index_view, updated this to use a deferred method
allowing you to...
from flask import flask
from flask.ext.admin import Admin

from backend.admin.views import CustomAdminView

app = flask()

app_admin = Admin()
app_admin.index_view = CustomAdminView()
app_admin.init_app(app)

app.run()

Any update on this?

Looks like this was fixed in commit https://github.com/flask-admin/flask-admin/commit/831c38f3e2f5ca9b702bd641e94b9e698177773c. Closing


Add dag_drawer to visualization init

So we can do from qiskit.tools.visualization import circuit_drawer, dag_drawer

Summary
Details and comments

The dag drawer requires optional dependencies and therefore is no longer loaded at init.

Closing with the above reason.

Fwiw, I don't really like that answer, I don't think nxpd and friends are really any different from the other visualization optional dependencies. I think this is something we can fix later (like we did for all the matplotlib visualization). But, I don't want to hold up the release on this though. So I'll push up a patch for it later.


Cannot recovery by insert token just before EOF

Because EOF never appears in follows
  canRecoverWithSingleTokenInsertion(
    this: MixedInParser,
    expectedTokType: TokenType,
    follows: TokenType[],
  ): boolean {
    if (!this.canTokenTypeBeInsertedInRecovery(expectedTokType)) {
      return false;
    }

    // must know the possible following tokens to perform single token insertion
    if (isEmpty(follows)) {
      return false;
    }

    const mismatchedTok = this.LA(1);
    const isMisMatchedTokInFollows =
      find(follows, (possibleFollowsTokType: TokenType) => {
        return this.tokenMatcher(mismatchedTok, possibleFollowsTokType);
      }) !== undefined;

    return isMisMatchedTokInFollows;
  }

Hmm, I don't think EOF can be automatically added to all follows set.
Not all parsing rules may terminate with EOF, normally it is only the root one.
Can you try explicitly adding $.CONSUME(EOF) at the end of your root rule?
Does it solve your issue?
    $.RULE("json", () => {
      $.OR([
        { ALT: () => $.SUBRULE($.object) },
        { ALT: () => $.SUBRULE($.array) },
      ]);

      // explicitly consume End Of File.
      $.CONSUME(EOF);
    });

closing, re-open if this workaround does not work.


[ALLUXIO-2341]Improve error handling in AlluxioMaster#startMasters

https://alluxio.atlassian.net/browse/ALLUXIO-2341
Improve error handling in AlluxioMaster

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/12262/
Test PASSed.


Flash plugin flickers on screen

When getting client plugins a flash plugin window briefly flickers onscreen. This is with flash disabled.

+1

+1
Please, can you fix this problem?

Yes I got the same problem too and would like to find a solution.

Please try out the new release (https://github.com/jackspirou/clientjs/releases/tag/v0.2.0). Feel free to reopen this ticket if you still have problems.


Support GRL harness release v27

This includes supporting ser2net in open thread controller, some bug fixes, some improvements to add new test cases as well as some command line new features to filter test cases.

Current coverage is 72.86% (diff: 100%)

Merging #871 into master will decrease coverage by 0.90%

@@             master       #871   diff @@
==========================================
  Files           112        111     -1
  Lines         16407      16398     -9
  Methods        2255       2250     -5
  Messages          0          0
  Branches       1981       1981
==========================================
- Hits          12103      11948   -155
- Misses         3643       3782   +139
- Partials        661        668     +7

Powered by Codecov. Last update 275d22d...801e9a3


remove log_replay_iter and process_batch in scan::log_replay

as titled, leftover code we can clean up. and remove dependency on either!

setup_batch_process is now only called by process_scan_batch. It's maybe okay that it's in its own function, be we should update it to be (maybe) called get_scan_adds and update the comment.

I'm in the middle of prototyping a complete rework of this logic that would depend on this change. Let's pick up naming issues there?


updating gems

This is a stab at updating some of the gems in the gemfile, mostly those that seemed like a safe/easy upgrade.
I wanted to PR my progress so far on this to keep it manageable, but I can keep working through updates on other packages once this much has landed.

Thanks!

@schneems is this fella ready to be closed now that it's merged or is it still hanging out there?

Not totally sure what you mean. The issue auto closes when the PR is merged.

@schneems this was emailed to me, via code triage, as an open issue after the PR was merged.

My day to day workflow is gitlab so I wasn't sure whether an issue could be merged and open (which it can in gitlab). I was just operating off what I was served with the CodeTriage notification.

Clearly the issue on #525 just hates things I've worked on :)


Groups button not working properly on all navigators

Under de Settings section, I can't remove tags from the Groups list.

Thanks for creating the issue. But this github repo is no longer in use.
Zeppelin project moved to it's source repo to http://github.com/apache/incubator-zeppelin and issue tracker to here. Do you mind create an issue https://issues.apache.org/jira/browse/ZEPPELIN ?

Done, thank you very much.
Best regards,
Rodrigo
2015-10-19 14:04 GMT+02:00 Lee moon soo<EMAIL_ADDRESS>
Thanks for creating the issue. But this github repo is no longer in use.
Zeppelin project moved to it's source repo to
http://github.com/apache/incubator-zeppelin
https://github.com/apache/incubator-zeppelin and issue tracker to here
https://issues.apache.org/jira/browse/ZEPPELIN. Do you mind create an
issue https://issues.apache.org/jira/browse/ZEPPELIN ?
—
Reply to this email directly or view it on GitHub
https://github.com/NFLabs/zeppelin/issues/424#issuecomment-149195992.


<EMAIL_ADDRESS>download via aria2 failed

Title says it.
Command line output:

WARN  Scoop uses 'aria2c' for multi-connection downloads.
WARN  Should it cause issues, run 'scoop config aria2-enabled false' to disable it.
Installing 'rust' (1.33.0) [64bit]
Starting download with aria2 ...
Download: 03/15 01:34:10 [ERROR] CUID#8 - Download aborted. URI=https://static.rust-lang.org/dist/rust-1.33.0-x86_64-pc-windows-gnu.msi
Download: Exception: [AbstractCommand.cc:313] errorCode=1 Network problem has occurred. cause:A socket operation was attempted to an unreachable network.
Download: [#8cf0e3 0B/0B CN:0 DL:0B]
Download: Download Results:
Download: gid   |stat|avg speed  |path/URI
Download: ======+====+===========+=======================================================
Download: 8cf0e3|ERR |       0B/s|C:/Users/Jonas/scoop/cache/rust#1.33.0#https_static.rust-lang.org_dist_rust-1.33.0-x86_64-pc-windows-gnu.msi
Download: Status Legend:
Download: (ERR):error occurred.
Download: aria2 will resume download if the transfer is restarted.
Download: If there are any errors, then see the log file. See '-l' option in help/man page for details.
ERROR Download failed! (Error 1) An unknown error occurred
ERROR https://static.rust-lang.org/dist/rust-1.33.0-x86_64-pc-windows-gnu.msi
referer=https://static.rust-lang.org/dist/
dir=C:\Users\Jonas\scoop\cache
out=rust#1.33.0#https_static.rust-lang.org_dist_rust-1.33.0-x86_64-pc-windows-gnu.msi
ERROR & 'C:\Users\Jonas\scoop\apps\aria2\current\aria2c.exe' --input-file='C:\Users\Jonas\scoop\cache\rust.txt' --user-agent='Scoop/1.0 (+http://scoop.sh/) PowerShell/5.1 (Windows NT 10.0; Win64; x64; Desktop)' --allow-overwrite=true --auto-file-renaming=false --retry-wait=2 --split=5 --max-connection-per-server=5 --min-split-size=5M --console-log-level=warn --enable-color=false --no-conf=true --follow-metalink=true --metalink-preferred-protocol=https --min-tls-version=TLSv1.2 --stop-with-process=15976 --continue

But it worked with aria2 disabled

Suppressed, Not relevant anymore


[CSApply] Always try to load arguments constructing object literals

AST rewriter needs to make sure that all of the arguments are loaded
since it's currently possible to pass l-value type as an argument to
object literal (#{file, color, image}Literal).
Resolves: rdar://problem/62927467

@swift-ci please smoke test

@swift-ci please test Windows platform


Wireless crafting terminal and other dimension issues

Issue description:
Wireless crafting terminal doesn't work in other dimensions
What happens:
When in another dimension, using the terminal does not open
What you expected to happen:
Use terminal and access storage system
Steps to reproduce:
1.Setup network transmitter in overworld.  Have wireless terminal linked to terminal in the overworld
2.Setup network receiver and wireless transmitter in another dimension.  I tested the nether and aroma's mining world.
3.Can use wireless terminal in overworld.  Doesn't work in other dimension.  Have placed terminal in other dimension and it does access storage.  Another note, you get no wireless transmitter error when there isn't one.  No error with one.
...
Version (Make sure you are on the latest version before reporting):

Modpack: FTB Beyond 1.5.3
Minecraft: 1.10.2
Forge: <IP_ADDRESS>4
Refined Storage: refinedstorage-1.2.25

Does this issue occur on a server? [yes/no]
On SSP, no server to test.
If a (crash)log is relevant for this issue, link it here:
[pastebin/gist/etc link here]

Both areas have to be chunkloaded

Really wish you wouldn't close this because both areas are chunkloaded.

Would have been a much better response to say that the wireless crafting terminal must linked to a normal crafting terminal on the same network segment your wirelessly connecting to.

That speaks for itself.

Wait... The wireless crafting terminal is not even from RS. RS only has normal terminals.
You are probably using this mod: https://minecraft.curseforge.com/projects/wireless-crafting-grid
In that case, you even made the issue on the wrong issue tracker. -.-

That does appear to be the case.  I apologize, but in my defense the item information doesn't identify that it is from a separate mod.


Ensure client parameters match their definition

When generating client property names in preparers ensure that the
calculated client property name matches the actual property name.

Fixes https://github.com/Azure/azure-sdk-for-go/issues/2174


Adds Cpu usage collection when running on OpenJDK

fixes #2332

Current coverage is 40.62% (diff: 100%)

Merging #2429 into master will increase coverage by <.01%

@@             master      #2429   diff @@
==========================================
  Files          1636       1636
  Lines         57370      57370
  Methods       11161      11161
  Messages          0          0
  Branches       7897       7897
==========================================
+ Hits          23307      23309     +2
+ Misses        34063      34061     -2
  Partials          0          0

Powered by Codecov. Last update b424c45...4c3f300


collapse main categories

categories should be closed by default, so that you have an overview of all main categories at the beginning, and to counteract the issue that sometimes groups are split between several pages. Because of the overview, this also solves #192, since there will be no need to search for a main category anymore

By default, you have to click on the arrow to collapse/uncollapse a group, try to find out if this can be modified

when setting  groupStartOpen: false, group stays closed even when searching for a subcategory, I find this confusing

changing from arrow to anywhere in the group header was easy, but trying to collapse/uncollapse groups when searching proved to be quite the challenge. Sometimes the "visible" property for a group is true even when collapsed, and in these cases show() has no effect.

going with #201 instead


LoggerEventSource should not be internal

I am trying to use EventSource provider and want to use rich data events provided by Keywords.Message keyword. However, the entire LogerEventSource class is internal so there is no way to change the keyword setting from my application.
I noticed it was originally public but changed to internal below:
https://github.com/aspnet/Logging/pull/505
Is there any reason why it is now internal? The comment for the class demonstrated using EventListener on the event source which is not possible when the class is internal.

Sorry I see how it works now it will always write event to all 3 forms by default unless EventListener specifically ask for a certain keyword. They still can't reference keyword by the enum because it is internal but an int should suffice. Closing this issue.


How to get styling to work ?

I can get the form to generate but styling does not load.

JSFE core is headless, all additionnal UI libs are implementation demos for you to copy, extend, modify…
For example, MD-ui is not officially maintained anymore. It's too much work (and not very interesting for me) to support every CSS implementations, which is all chaos in the current state of the front-end.
I recommend to copy Shoelace or System, and hack on it ;)


Feedback

Hey Pieter, I came across your project today out of luck, I know it's barely 24 hours old but it was quite relevant to what I was trying to do and was having a bit of an hard time as you can see here https://github.com/offlinehacker/openvpn-k8s/issues/4
I've tried out your POC with limited success so I thought I'd provide feedback in case it's helpful.
First off, when running ./kube/deploy.sh I got bad substitution on line 144, replacing with literal values allowed me to progress.
To get ovpn_getclient working I had to execute the following docker run -v $PWD:/etc/openvpn -e OVPN_SERVER_URL=tcp://vpn.dev-cloud.sedexonline.com:1194 --rm ptlange/openvpn /etc/openvpn/bin/ovpn_getclient danny1 > danny1.ovpn
I also seem to only be able to randomly make a TCP connection, not sure if that's related to my setup or not.
Finally, although I was able to route to the Internet, I was unable to reach the Kubernetes Service subnet, I did edit the configmap as per the readme, although I'm unsure at this stage if that's relevant.
Think thats pretty much it, happy to assist further if required.

Feedback is great! I totally agree with you that there's actually a whole cloud (pun intended) of problems not covered yet within this project.

First off, when running ./kube/deploy.sh I got bad substitution on line 144, replacing with literal values allowed me to progress.
I'm trying some relatively fresh bash features here (upper/lowercasing variables). What version of bash are you running? It should work with version 4.

I'm happy to help you 1-1 (if you're within the same timezone) getting this up and knowing where friction in using this is. Please ping me on slack (plange), i'm on most kube channels.

Great! I'm in UK so similar timezone.  I'm using a Mac (10.11.4 El Capitan) which comes with BASH version 3.2.57.
I've pinged you on Slack.

Made some small changes to help OSX users still running bash3.2
As discussed, i want to evaluate if i can fit deployment for this tool in helm at a later point so we can do away with this crummy script altogether.

I've addressed the bash script portability issues. Do you need more help or can i close the issue?


Sorting/Copying

First of all, this project is excellent, thanks a lot! I really like it.
Two things (which are probably more or less specific to my setup).

The sorting function doesn't work (it seems that it has sorted in the background, but doesn't display the content. But when I click on a line, that line will be updated with the sorted values)

Copying doesn't work, I receive the following error message:

Traceback (most recent call last):
File "c:\temp\dfgui\dfgui\dfgui.py", line 230, in _on_right_click
unit_x, unit_y = self.GetMainWindow().GetScrollPixelsPerUnit()
AttributeError: 'ListCtrlDataFrame' object has no attribute 'GetScrollPixelsPerUnit'

Do you have any suggestion what the problem could be?
When I start dfgui.show(df), I receive the following error, not sure if it might be connected to this.

Thanks,
Tobias

Very difficult to say, because I'm not a Windows user. But yes, both issues could be related to this warning. Did you find anything on Google for that message? What are your Python and wxpython versions?
The first issue could also be related to the kind of data you are using. Maybe you can come up with a minimal example based on some toy data which reproduces the problem.

windows user here, confirming sorting doesn't work. handy tool . would be perfect with ability to live update via updating csv and sort

hello
great GUI. wanted to say that as well :) thanks for the publication !
i noticed the problem of not refreshing the table view also. and i see it is not my problem alone ..
but i have a solution that worked for me :
just add 'self.Refresh()' in the last line for the function ' _on_col_click' .
i.e:
def _on_col_click(self, event):
...
....
# delete temporary column
del self.df[self.TMP_SELECTION_COLUMN]
    self.Refresh()

Regards.

I was having the same issue but on Ubuntu, with python 2.7.12 and wx 3.0. Took me forever, but I ended up changing the _on_right_click method of ListCtrlDataFrame as follow:
232             scroll_range = self.GetScrollRange(wx.HORIZONTAL)
233             # this is crucial step to get the scroll pixel units
234             #unit_x, unit_y = self.GetMainWindow().GetScrollPixelsPerUnit()
235
236             #col = bisect(col_locs, x + scroll_pos * unit_x) - 1
237             col = bisect(col_locs, x + 1.*scroll_pos/scroll_range*loc) - 1

In a few words, instead of transforming the scroll_pos in pixels to check, we know the length of all columns in pixels, and we know both the position of the scrollbar and the length of the scrollbar in "scroll units", so taking the ratio gives the same result without using the GetScrollPixelsPerUnit method


add basic support for chords and lead sheets

LGTM

-LGTM, indentation errors

Should be fixed now.

LGTM


bugfixadd java error log output

Proposed changes

In the pipline, this spark-connector case has a 4% chance of error, but since there is no error log, the cause of the error cannot be located.
Therefore, an error log is added to facilitate problem location later.

run buildall

run buildall

run external


Areeg

Please fill in the following fields:
Firebase C++ SDK version:
Firebase plugins in use (Auth, Database, etc.):
Additional SDKs you are using (Facebook, AdMob, etc.):
Platform you are using the SDK on (Mac, Windows, or Linux):
Platform you are targeting (iOS, Android, and/or desktop):
Please describe the issue here:
(Please list the full steps to reproduce the issue. Include device logs, and stack traces if available.)
Please answer the following, if applicable:
Have you been able to reproduce this issue with just the Firebase C++ quickstarts (this GitHub project)?
What's the issue repro rate? (eg 100%, 1/5 etc)

Areeg


update Cargo

I hope I did this right. Things still seemed to work in a quick local test.

What I do not understand is why, even after bumping Cargo to 0.37 here, crates using default-run still fail when using cargo tree. @ehuss isn't it the case that with Cargo 1.37, default-run got stabilized?

Oh, seems like the versions are off by 1. :/
that might be worth calling out on the crates.io page.

Thanks!


Having exec working with scripts other than lua

Version 3 supports the exec action, but it is only supporting Lua script.
As per the manual state, the exec action should also support other kinds of scripts or elfs.
The manual entry that describes the exec action: ModSecurity Reference Manual#exec
Here we can find the current code for the exec action on v3 -
https://github.com/SpiderLabs/ModSecurity/blob/d5b93c1013ca9c9a96c75369576b54832e9a2b48/src/actions/exec.cc#L1-L10

Reference to v2 code -
https://github.com/SpiderLabs/ModSecurity/blob/176276a931a6d61db38b908237c9aad1cea0d6ad/apache2/re_actions.c#L2219-L2244

Any news on this?

Hi @gfranco93,
Not yet. Sorry.

Hi Felipe @zimmerle,
do you have any update on when this will be fixed in v3? We would love to introduce ModSecurity but need this to review rules in Setry. Great piece of software!
Best, Moritz

Hey @MrMorice,
thank you!!
Unfortunately, the code is not yet ready. As an alternative, you can try to use: @InspectFile. There is an option to run the operator as a chained rule. Depending on the use case scenario you may be able to accomplish the same functionality.

Hey @zimmerle,
that sounds good, indeed! Thank you very much!


Clarify When User Needs Main vs. Staging API Keys

During our user tests, it was made apparent that it's unclear that you don’t need the Main API key at the same time as the staging API for the tutorial. While this was fixed in #438 with further documentation on the Tutorial page, there still remains some ambiguity as to when you need each API key.
How can we make clearer which API key you need when?
When the user is redirected to the Settings page because the required API key doesn't exist, we indicate directly in the popup which of the API key fields is missing. Maybe we auto-focus that input?

@garrettmflynn I feel like this has been elaborated across various descriptions now - agree?

Yep! This has been achieved at several different levels.


Future Request: WeasyPrint Engine

Hello there
I was evaluating this great project for our needs, however, in the end i choose an alternative, which supports WeasyPrint as Rendering Engine. But (with no knowledge of GOlang at all), I think it would be possible to add another engine for PDF-rendering to this project.
I suggest considering WeasyPrint  as rendering engine. WeasyPrint is written in Python, and supports much more functionality when it comes to requirements that is relevant for printing, e.g. individual page design, @page selectors, Footnotes, ...
Due to lack of time and GOlang knowledge, I'm not able to create a PR for this, unfortunatelly.

Hello @kohlerdominik,
Thanks for the headsup!
I clearly do not have the time, but the project is interesting nonetheless! I could seed a new module using their CLI https://doc.courtbouillon.org/weasyprint/stable/api_reference.html#command-line-api.


Package Receipt not being written to the system

Hi There,
I am creating several user packages, which when installed on the destination device create the user accounts, with the correct settings as expected. The issue I am seeing, is that receipts for the PKG files are not being created, which means I cannot use Munki to deploy them as part of the initial build. Is this expected or a bug I have encountered?
I am using mkuser --username smc_exam --uid 503 --password Sup3rS3cr3tPa55w0rd --hidden --pkg to generate the pkg. The smc_exam account is listed in the directory utility, however pkgutil --pkgs does not list the receipt mkuser.pkg.smc_exam. I also tried by appending --pkg-id  au.edu.qld.stmonicas.smc_exam as the package id when creating the pkg.
This isn't a huge issue, as rather than deploy via Munki I can modify my workflow to deploy it another way, or use a different detection method potentially. Just wanted to raise this in case it is indeed a bug.
Thanks
Gavin

Thanks for taking the time to report this.
But, this is indeed the intended behavior and is mentioned in the --package help info: https://github.com/freegeek-pdx/mkuser#--package-path---pkg-path---package---pkg--folder-path--pkg-file-path--no-parameter-working-directory-

The distribution package (product archive) created will be suitable for use with startosinstall --installpackage or installer -pkg or "Installer" app, and is also "no payload" which only runs scripts and leaves no receipt.

If you are just wanting to detect that the mkuser process was fully completed successfully via package, you could check the "/var/log/install.log" file for the "postinstall: mkuser: Successfully created" lines. Those lines will contain the user type and username, so they will be unique for each user and you could automate that check with grep in a script if you wanted to after manually checking a successful install for the exact line contents to grep for (I would not recommend grepping for the entire line that includes the number of verifications as that number may change in future versions, just grep through to the end of the username which I do not plan on ever changing for that log line).
If there is ever an error, those will be listed in the "install.log" file as well and that success line will not be written.
I don't use Munki myself, does it rely on package receipts to know if the package has already been installed or something?
I personally like that the "no payload" package does not leave a receipt since it means it leaves no "unnecessary" traces other than the user (and the log). mkuser can also be run directly without creating a package which would not leave a receipt anyways, so I also liked that consistency between the different user creation methods.

My apologies. I completely missed that bit when I was reading the docs.
Munki can use package receipts. It can also use other scripting methods to detect whether packages are required / applicable or not. I will use a script to detect whether the user account exists or not for now, before eventually moving it out of Munki into part of the OS deployment.
Thanks for pointing out the error of my ways! :)

Checking that the user account exists should be fine most of the time, but keep in mind that there are things that happen after the initial user creation (depending on the options specified) that could also fail (but shouldn't). So, it isn't impossible for the user account to exist, but some later part of the mkuser process had an issue (but again, errors shouldn't generally happen at all, just pointing it out in the interest of knowing how things in mkuser work).


Release notification server

Purpose
Add release flow for notification server
Changes
Docker compose for local setup of images
Add release target for notification server
Dockerfile created
Checklist

[x] My code follows the style of this project.
[x] The code compiles without warnings.
[x] I have performed a self-review of the changes.
[x] I have documented my code, in particular the intent of the
hard-to-understand areas.
[x] (If necessary) I have updated the CHANGELOG.

Sadly unable to proceed for now because of environment constraints.


MSID: 89366 Version: 2 DOI: 2<IP_ADDRESS>5874

[PLACE MANUSCRIPT AND EDITOR DETAILS HERE WHEN AVAILABLE] See step 3
[PLACE PDF URL HERE WHEN AVAILABLE] See step 7
Step 1. Inform bioRxiv
Who can help: @QueenKraken, @nlisgo, @scottaubrey

[x] Manuscript is in data hub index (https://data-hub-api.elifesciences.org/enhanced-preprints/docmaps/v2/by-publisher/elife/get-by-manuscript-id?manuscript_id=89366)

Step 2. Create preview of manuscript
Who can help: @fred-atherden, @nlisgo, @scottaubrey

[ ] Manuscript is available for preview (https://prod--epp.elifesciences.org/preview/89366/v2)

[ ] Trigger import for staging: https://prod--epp.elifesciences.org/import
[ ] Sync data folder with s3://prod-elife-epp-data/data
[ ] Trigger import for staging: https://staging--epp.elifesciences.org/import
[ ] Sync data folder with s3://staging-elife-epp-data/data
[ ] Pull request created to enhanced-preprint-data
[ ] Post link to pull request below

Pull request: [PLACE LINK TO PULL REQUEST HERE]
Detailed instructions: https://github.com/elifesciences/enhanced-preprints-data#add-a-manuscript
Step 3: Awaiting public reviews and QC
Who can help: Production team

[ ] QC passed or QC not needed

Who can help: Editorial team

[x] Public reviews available on sciety (https://sciety.org/articles/activity/10.1101/2<IP_ADDRESS>5874)
[ ] Manuscript and editor details at the top of this issue (Supplied by Editorial team)
[ ] Reviews returned by EPP - https://prod--epp.elifesciences.org/api/reviewed-preprints/89366/v2/reviews

Example
"msas": "Genetics and Genomics", "Neuroscience"
"msid": "84628"
"version": "1"
"preprintDoi": "10.1101/2<IP_ADDRESS>4241"
"articleType": "Reviewed Preprint"
"status": "Published from the original preprint after peer review and assessment by eLife."

"Reviewed Preprint posted": "2023-01-02"
"Sent for peer review": "2022-10-28"
"Posted to bioRxiv": "2022-11-21" (link: "Go to bioRxiv": "https://www.biorxiv.org/content/10.1101/2<IP_ADDRESS>4241v1")

Editors:
Reviewing Editor
Michael B Eisen
University of California, Berkeley, United States
Senior Editor
Michael B Eisen
University of California, Berkeley, United States

Step 4: Modify manuscripts.json (no PDF)

[ ] Reviewed preprint is published to EPP (https://elifesciences.org/reviewed-preprints/89366)

[ ] Pull request created to enhanced-preprint-client
[ ] Pull request switched to "Ready for review"
[ ] Post link to pull request below

[ ] Request that a doi be registered

[ ] Post link to #enhanced-preprint comment below

Pull request: [PLACE LINK TO PULL REQUEST HERE]
#enhanced-preprint comment thread: [PLACE LINK TO COMMENT HERE]

Instructions to modify manuscripts.json

Visit: https://github.com/elifesciences/enhanced-preprints-client/actions/workflows/publish-manuscript.yaml
Click: Run workflow
Complete the form and click "Run workflow"
A successful run should result in a new pull request at https://github.com/elifesciences/enhanced-preprints-client/pulls
Open the pull request and click the "Ready for review" button to trigger tests
Once the tests pass and you are happy with the changes the PR can be merged

Example pull request: https://github.com/elifesciences/enhanced-preprints-client/pull/334/files
Once the pull request is merged in it should be available a few minutes later.

Request that a doi
Post the following in #enhanced-preprint:
@Fred can you register a doi for https://elifesciences.org/reviewed-preprints/89366

Step 5: Awaiting search reindex

[ ] Reviewed preprint is avaliable on journal homepage (https://elifesciences.org)

The search reindex is triggered once an hour. We need the reviewed preprint to be indexed as the search application serves the journal homepage.

Additional info
If needed, the jenkins pipeline to reindex search can be triggered sooner.
https://alfred.elifesciences.org/job/process/job/process-reindex-reviewed-preprints/

Step 6: Published! Request PDF generation

[ ] Post the link to the PDF url at the top of the issue

[ ] Post link to #enhanced-preprint comment below

#enhanced-preprint comment thread: [PLACE LINK TO COMMENT HERE]
Post the following to the #enhanced-preprint on slack:
@Ryan Dix-Peek please can you generate a PDF for https://elifesciences.org/reviewed-preprints/89366

Step 7: Introduce PDF to data folder and git repo

[ ] PDF is avaliable at https://github.com/elifesciences/enhanced-preprints-data/raw/master/data/89366/v2/89366-v2.pdf

Detailed instructions: https://github.com/elifesciences/enhanced-preprints-data#add-a-pdf
Step 8: Add PDF url to manuscripts.json

[ ] Reviewed preprint PDF is available for download (https://prod--epp.elifesciences.org/reviewed-preprints/89366)

[ ] Pull request created to enhanced-preprint-client
[ ] Pull request switched to "Ready for review"
[ ] Post link to pull request below

[PLACE LINK TO PULL REQUEST HERE]

Instructions to add PDF url to manuscripts.json

Visit: https://github.com/elifesciences/enhanced-preprints-client/actions/workflows/add-pdf-url-to-manuscript.yaml
Click: Run workflow
Complete the form and click "Run workflow"
A successful run should result in a new pull request at https://github.com/elifesciences/enhanced-preprints-client/pulls
Open the pull request and click the "Ready for review" button to trigger tests
Once the tests pass and you are happy with the changes the PR can be merged

Example pull request: https://github.com/elifesciences/enhanced-preprints-client/pull/397/files
Once the pull request is merged in it should be available a few minutes later.

Step 9: Done!

[ ] Kettle is on!

elocations missing from references.


Add built-in compilation of extra symbols

The previous macos build took 2:36 and this one took 6 minutes.
This will allow us to test those generated symbol files but we wont be testing the exact same behavior that everyone else will be getting and I don't think that's good. Sometimes things will work in CppSharp and give us false confidence that it's working for everyone else.
Most constructors / destructors that have been removed are trivially_constructible or trivially_destructible and as I've said before they don't even need to be generated at all.

* The previous macos build took 2:36 and this one took `6` minutes.

The builds serve us, not the other way around. Removing any trivial constructors or destructors would hardly make it faster. But we need this feature and if it slows the build down a little, so be it.

* This will allow us to test those generated symbol files but we wont be testing the exact same behavior that everyone else will be getting and I don't think that's good. Sometimes things will work in CppSharp and give us false confidence that it's working for everyone else.

I don't understand why the behaviour would be different. Users would have the same code and options we do.

* Most constructors / destructors that have been removed are `trivially_constructible or trivially_destructible` and as I've said before they don't even need to be generated at all.

It should be easy to skip these in the symbols, I'll try it.

It should be easy to skip these in the symbols, I'll try it.

We also need to change the codegen to avoid calling them.

* The previous macos build took 2:36 and this one took `6` minutes.

The builds serve us, not the other way around. Removing any trivial constructors or destructors would hardly make it faster.

The build taking so long was a hint that something was being done wrong. Things look much better now that you are using better compiler flags but it's weird how you care very little about making builds 2.5 times slower. Without all the optimizations that we've done last week the build would take about 10 minutes and maybe only then you would notice or care about it. We also need to be aware that there are people with much bigger projects than CppSharp and making their builds 2.5 times slower might be an eternity and not just 4 minutes.

Removing any trivial constructors or destructors would hardly make it faster.

The reason why I want to remove trivial constructors / destructors is because I also want to remove the calls to those constructors / destructors and avoid a managed to native transition that are always expensive and should be avoided whenever it's possible.

But we need this feature and if it slows the build down a little, so be it.

Need is overstated here: the only thing that this give us that couldn't be done by just creating c wrappers is partial support for extending "native abstract classes" in the managed side but as I've mentioned before this is not really an issue because we never had complete support for it and we won't until someone decides to add it and that could take unknown time.

There is no easy way for the user that extends that class to implement it properly since there is no easy way to access the copy/move constructor nor the copy/move assignment operators.
The user cannot implement a pure function from the native side because for this to happen the vtable needs to be modified and  we are not doing that.

You only noticed about 1 of the many problems that these abstract classes have because there was a test giving false positives. I still think that it would have been better to ignore this test and add a TODO because the support for the whole thing wasn't trivial rather than merging the previous commit that is supposed to fix it but the CI couldn't event test so we had no idea whether the fix was fully correct or not.

* This will allow us to test those generated symbol files but we wont be testing the exact same behavior that everyone else will be getting.

I don't understand why the behaviour would be different. Users would have the same code and options we do.

My bad here. I thought that this would only be used internally.

Overall if I had to chose between making builds slower and slowing all of us that are trying to contribute to CppSharp vs adding partial support for something that we have never supported  and none has requested or complained then I'd chose the first one for as long as it's possible.
Here is a suggestion to dodge these build time problems (if they are still a problem): add an option that will allow us to enable or disable these special compiler flags that are required for the symbols of these abstract classes to show up.
This way we can create a specific project that tests those abstract classes and we only would have to use those special compiler flags that slow down the build for a single project and not all of them, thus slowing the build time as little as possible.


add basic type checking using mypy

and fix some minor issues found by mypy
i couldn't resist.
suggested by @jermnelson here: https://github.com/LD4P/ils-middleware/pull/47#discussion_r718948133

the return type changes were all made to address type errors mypy caught.
the __init__.py additions were to address a complaint by mypy where it was unable to find imports.
i'm not sure i understand the 13.7% drop in total coverage.
so far, mypy is feeling slightly less finicky than sorbet (probably the most similar tool i have experience with).


[PRM-153] - Added guidance link

@Chris-Williams-QA Rebased and addressed your silencing comment, thank you!


Separate the batched update DAG into manually and automatically triggered DAGs

Description

Currently we have one batched_update DAG which runs all the batched updates: https://airflow.openverse.org/dags/batched_update/grid. This includes both the batched updates that are triggered by the popularity refresh every month, as well as batched update runs that are kicked off by maintainers (e.g. #4091). Since all the DAG runs show as "manually triggered", this can make it hard to see a history of the latter cases that isn't muddied by the regular, automatic batched update runs.
We should split the batched update DAG into two separate DAGs: automated_batched_update and batched_update. automated_batched_update will be the DAG which is triggered by the popularity refresh and any other similar refresh/update processes we may include in the future. batched_update would serve as the entrypoint for running one-off cleanup or update operations which are triggered by maintainers. Note: The DAG names are split this way because we may want to preserve the existing manual-run history for aggregated reporting down the line, and it's easier to filter out the popularity runs from the existing DAG's history than it is to select runs across multiple DAGs.
Both DAGs would be functionally equivalent for the time being, although we may add new functionality to the manual batched update DAG eventually. This means we should be able to convert the batched update DAG definition into a dynamic DAG which produces both DAGs under different names. The popularity refresh DAGs will also need to be updated to trigger against the autiomated_batched_update rather than the current one.

@AetherUnbound just wondering about this, is there a way (and if so, a difference) to inline the tasks from one DAG into another DAG, rather than triggering a remote DAG and waiting for it?
For example, in https://github.com/WordPress/openverse/pull/4429, it almost seems like there could be some way of calling batched_update_dag.function() to get the tasks inlined into the trim_and_deduplicate_tags DAG, rather than remotely triggering it. If it doesn't work because of the batched update DAG's templated param references, if there was an intermediary function that created the tasks, and accepted each of the params as a real argument, that was then called by inlining DAGs with direct arguments, as well as the batched_update_dag with the template strings... would that work, without needing to separate DAGs and to separate the history of popularity refreshes from the DAG that controls them? I don't know if this is actually better or worse, for what the goal of this issue is, and is mostly a curiosity rather than a specific suggestion. If this is possible, then I'm curious of the trade-offs, from your perspective (no need to go into too much depth either way).
Thinking something like this:
def batched_update(select_query, update_query, *etc, **etcetc):
    ...

@dag(...)
def batched_update_dag():
    batched_update(select_query="{{ param.select_query }}", update_query="{{ param.update_query }}", ...etcetc)

@dag(...)
def trim_and_deduplicate_tags():
    for media_type in media_types:
        batched_update(select_query=..., update_query=..., ...etcetc)

etc for popularity referesh

One thing I can guess is that it's clearly not as well suited for mapped tasks (like the media type loop in the example).
Is it better practice to have separate DAGs like this, rather than putting the tasks into a "parent" DAG? Having a bunch of isolated DAGs triggered by each other has interesting parallels to the actor model of programming, compared to this inlining which might be analgous to some other kind of composition (subroutines?).
Again, just a curiosity, not trying to make a recommendation about this specific issue, and I can move this somewhere else if you prefer to discuss this elsewhere, just let me know!

Thanks for asking this @sarayourfriend! There used to be a more canonical way to do this using SubDAGs, but that approach has been deprecated in favor of TaskGroups. You can read more about each in the documentation linked above.
In my mind, what this approach comes down to is how easy it is for us to access the run history in Airflow. With the trim_and_deduplicate_tags example where the tasks are inlined - once we delete the code for that DAG, the DAG itself will still exist (I believe) in the DAG list. We can remove it from the list too, but doing so would make it much harder to see the batched update run if those tasks were inlined into the trim_and_deduplicate_tags DAG itself. Having separate batched_update DAGs which can be triggered by other DAGs means that we can add or remove those other DAGs and their history as much as we like, without having to worry about preserving the history of the batched runs themselves. This is way more advantageous than having (for example) a dozen different DAGs existing in perpetuity, which have each called the batched update tasks inline and thus need to exist in the DAGs list forever. Hope that makes sense!
To answer your general question though, TaskGroups are indeed the way to go for that sort of thing, even if it's not as useful in this case.

Makes sense, thanks.
I wonder if we are approaching getting a historical record of the runs in a roundabout way (for the specific information we hope to retain and learn from a run history). I know the DAG run history is nice because it just exists automatically and feels implicit, but could the batched update history be recorded elsewhere? If we pushed batched update history to a different location, we wouldn't need to make it so awkward to do manual runs, and it might even be easier to understand the history at a glance (we can give every run a meaningful name, perhaps output interesting information about the number of affected records directly into the history, instead of needing to dig into the logs). It could be a good idea if this history incorporated (or was itself incorporated into) a broader history of transformations to the data model and data set itself.
I also wanted to ask about whether concurrency would need special consideration. Is there any at all? The existing batched update DAG allows 10 concurrent runs. Would these need to be split between the automated and manual dags?

Your question about an execution log actually came up during the monthly priorities meeting this last month, which is what prompted the creation of this issue:

A number of issues around possible changes to the batched update, including splitting manual & automated runs into separate DAGs, increased reporting for manually run batched updates, a document defining how batched updates should be run, and possibly a document which serves as an “execution log” for batched updates (we might be able to use airflow directly for this, especially if we split the runs out).

Splitting these runs out into two separate DAGs is the easiest way, right now, to separate this information and make it simpler to access going forward. There's a difference between the automated, regular updates to the database (e.g. popularity refresh, maybe dead link ETL down the line) and updates that we perform manually to correct something. Having separation of those not only the way we look back at histories but also in Airflow itself would be useful, hence the creation of separate DAGs. I'm not against also having the manual runs documented somewhere else, too!
The concurrency is a good question too. It's hard to manage concurrency in a simple way because we can't be certain which batched updates might be conflicting unless we force batched updates to operate by provider as well and manage concurrency that way (or similar). Given the regularity of the automated updates, and the irregularity of the manually triggered updates, I think having the same concurrency for each set is fine.


Houdini: Publish any ROP node (Generic Creator)

Implement Generic ROP publish from https://github.com/ynput/ayon-core/pull/542

Changelog Description
This PR implements an idea for "lower level publishing" in Houdini. This implement Generic ROP publishing. Just create any Houdini ROP node (or custom Rop node HDA) and publish your product types from it!

This PR adds a Generic Creator:
Currently, It injects the ayon meta data onto ROP nodes to behave as though they were produced by various creators from Ayon. This ensures that validators and collectors are retained without loss.
e.g. using Generic creator while setting product type to pointcache should yield similar/same results as using pointcache creator
Here is an example using labs karma node and make it behave as though it was made by karma creator.
https://github.com/ynput/ayon-core/pull/542#issuecomment-2179333623

Additional info
As part of the Ynput Houdini Workgroup sessions I developed this quick prototype to expose a way to batch publish and ingest files. Consider it more of an exploration of what's possible then a "drop it in production now" ready-to-go solution.
Explainer
Yes, this requires some explanation. Here you go.
https://github.com/ynput/ayon-core/assets/2439881/5f767493-10bd-41f6-b54f-89313a96da33
What I forgot to add is that it currently still relies on detecting what the output files are for a ROP node based on a "file" parm that is often unique in Houdini per node. If anyone knows a way to just query the expected output files for a ROP node (similar to what PDG/TOPs seems to do I'd love to know!) but otherwise we'll just need to expand that list.
However, I also played with the idea of having "custom file list" attributes on the Node that when enabled could override the "Collector" logic and would instead use that list of files as the publish instance's files. So that e.g. one instance could also publish multiple representations. For that, @MustafaJafar did this lovely non-functional 'prototype' but it does get the idea across:

Demo scene file
The demo scene file:
ynts_char_hero_pdg_v012.zip
TODO 📔 ✏

[x] Use $OS as default variant name.
[x] Add "OnCreated" callback to allow a studio to configure which nodes to automatically imprint as 'publishable'. See Fabia's first video here at 08:10 and code here

[ ] This is currently a first iteration - should still be exposed to project settings and allow user configuration.

[x] The product types being an enum with predefined standards per node type.
[x] The comment field being available on the ROP nodes

[x] This does actually already work if you'd add the product type to ayon+settings://core/publish/collect_comment_per_instance ✅ - I guess this is fine for now.

[x] Add "Publish" button directly on the node similar to "self publish" or whatever we had for others
[ ] Allow exposing to settings which node types may need to be "automatically imprinted" on create for a studio to customize, with also settings for allowed product types (the default type, default variant name, etc.)
[ ] Implement good support for multiple representations, e.g. like this and Fabia's second video near the end.
[ ] Make sure this API approach remains a goal of the PR.
[ ] Keep in mind Fabia's usage where their AX Publish Submitter  can submit just the publish of existing files to the farm and create e.g. a nuke dependency job that generates custom previews of the intermediate media, etc.
[ ] The product name is currently an 'exposed field' but should actually be locked off to the user for manual editing and should always adhere to the get_product_name logic of the Creator and the product types so that they adhere to project templates correctly.

[ ] We need to make sure this isn't an expression that is heavy and re-evaluates the whole time. May need to be based on callback on "product type" and "variant" changes.

Known issues / Blockers 📔 🔥

The publisher, creator and create context can't differentiate attributes per instance, as such it can't e.g. differentiate whether "split export and render job" makes sense or not for a particular instance's attribute definitions. See comment
It will only detect the single file defined by the ROP node's output path and does not recognize whether e.g. a USD ROP or GLTF rop may generate other sidecar files (which often also if moved along should also remap paths, etc. so it might be non-trivial to support those out of the box)

This basically makes it useless for GLTF exports, and in many cases useless for USD (if it were to write multiple files which also need remapping) and rendering (which may generate export files, need dedicated Deadline plug-ins to maybe render certain outputs (e.g. Husk) or may generate multiple sequences of files (like AOVs to separate files).

We do not know or detect the colorspace of the output files (there is no 'generic' way for ANY ROP to return what colorspace it may result in.
If certain ROPs may override the output FPS other than the workfile, we do not know it.
Due to Creators enforcing a single product type for which the attribute definitions are populated/created based on the Creator's product type we run into issues with a generic Creator that basically switches product type on the instances dynamically. See comments here.

Testing notes:

Check out the branch
Check out the explainer video
Test the demo scene file
Comment with great ideas on how to improve

Also, it would be great, if the node changed color or added a sticker on top when published, especially since this helps to identify the publish node easily in the scene. Not a very important one, but helps little.

Also, it would be great, if the node changed color or added a sticker on top when published, especially since this helps to identify the publish node easily in the scene. Not a very important one, but helps little.

Interesting request - when would you like the 'clear' this color or sticker? I can imagine it otherwise always having the sticker after the first publish you've done from the workfile.

Adding screenshot of what it looks like now:

Hey!
Just testing this branch, we cannot publish anything from houdini(workfiles/caches etc), somehow its getting conflict, I've attatched the error when we tried to publish workfiles with this branch

Error: The attempted operation failed.
Parameter name 'AYON_self_publish' is invalid or already exists

Traceback (most recent call last):
  File "C:\Users\avril3975\AppData\Local\Ynput\AYON\addons\core_0.4.4-dev.1\ayon_core\pipeline\create\context.py", line 2484, in _save_instance_changes
    creator.update_instances(update_list)
  File "C:\Users\avril3975\AppData\Local\Ynput\AYON\addons\houdini_0.3.12-dev.1\ayon_houdini\plugins\create\create_generic.py", line 393, in update_instances
    self.imprint(
  File "C:\Users\avril3975\AppData\Local\Ynput\AYON\addons\houdini_0.3.12-dev.1\ayon_houdini\plugins\create\create_generic.py", line 641, in imprint
    self.create_attribute_def_parms(instance_node, created_instance)
  File "C:\Users\avril3975\AppData\Local\Ynput\AYON\addons\houdini_0.3.12-dev.1\ayon_houdini\plugins\create\create_generic.py", line 624, in create_attribute_def_parms
    node.setParmTemplateGroup(parm_group)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.547/houdini/python3.9libs\hou.py", line 16733, in setParmTemplateGroup
    return _hou.OpNode_setParmTemplateGroup(self, parm_template_group, rename_conflicting_parms)
hou.OperationFailed: The attempted operation failed.
Parameter name 'AYON_self_publish' is invalid or already exists

Hey!
Just testing this branch, we cannot publish anything from houdini(workfiles/caches etc), somehow its getting conflict, I've attatched the error when we tried to publish workfiles with this branch

Error: The attempted operation failed.
Parameter name 'AYON_self_publish' is invalid or already exists

Traceback (most recent call last):
  File "C:\Users\avril3975\AppData\Local\Ynput\AYON\addons\core_0.4.4-dev.1\ayon_core\pipeline\create\context.py", line 2484, in _save_instance_changes
    creator.update_instances(update_list)
  File "C:\Users\avril3975\AppData\Local\Ynput\AYON\addons\houdini_0.3.12-dev.1\ayon_houdini\plugins\create\create_generic.py", line 393, in update_instances
    self.imprint(
  File "C:\Users\avril3975\AppData\Local\Ynput\AYON\addons\houdini_0.3.12-dev.1\ayon_houdini\plugins\create\create_generic.py", line 641, in imprint
    self.create_attribute_def_parms(instance_node, created_instance)
  File "C:\Users\avril3975\AppData\Local\Ynput\AYON\addons\houdini_0.3.12-dev.1\ayon_houdini\plugins\create\create_generic.py", line 624, in create_attribute_def_parms
    node.setParmTemplateGroup(parm_group)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.547/houdini/python3.9libs\hou.py", line 16733, in setParmTemplateGroup
    return _hou.OpNode_setParmTemplateGroup(self, parm_template_group, rename_conflicting_parms)
hou.OperationFailed: The attempted operation failed.
Parameter name 'AYON_self_publish' is invalid or already exists

Thanks @krishnaavril - should be solved now. It will now log a warning on creation.
It's because it tries to create it twice somehow? you have the creation of self publish button enabled in settings.
But that would only have been the case if you triggered the generic creation on a node that was already a publishable instance. I've changed the code, but would still be lovely if you could share a screen capture of how you get it to the state that it breaks.

Thanks, Yes I do have the self-publish button enabled in Ayon settings.
But that would only have been the case if you triggered the generic creation on a node that was already a publishable instance.
No, I tried this with our inhouse otl, adding the parameter and otl name hardcoded in the ayon code which allows publishing our own cache and render nodes.
Will test these pushed changes today, thanks

I've tested the code with the labs karma node, I think the issue is still there

Error: The attempted operation failed.
Parameter name 'AYON_self_publish' is invalid or already exists

Traceback (most recent call last):
  File "C:\Users\avril3975\AppData\Local\Ynput\AYON\addons\core_0.4.5-dev.3\ayon_core\pipeline\create\context.py", line 1278, in _save_instance_changes
    creator.update_instances(update_list)
  File "C:\Users\avril3975\AppData\Local\Ynput\AYON\addons\houdini_0.3.14-dev\ayon_houdini\plugins\create\create_generic.py", line 392, in update_instances
    self.imprint(
  File "C:\Users\avril3975\AppData\Local\Ynput\AYON\addons\houdini_0.3.14-dev\ayon_houdini\plugins\create\create_generic.py", line 640, in imprint
    self.create_attribute_def_parms(instance_node, created_instance)
  File "C:\Users\avril3975\AppData\Local\Ynput\AYON\addons\houdini_0.3.14-dev\ayon_houdini\plugins\create\create_generic.py", line 623, in create_attribute_def_parms
    node.setParmTemplateGroup(parm_group)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.547/houdini/python3.10libs\hou.py", line 16733, in setParmTemplateGroup
    return _hou.OpNode_setParmTemplateGroup(self, parm_template_group, rename_conflicting_parms)
hou.OperationFailed: The attempted operation failed.
Parameter name 'AYON_self_publish' is invalid or already exists

after create was done and when I clicked on publish this error popped up!


Fix issues with code and discussion of Sidre examples.

Summary

This PR contains code and documentation bugfixes, as well as documentation enhancements.

Here is a link to the relevant documentation page: https://axom.readthedocs.io/en/bugfix-rhornung67-sidre-example/axom/sidre/docs/sphinx/data_vs_metadata.html
There were a couple of issues with the code (thanks @nselliott for pointing them out!) that caused some confusion in recent discussion. The issues should be resolved with this PR. Although we became a bit sidetracked, the discussion was productive, I think, and helped focus in on a couple of things to consider changing in a future PR.
Specifically, here is what we have now.  If the data associated with the view is held in a buffer with no other views attached, the buffer will be deallocated and destroyed. However, View::deallocate() will deallocate the associated buffer if it is not shared and leave the buffer intact and attached to the view. Group::destroyView() and Group::destroyGroup() operations will destroy groups and views but leave associated buffers intact (in the datastore) and allocated.
Question: does it seem cleaner and more natural to deallocate and destroy a buffer whenever the last view attached to it is destroyed?  I believe if we did this, it may simplify and/or eliminate users' awareness of or use os Sidre buffers. It would probably have avoided issues users were running into when destroying groups and leaving allocated buffers behind (which appear to them as orphaned memory allocations). This change also may alleviate the need to make the changes in https://github.com/LLNL/axom/issues/674 and we could remove some methods such as Group::destroyViewAndData(), since Group::destroyView() would suffice.  @kennyweiss  @agcapps @cyrush @ltaylor16 @nselliott what are your thoughts on this? Others should feel free to weigh in as well. I only tagged the subset of folks who have the most experience with the inner workings of Sidre.

Regarding the question whether destroyView should also destroy the buffer once the buffer is unattached from all views, it seems like there are use cases where it makes sense and cases where it doesn't.
If the usage is like this:

create View v with createViewAndAllocate
do stuff with v, never directly interacting with the Buffer class
destroy v with destroyView

It is understandable that a user's expectation, if they don't closely read the documentation, is that all data associated with v should be gone.  That's not what happens, but it's a reasonable expectation.
Then there is this use case:

create and allocate Buffer buf
create View v
attach buf to v
do stuff with v
destroy v with destroyView

Here the buffer pre-exists the view, and was allocated independently of the view, so the reasonable expectation would be that the buffer remains after destroyView, and the distinction from destroyViewAndData is important.
I don't have the answer for what we should do, but this lays out some of the issues. There also can be examples that mix aspects of both of these usage patterns.

@nselliott I understand the second use case you cite. What do you think about, as @white238 suggested at the last meeting, adding a boolean argument to the appropriate methods, such as Group::destroyView(...., bool destroy_buffer = true), where the default would be to destroy the buffer. But, sophisticated users would have the option to pass 'false' as the last arg to keep the buffer intact?

Maybe it would be more clear to have the boolean as a required argument?

Since Sidre can be used without being aware of buffers, any buffer created by Sidre as part of the implementation, behind the scenes, should be deleted by Sidre. But if a user explicitly creates a buffer and gives it to the view, then the user should be responsible to deallocate the buffer as well.  This is similar to the Marbl case which creates a large buffer to share with views.  There should be a corresponding deallocate call by the user.
There is another case where Sidre creates the buffer but then the user gets a pointer to the buffer and gives it to another view (the odd-even example). The user assumes responsibility for memory when explicitly manipulating buffers.

@ltaylor16 I tend to agree with you. I think we can do some work on this to refine and simplify the API and potentially beef up the internal bookkeeping to make it easier to use for simpler use cases. It's not clear to me how we track who is responsible for buffer deallocation/destruction unless we add private methods for buffer creation and access that are only called by Sidre internals. They would call the analogous public methods but allow us to set a boolean, for example, on each buffer to track who "owns" it. Thoughts?

Folks, I would like to merge this PR since it contains fixes to the example code and the user documentation. Please look it over and comment/approve.
@kennyweiss @cyrush @agcapps if you have any thoughts on the conversation that has take place here, please share.
Thank you.

Folks, I would like to merge this PR since it contains fixes

I'm reviewing.

@ltaylor16 I tend to agree with you. I think we can do some work on this to refine and simplify the API and potentially beef up the internal bookkeeping to make it easier to use for simpler use cases. It's not clear to me how we track who is responsible for buffer deallocation/destruction unless we add private methods for buffer creation and access that are only called by Sidre internals. They would call the analogous public methods but allow us to set a boolean, for example, on each buffer to track who "owns" it. Thoughts?

Since createBuffer is only called in a couple of places internally, I'm not sure if a private method is better or just set the flag explicitly after calling createBuffer.  It's a balance of don't-repeat-yourself vs too many layers of abstraction. I'll create an issue and we can discuss it there.

Thanks @rhornung67 . Final suggestion: remove the comma after each i.e. and e.g.

It can go either way, and I have been using a comma after those as long as I can remember. Apparently, the consensus in American writing is to use a comma, wheres in British writing it is not used. For example, see https://www.dailywritingtips.com/comma-after-i-e-and-e-g/

one thing we need to fix for sure:
there is no path that when you destroy child groups that will destroy the data in views below if there is a 1-1 between those views and their buffers.
We need a destroyGroupsAndData that does this - regardless of what we call it.
Also, if you are looking to clean up a group and then reuse it, I think you would still have to call both:
destroyGroupsAndData() // we might not call it this!
destroyViewsAndData()

People will get this wrong.
We might want:
destoryChildrenAndData()

(or something, I am not thinking final names but just the cases we need to support)
For noah's second case. That is tricky, can a user detach a buffer (or is that only internal api speak)?

For noah's second case. That is tricky, can a user detach a buffer (or is that only internal api speak)?

Yes, attachBuffer and detachBuffer are in the public API of View. detachBuffer returns a pointer to the buffer so that it can be done without losing track fo the buffer.

@cyrush we have your concern captured here: https://github.com/LLNL/axom/issues/674 @ltaylor16 will be working on it this month.

@nselliott thanks!
teaching folks the pattern for the more complex case (I want to keep that buffer) might pay more dividends than trying to make the API bullet proof.

A recipe guide, like what Umpire has, would be a good way to document these things, I think.


Add kdtree test

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@d75f6bf). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master     #211   +/-   ##
=========================================
  Coverage          ?   63.87%
=========================================
  Files             ?       68
  Lines             ?     5944
  Branches          ?        0
=========================================
  Hits              ?     3797
  Misses            ?     1821
  Partials          ?      326

Impacted Files
Coverage Δ

kdtree/kdtree.go
100% <ø> (ø)

kdtree/heap.go
100% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d75f6bf...dccabca. Read the comment docs.

I sent pull request to wrong branch.....


Pinned announcements results in notification spam

Announcements can be pinned and such announcements always show up in notifications for that specific announcements. This is most likely because the REST responses has such discussions at the top, which the app probably doesn't take into account.

After switching to push notifications, this should no longer be relevant.

After switching to push notifications, this should no longer be relevant.


Profile - Uploading a non-image file will show infinite loading and blocks the user from uploading a new profile picture

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Navigate to settings
Click on Profile
Click on the edit icon in the avatar
On the file browser, select to display all files
Select a non-image file

Expected Result:
There should be a message about a invalid file uploaded.
Actual Result:
There's an infinite loading displayed under the avatar icon. No option to upload a new image is displayed.
Workaround:
None found. Refreshing doesn't fix the issue. Only logging out and back will remove the infinite loading.
Platform:

Where is this issue occurring?

Web

Version Number: 1.1.3-0
Reproducible in staging?: yes
Reproducible in production?: yes
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://user-images.githubusercontent.com/44479856/135350563-03de1a78-73bb-42c2-9dd0-8b899d29c7fd.mp4
Expensify/Expensify Issue URL:
Issue reported by: @hanaffi
Slack conversation: https://expensify.slack.com/archives/C01GTK53T8Q/p1632870641227600
View all open jobs on GitHub

Triggered auto assignment to @yuwenmemon (Engineering), see https://stackoverflow.com/c/expensify/questions/4319 for more details.

Proposed Solution: (If considered as external)
There are two problem here:
First is Non Image file (e.g. txt) Uploaded:
i.e. Whenever Non Image file uploaded at that time API.User_UploadAvatar() api throws error as
{jsonCode: 404, message: 'Invalid file. Please try a different image.', requestID: '69771aaa7f83697d-YVR'}
OR Sometimes return response within .then()  block as
 {jsonCode: 502, message: 'PDF Box can not upload user asset, requestID: '69771c200bbe31d0-YVR'}
I am not sure why sometime api throw error with {jsonCode: 404, ... } OR Sometime returns response with  {jsonCode: 502, ... }. May be server side issue or maintenance.  So I suggested solution that will handle all situation.
Second is Unsupported Image  Uploaded:
i.e. Whenever SVG image Uploaded at that time API.User_UploadAvatar() returns response within .then()  block as
 {jsonCode: 502, message: 'PDF Box can not upload user asset, requestID: '69771c200bbe31d0-YVR'}
So in both case it goes to loading state because we are just handling (response.jsonCode === 200) and there is no .catch() block within setAvatar() action in src/libs/actions/PersonalDetails.js file as shown under:
https://github.com/Expensify/App/blob/fa7cb605574f1ddd147615eb070d828985dfc20a/src/libs/actions/PersonalDetails.js#L273-L281
So we have to update setAvatar() actions as under within src/libs/actions/PersonalDetails.js file:
function setAvatar(file) {
    setPersonalDetails({avatarUploading: true});
    API.User_UploadAvatar({file})
        .then((response) => {
            // Once we get the s3url back, update the personal details for the user with the new avatar URL
            if (response.jsonCode === 200) {
                setPersonalDetails({avatar: response.s3url, avatarUploading: false}, true);
            } else {
                setPersonalDetails({avatarUploading: false});
                Growl.show(translateLocal('profilePage.avatarUploadFailureMessage'), CONST.GROWL.ERROR, 3000);
            }
        })
        .catch((error) => {
            setPersonalDetails({avatarUploading: false});
            if (error.jsonCode === 404) {
                Growl.show(translateLocal('profilePage.invalidFileMessage'), CONST.GROWL.ERROR, 3000);
            } else {
                Growl.show(translateLocal('profilePage.avatarUploadFailureMessage'), CONST.GROWL.ERROR, 3000);
            }
        });
}

Also within language file we have to add translation as under:
src/languages/en.js
profilePage: {
    ...,
    invalidFileMessage: 'Invalid file. Please try a different image.',
    avatarUploadFailureMessage: 'An error occurred uploading the avatar, please try again.',
},

src/languages/es.js
profilePage: {
    ...,
    invalidFileMessage: 'Archivo inválido. Pruebe con una imagen diferente.',
    avatarUploadFailureMessage: 'No se pudo subir el avatar. Por favor, inténtalo de nuevo.',
},

Below is screen record, How it works after update:
https://user-images.githubusercontent.com/7823358/135664939-687353e8-1a28-4632-bc5e-c13b295335c0.mov

This can be worked on by a contributor.

Triggered auto assignment to @dylanexpensify (External), see https://stackoverflow.com/c/expensify/questions/8582 for more details.

Working on this now!

Posted!
Internal: https://www.upwork.com/ab/applicants/1445266428359061504/job-details
External: https://www.upwork.com/jobs/~01bf3c369a15b3262e

Triggered auto assignment to @thienlnam (Exported), see https://stackoverflow.com/c/expensify/questions/7972 for more details.

@dylanexpensify, If my suggested solution https://github.com/Expensify/App/issues/5593#issuecomment-932443740 work then kindly accept my proposal on upwork. So I can proceed to submit pr.

please accept my proposal so that I can work according to your requirements

@PrashantMangukiya Great proposal and good catch on the handling with the .catch and the .then.  This solution works but as you mentioned we have some server side handling that will sometimes not throw the error.  That's something I will look into internally but for your proposal, lets only keep the handling in .catch and remove the if/else in the .then

@gaurav27ats Thanks for your interest! Please read over https://github.com/Expensify/App/blob/main/CONTRIBUTING.md#working-on-expensify-jobs for future reference as you'll need to propose a solution in the Github issue

Please refer to this post for updated information on the n6 hold, we've raised the bonus to $250 for all issues where a PR is created before the N6 hold is lifted.

@PrashantMangukiya Thanks, that looks good 🟢 on the job - @dylanexpensify Could you please hire @PrashantMangukiya for the job and assign him on this issue?

I think you should handle the wrong codes in then block and even if you want to catch them, then throw them so that they propagate to catch.
API will always respond with 200 HTTP code until something went wrong with the server or Network issue and errors are handled internally with jsonCode field. If you know that API throws 400 and 502. then you can handle both. Right?

@PrashantMangukiya The mentioned PR is on production and I also updated one of the codes to a 405 instead of a 404 for an invalid image.  Please give it another test and let's continue implementation specific comments in the PR

@thienlnam thanks for the message. Today I am busy, so tomorrow I will test and continue for implementation in the PR. I will message if need any other help. Thanks again.

@thienlnam  I tested it with latest version. Still it is returning 502 code within then() block if invalid image file  uploaded. Here is latest screenshot from web debug.

It looks like there is not any api changes at server side etc.

Gotcha, in that case go ahead and use parasharrajat's suggestion and handle errors by throwing inside the .then() if it's not a 200

Thanks @thienlnam  I will implement it as per parasharrajat's  suggestion i.e. handle errors by throwing inside the .then() if it's not a 200.   So once I will test it properly thereafter submit pr as soon as possible.

not overdue

The solution for this issue has been :rocket: deployed to production :rocket: in version 1.1.14-4 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

https://github.com/Expensify/App/pull/6189

If no regressions arise, payment will be issued on 2021-11-22. :confetti_ball:

@dylanexpensify Ping for Upwork payment.

Issue reported by: @Hanaffi
Ahmed needs to be paid for reporting, I've hired them in Upwork, waiting for them to accept then I'll pay.  Managing via email.

Ah shoot, thanks @mallenexpensify! Completely missed this, my bad!

Paid @hanaffi in Upwork for reporting


Adding Libre 3 sensor

Hi,
I am all new to GlucoseDirect. I am currently wearing a Libre 3 sensor, with 11 days remaining. Is it possible to add this sensor to the GlucoseDirect app? Or do I have to replace the sensor with a new one, and start it / scan it for the first time via the GlucoseDirect app, and not the Libre app?
I have scanned the sensor, and I have entered my Libre Link Up details. But it just says "No Data" and state is "Scanning". It also says that the sensor is in starting phase, with 1h warm up. But this is not correct because I am already using the sensor, with 11 days remaining?

@christoffer22 This can occasionally happen on first scan. If you close the app and open it again does it still say No Data?
If it does, try force shutting the app (slide up) and reopen - try rescanning then.

The culprits are the lines
https://github.com/creepymonster/GlucoseDirect/blob/e3636d0e47cd490bb02db9e10b984bc0b53ca70d/App/Modules/SensorConnector/LibreConnection/LibreLinkUpConnection.swift#L264
and
https://github.com/creepymonster/GlucoseDirect/blob/e3636d0e47cd490bb02db9e10b984bc0b53ca70d/App/Modules/SensorConnector/LibreConnection/LibreConnection.swift#L49
because they don't ascertain that sensorAge and  result.sensor.age > 0.
I reported this bug multiple times but unfortunately @creepymonster didn't reply to me anymore even privately... ;-/

I’m having the same issue after things working for almost two years. It seems it might have started with iOS 18.
Force quitting the app doesn’t do anything.
It doesn’t just say No Data, it also says Missing User or Token.

Is this something anyone else is experiencing? I haven't had this app work for months.


regex for string 'is' fails

Let's say I have two entries. Louis Armstrong , Miles Davis and Ella Fitzgerald.
The real world example is much more complex (multiple positive lookeheads), but the issue is the same. Every other string that i have tested worked well.
Here is my query variable:
let query = {
  name: {$regex: new RegExp('is', 'i')},
};

Please follow the bug reporting guidelines when reporting a bug.

Ok, i'll do so. Sorry for bothering you at this point :)