Feature Request - Vizceral addon

Create an addon to view Istio metrics in Vizceral. The metrics can be gathered from Prometheus. I'd like to propose to name it Vistio :)
Can reference this project as a starting point https://github.com/nghialv/promviz

subscribe and + @esnible

FYI: Kiali is a front end to istio currently in development (no official release just yet) - it does take the Prometheus metrics and visualizes them (along with visualizing the mesh and other things such as visualizing circuit breakers, route rules and integrating Jaeger tracing). See https://kiali.github.io/ for details. We have a JIRA issue created to propose it as an istio add-on which would address this istio issue... see https://issues.jboss.org/browse/KIALI-386

@nmnellis Having options is always good  :)
If you get a chance to give Kiali a try, we would be very interested in your feedback, and if there are any features you would like to see let us know.

When I tested Promviz there were three difficult things I had to do.  Nick has already talked about needing to add a complex set of values to promviz/helm/promviz/values.yaml
Promviz expects "standard" storage which is not provided by IBM Cloud.  My work-around was to make "standard" a synonym for bronze but there shouldn’t be a requirement on a storage type.  Also it might be good to have a way to run without storage for folks who are experimenting with the free cluster.
kubectl get storageclasses ibmc-file-bronze -o yaml | \
    sed s/ibmc-file-bronze/standard/ | \
    kubectl create -f -

Promviz-Front (specifically promviz-front/src/app.jsx) calls WebFont.load() and does nothing until the font because active.  On my browsers the font never became active; I had to modify the code so that it would call fontsActive() even if WebFont.load() reported inactive status.

i want to look and see if we can just provide an inmem solution that translates prometheus metrics to vizceral. that would make it a lot easier to install and use for first time users.

@nmnellis  would you happen to have sample configuration for promviz I could  use as starting point? I have no clue whatsoever where to start. (great talk at Kubecon btw)

for my demo i used statsd-to-prometheus to get metrics from envoy directly
apiVersion: v1
kind: ConfigMap
metadata:
  name: promviz-config
  namespace: monitoring
data:
  promviz.yaml: |-
    graphName: Istio
    globalLevel:
      maxVolume: 50
      clusterConnections:
        - query: sum(rate(envoy_upstream_rq_count[1m])) by (job, status)
          prometheusURL: http://prometheus:9090
          source:
            replacement: INTERNET
          target:
            label: job
          status:
            label: status
            warningRegex: ^4..$
            dangerRegex: ^5..$
    clusterLevel:
      - cluster: envoy
        maxVolume: 10
        serviceConnections:
          - query: sum(rate(envoy_upstream_rq_count[1m])) by (cluster,status,job)
            prometheusURL: http://prometheus:9090
            source:
              label: job
            target:
              label: cluster
            status:
              label: status
              warningRegex: ^4..$
              dangerRegex: ^5..$
            notices:
              - name: HighErrorRate
                title: "[{{ .value }}] HighErrorRate"
                statusType: danger
                severityThreshold:
                  warning: .05
                  error: .10

Initial release is here, if everything looks good i will submit a PR to have it be an optional addon in Istio https://github.com/nmnellis/vistio

@nmnellis What is the difference of this with Kiali?

Just because kiali was first to be integrated as an addon doesn't mean it should be the only choice for visualization.
That said, before this PR can be approved, need validation that the proposed MIT license is compatible with the ASL2 as per whatever legal requirements Istio has on dependencies/transitive dependencies.
@smawson or @jasminejaksic any chance you can answer this license question?
Cheers
-steve

I see, thanks @sdake
The reason that I have this question is because there are many open source projects doing visualisation, but seems no one is creating a dashboard which can do write operations for istio, such as creating gateway, virtualservices etc.

Kiali today has the ability to view Istio configs (there is some support to also flag the config when something looks invalid). See snapshot below.
We are thinking about adding the ability to "write" these configs from this screen (add new ones, change existing ones, etc).
We are always willing to hear what features the community wants to see. If this is something people want, please let the Kiali team know. Community input is important to us - we want to know what people want. :)

Thanks @jmazzitelli , opened an issue at Kiali https://github.com/kiali/kiali/issues/261 :-)


Allow image labels to contain right padding

Useful when you are stacking several shields vertically, it can be more visually appealing to have the left labels align
Before:

After:

I am sorry, especially considering that your patch is well above average quality-wise, but I feel like this functionality is out of scope of the shields project and introduces too much complexity (for instance, it would take a few short weeks before someone added a patch to have the right-hand side match, as well).


Unset data source while closing pool resource

This prevents exception being thrown when pooled resource is being closed multiple times.
Some of probable causes of those exceptions:

java.lang.IllegalStateException: Object has already been returned to this pool or is invalid
java.lang.IllegalStateException: Invalidated object not currently part of this pool

@marcosnils @HeartSaVioR

@smadasu should this be released in 2.9.1?. What if someone was catching exceptions and doing something when this happened?. Seems like a backwards incompatible change. How about releasing it in 3.0?

@marcosnils I saw a couple of issues where it turned out that the reason of the exception was that the user was closing Jedis twice, unknowingly! I considered, throwing exception while closing a already closed Jedis, as a bug. So as a bugfix, I set this PR for 2.9.1.

Ok, I guess it makes sense.

Merged into master, 2.10, 2.9.


is it possible to know the name of the file on the disk while downloading ?

I am trying to implement caching and streaming of video using this library.
I do this by downloading the video file and give it a filename. But when downloading it has a different file name so i am unable to access the file during download. But when the download is successful I get it automatically renamed.

The temp file name is actualname+".temp". Example: image.png.temp


[newbie] UDS client or API specification?

I've browsed a little bit through the repo source code and I've seen the internal udsserver and uds packages. The uds package seems to be the part I've marked in the above illustration, taken from the documentation in this repository.
What I don't understand yet and haven't found any explicit documentation or examples (if I'm not mistaken): how do I now use this in my container workload, how do I integrate the UDS client into my own workload code?

Hi @thediveo
The little purple square on the diagram is not a UDS client, it's just representing the actual UDS socket file.
The uds package is just for basic read/write/dial/listen functions. The udsserver imports uds and implements the little server we have for handing over the XSK Map file descriptor to the containerised workload, allowing the workload to go on and create the AF_XDP socket. This handover involves a little "handshake" across the UDS.
I think what you're asking for is a udsclient that you can import into your code and would it perform the "handshake" for you. I'm afraid this does not exist, though it's a really nice idea, I like it.
So, at least for the moment, without such client, your workload has to perform the "handshake" itself. This entire list of calls in the handshake can be found here.
Not all calls are necessary. The main ones are:

/connect, hostname where hostname is the name of your pod - $HOSTNAME within your pod
you should hopefully receive a  /host_ok response
then you can request the xsk fd with /xsk_map_fd, dev where dev is the network device you want the FD for. This can be called multiple times for pods with multiple netdevs attached.
all going well, the device plugin will return a /fd_ack and the XSK FD will be contained in the control buffer part of this message.
/fin will close the uds connection

You can see some of this implemented in our little end-to-end test here and the CNDP project implement it (in C code) in the uds files here.
KR,
Gary

@patrickog11 @davecremins @maryamtahhan thoughts on the client idea? A package that folks could import into their code and it would do the handshake for them.

Thanks for the responses!

Some devs and projects might need a C binding. Today, there's libxdp.
Is the fd passed to the client really an XSK fd or rather the fd for a map of XSKs? Because otherwise the XSK would need to be created using some undocumented default values for the various rings.

actually, I was looking to the C client to use as I would suspect many people to head for C when dealing with XSKs on the data plane.


Merge fixes from docker-fixes to GUI dev

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.4 out of 5 committers have signed the CLA.:white_check_mark: ltcptgeneral:white_check_mark: zpan1:white_check_mark: jzpan1:white_check_mark: devksingh4:x: == seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

Necessary to freeze superscript branch


All graph types do not show the lowest value in the dataset

When plotting data with any of the three chart types, the lowest value in the dataset is never drawn. This is perhaps most obviously demonstrated with a bar plot.
For example, If I create an array called "test" and add values from 1-10, like this:

You get a chart that looks like this:

You can see that the y-axis starts at 2 and not 1, and the first value is completely missing.
The result should look like this:

After doing some debugging, I believe the reason for this is because of two issues in Show-Graph.ps1.
On line 148 the below code never processes the last value in the set:
For($i=$NumOfRows;$i -gt 0;$i--)
Simply changing the -gt comparison operator to -ge ensures the for-loop processes all the data points:
For($i=$NumOfRows;$i -ge 0;$i--)
That alone does not resolve the issue though. The second section of code that needed changing is at line 221:
elseif($RangePercent -le 40 -and $RangePercent -ge 1)
A similar issue to the above, the code is not processed when $i is equal to 0:
$RangePercent = $i/$NumOfRows * 100
elseif($RangePercent -le 40 -and $RangePercent -ge 1)
Simply changing line 221 to elseif($RangePercent -le 40 -and $RangePercent -ge 0) allows the chart to be plotted correctly.
I hope this is useful.

@bennyguk , this is very useful. I have encountered this in the past. I thought it related to not setting the reference valus for the x-axis.
I suggest put this forward as a push request for the project.

Hi @MMitsialis No problem. I'll do that ASAP.
Thanks.


only search checkout-v3 docs

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Q: Merging a fork which makes this package more generic?

Hi,
Firstly, thanks for open-sourcing this package!
I ended up needing TUPAS functionality in a single-page app built on top of hapi, and instead of rolling my own implementation, decided to modify and use this package. This required a small amount of changes (+ did some additional fixes), a high-level summary:

Remove express dependency from the core
Allow creating an instance without an appHandler (to be able to use the internals also on client-side without providing dummy appHandler implementation)
Cleaned up the form (no JS needed for submission, no CSS in style attribute, more semantic HTML)
Validating (some of the) function arguments (with tests)
Add a hapi plugin sample

Breaking changes:

express not exposing the static img directory by default anymore, so this needs to be done in the host application
Requires hostUrl to use https (this is probably the case anyway, though)

Some further considerations:

This package doesn't state the targeted supported node version nor the targeted supported browsers. Thus worked based on the assumption that these targets are node >0.10 and browsers starting from ancient IE6, although there's not much to break regarding these.
The hapi plugin uses some ES6 features and thus works only on node >4.x. This is, however, the setup for the latest hapi itself, too, so not sure if this is a problem. The main annoyance due to this is seeing warnings during npm install (when developing this package, not when using this, hapi dep is a dev-dep).

Full diff available here https://github.com/reaktor/node-tupas/compare/master...harriha:master
I understand that this package, as it is now, is a quite simple plug'n'play solution for a traditional web site built on top of express. Nevertheless, would you be interested in merging these changes to enable a wider audience for this package? I'd be happy to open one or several PRs to get those changes in, whatever you'd prefer.

Just wanted to say that I'm ashamed and sorry that I didn't reply to this in any manner. I agree that it would be great to get the express dependency out. I know merged your changes to the current master in https://github.com/reaktor/node-tupas/tree/extract-express-dependency branch. I haven't been personally involved in projects using this module in production in many years, but I will ask from guys who are using it. At least I have to write instructions for migration before this can be merged.

Hi,
No worries whatsoever, thanks for taking the time for coming back to this. As per the original intention of my (perhaps a bit unusual) issue, this was mostly a heads-up of a refactored and generalized version, I didn't expect the branch to fly in to the master immediately anyway as the library is a fit for purpose as-is. In any case, feel free to use what you see useful and close the issue.
Now, when re-reading the description and the diff above, a couple of things come to my mind:

hapi has gone through significant changes in its latest version, so the hapi example might not be compatible anymore (though, it's for educational purposes anyway, so an old version might be just fine)
the requirement for having hostUrl  to be with https protocol also during dev can be a PITA in many cases, so not so sure anymore about that change in that exact form at least.


[#5051] GravitinoClient needs metalake to obtain version

What changes were proposed in this pull request?
[Improvement] GravitinoClient needs metalake to obtain version
Why are the changes needed?
Fix: #5051
Does this PR introduce any user-facing change?
No

Thanks for your contribution. There are a couple of unnecessary changes to the documentation here, can you please fix those? It also looks like you using an old branch, it would be best to use a new branch off the main one.

Thanks for your contribution. There are a couple of unnecessary changes to the documentation here, can you please fix those? It also looks like you using an old branch, it would be best to use a new branch off the main one.感谢您的贡献。这里的文档有一些不必要的更改，您能修复这些吗？它看起来也像您使用旧分支，最好使用主分支之外的新分支。

Hello, I didn't quite understand what it meant. Can you explain it to me

Your PR include changes to apache-hive-catalog.md and a couple of other .md documentation files, these changes are not related to the issue and should not be in the PR. Your branch is also 3 months out of date, it would be best to base your PR on a newer branch.

Your PR include changes to apache-hive-catalog.md and a couple of other .md documentation files, these changes are not related to the issue and should not be in the PR. Your branch is also 3 months out of date, it would be best to base your PR on a newer branch.您的 PR 包括对 apache-hive-catalog.md 和其他几个 .md 文档文件的更改，这些更改与问题无关，不应出现在 PR 中。你的分支也已经过期 3 个月了，最好将你的 PR 建立在较新的分支上。

Hello, I am unable to use a specific commit file specified in the submitted PR. Can you help me


gcp_model: typo and potential issue

anyAntiHypertensive consistently has a typo on this file
Also, the calculation on line 86 assumes a constant antihypertensivecount but I assume in general that is not a constant.

JB

i’m not sure that i get your concern — how did you get the conclusion that it assumes a constant count?

the model that its based on doesn’t attend to the number of anti-hypertensive medication, instead it counts as a risk factor whether a patient is on ANY antihypertensive medication.
so, the “anyAntiHypertensive” input is tabulated on either line 127 or 145…and it determines if the sum of the antiHypertensiveCount + bpMedsAdded > 0
CD
Now I understand correctly, this is a 0/1 value not a count as I thought it was.....

A question.....the gcp_model includes this definition:
anyAntiHpertensive=((person.antiHypertensiveCount + person.totalBPMedsAdded)> 0) (line 127 and 145)
and this seems to be a different definition of the risk factor that I think is used in the rest of the models (which is just antiHypertensiveCount).
Should the gcp_model use antihypertensives+bpmeds as the risk factor or just antihypertensives?

outstanding question.
the honest answer is that i don’t know.
i think teh best answer is that it probably SHOUDL include BP meds added here, because of the direction of the effect. for CV risk, “any anti hypertensive” increases risk and there is no way that an added BP med shoudl do that.
here, adding a BP med improves GCP and that is more likely to be the correct direction of the causal effect (and we don’t have sufficient data to do a good recalibration), so I’d probably leave it as is…
Jim

On Mar 14, 2023, at 11:17 AM, Christos Deligkaris @.***> wrote:
A question.....the gcp_model includes this definition:
anyAntiHpertensive=((person.antiHypertensiveCount + person.totalBPMedsAdded)> 0) (line 127 and 145)
and this seems to be a different definition of the risk factor that I think is used in the rest of the models (which is just antiHypertensiveCount).
Should the gcp_model use antihypertensives+bpmeds as the risk factor or just antihypertensives?
—
Reply to this email directly, view it on GitHub https://github.com/jburke5/microsim/issues/41#issuecomment-1468300932, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDZITWKVXGYH55R5U5NVPDW4CDYBANCNFSM6AAAAAAVN3B5QM.
You are receiving this because you are subscribed to this thread.

Sounds good. I will take care of the typo in this file because this variable is part of function arguments and I will close this issue when the typo is taken care of.


Customisation

Great to see my original effort has been forked and improved so much :-) I'm starting a new API project and so am looking at this again.
I feel like some may be put off from using the generator due to it not being comprehensive and there is always something someone might want to add or change in their OpenAPI spec.
My suggestion is to allow a developer to extend the info, servers, paths, components, security, tags and external docs that ultimately get sent to the GoldSpecDigital\ObjectOrientedOAS\OpenApi instance. This would give the developer complete freedom to tweak whatever they like.
Would you consider a PR for this?

Hi Glenn, nice to see you here! We like to make this generator feature complete, but unfortunately haven't had the time to make any real progress yet. What you suggest sounds good and we are certainly open for a PR!


Update update

Fixes #1452
Feature Summary and Justification
There's a newer endpoint (that uses PATCH) to update subreddit settings that takes only the settings that are being updated. Also, it supports the welcome message stuff from #1452. This feeds two birds with one scone, so to speak, because we get welcome message support and we can cut out the messiness of fetching (and remapping!) all current settings just to update a few.

Regarding the welcome message—despite what the redesign interface for the welcome message, https://new.reddit.com/r/TEST_SUBREDDIT/about/edit?page=community, would have you believe, the actual size limit for the welcome message is 10000 characters, Reddit's usual limit for markdown in messages, not 5000. If you set it to something between 5000 and 10000, the redesign page will says say that there are negative characters remaining, but new Reddit will still let you trim the message to something greater than 5000 characters. If you try anything over 10000, the welcome message will be set to the empty string. That could be useful info to add to the documentation.
There is also an api endpoint for sending out test messages, api/v1/subreddit/test_welcome_message. It too has a character limit of 10000 chars, Example usage:
requests.post("https://oauth.reddit.com/api/v1/subreddit/test_welcome_message?raw_json=1", data={"test_message_text": "Test message goes here","sr":"TEST_SUBREDDIT"}, headers={'User-Agent': 'xxxxxxxxxxxxxx', 'Authorization': 'bearer xxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxx'})
will send a message from /u/welcomebot to your inbox. Unlike api/v1/subreddit/update_settings, for which the value of "sr" is the sub's full name (t5_xxxxx), in api/v1/subreddit/test_welcome_message, the value of "sr" is the sub's "regular name" (I forget the official term), e.g. redditdev.  For example, running the following
requests.post("https://oauth.reddit.com/api/v1/subreddit/test_welcome_message?raw_json=1", data={"test_message_text": "[Shady link here](//example.com)","sr":"reddit.com"}, headers={'User-Agent': 'xxxxxxxxxxxxxx', 'Authorization': 'bearer xxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxx'})
will send a message that looks like  to your inbox. @kemitche, is this a security issue? The endpoint isn't documented, so I don't know what the scope is. If someone gives a shady third party app oauth access with a scope that includes that endpoint, that third party could use /u/welcomebot to send the unsuspecting user a malicious pm.

Thanks for calling out those issues.
I don't see the test PM as a security issue, since the user just ends up with a PM to themselves, but it's certainly a point of confusion. I've flagged the concerns to the right teams internally.
Note that the test_welcome_message endpoint isn't listed on https://www.reddit.com/dev/api, and so may change or be removed at any time as it's not officially listed as a public API endpoint.

Thanks!


Create Node class to lay groundwork for better node handling.

Nodes should hold a little more information outside of uri - ie, status, prefix, version, etc. (Dynamic Global Properties + some extra stuff). This will allow us to start building a more intelligent node handling framework within steem-python.
This is part of the attempt to remove the SQL Config from steem-python. Reference issue #216 .

When you say "more intelligent node handling" does that also means handling scenario which are down / unrechable ?


Fix getFormError Flow typedef.

The getFormError selector takes in a form name but it was not accounted for in the Flow typedefs.
Also, the GetFormErrorInterface is not a polymorphic type.
See docs for this selector here:
https://redux-form.com/7.4.2/docs/api/selectors.md/

@erikras I re-created this PR from #4130. Would really appreciate a review since we are now using a fork of redux-form just to use this PR.

@erikras @gustavohenke any chance we can push this to master?

@davidgomes we have this exact issue. Our temporary solution is explicit typecasting through any.
formError: ((getFormError: any): (form: string) => (state: {}) => any)(form)(state),


Cannot deploy resources added by other developers, they do not appear in status.

Describe the bug
We use a feature branching workflow and each of our developers works in their own sandbox.  When a developer adds a resource (we tested this with cognito triggers, but I believe it to be generally applicable to all added resources) and I checkout their branch and try to push their configuration to my amplify environment, the resource isn't on my resource list to be deployed.
I did a little sleuthing and it noticed that while the resource is listed in the backend-config.json, it is not present in the amplify-meta.json. If I add an entry for the resource to amplify meta, then it appears in my status.
I would expect that because the resource is in backend-config.json that if it were missing from the amplify-meta.json it would be added and deployed on amplify push.
running amplify init after checking out the feature branch seems to resolve this.
Amplify CLI Version
tested with both

4.36.2
4.41.2

To Reproduce
These are specific to our test scenario

Developer A Create a repo with amplify init. sandbox A
Developer A amplify add auth and setup a cognito, push it to sandbox A
Developer A Commit Code.
Developer B Checkout code, amplify init sandbox B, amplify push
Developer B Create Feature Branch
Developer B amplify auth update, add a PostConfirmationTrigger,  amplify push to sandbox B
Developer B commit and push amplify config updates to branch.
Developer A checkout feature branch
Developer A amplify push to Sandbox A, notice the curious lack of the PostConfirmationTrigger

Expected behavior
That Developer A could successfully deploy the canonical version of the code to their current environment without needing to run amplify init.

@dopry look at the Sandbox environments page where we document exactly the same workflow. As you also noticed, Amplify carries env specific (sandbox specific) information that is not under source control and when working in a team setup, Amplify environment operations should be used in sync with git branch operations. amplify-meta.json contains information about the current env according to your own cloud deployment. The step you are missing I think is the amplify env checkout in step 4 instead of init. env checkout rebuilds the meta file based on configuration parameters in resource directories and backend.json (which has env agnostic resource information).
If you think we could make if more clear in the documentation somehow, give some suggestions on how could we improve it based on the problem you ran into.

@dopry look at the Sandbox environments page where we document exactly the same workflow. As you also noticed, Amplify carries env specific (sandbox specific) information that is not under source control and when working in a team setup, Amplify environment operations should be used in sync with git branch operations. amplify-meta.json contains information about the current env according to your own cloud deployment. The step you are missing I think is the amplify env checkout in step 4 instead of init. env checkout rebuilds the meta file based on configuration parameters in resource directories and backend.json (which has env agnostic resource information).
If you think we could make if more clear in the documentation somehow, give some suggestions on how could we improve it based on the problem you ran into.

I do not want to checkout a different sandbox. I do not have access to any of the other developers sandboxes. I want to push the configuration in the code to my current sandbox.

I do not want to checkout a different sandbox. I do not have access to any of the other developers sandboxes. I want to push the configuration in the code to my current sandbox.

We do this every day with terraform... git checkout; terraform apply as our configuration evolves. I don't understand why this workflow has to be so unwieldy with Amplify. Amplify is really promising, but the developer experience can use a lot of love.

We do this every day with terraform... git checkout; terraform apply as our configuration evolves. I don't understand why this workflow has to be so unwieldy with Amplify. Amplify is really promising, but the developer experience can use a lot of love.

I followed your instructions to use amplfy checkout env to re-checkout my current environment it just tells me the environment can't be recognized. I have to amplify init again to get it into a working state.

I followed your instructions to use amplfy checkout env to re-checkout my current environment it just tells me the environment can't be recognized. I have to amplify init again to get it into a working state.

I've also done some manual edits to dependencies in my backend-config.json file.... When I do, the dependencies in amplify-meta are taking precedence. I feel like amplify-meta should only be the cache of remote state, while backend-config should be authoritative for the desired state. It seems like that is the root of my issue. I expect the code I've checked from out to be authoritative and deployable to my current environment.  I think this is a fairly consistent expectation for any devops practitioners who work with IAC.

I've also done some manual edits to dependencies in my backend-config.json file.... When I do, the dependencies in amplify-meta are taking precedence. I feel like amplify-meta should only be the cache of remote state, while backend-config should be authoritative for the desired state. It seems like that is the root of my issue. I expect the code I've checked from out to be authoritative and deployable to my current environment.  I think this is a fairly consistent expectation for any devops practitioners who work with IAC.

@dopry Yes, right now the behavior is to run amplify env checkout <env-name> to update all the metadata files when adding/updating custom stacks/resource or updating backend-config.json. I'll mark this as an enhancement.

Go for it. We abandoned amplify and never plan to revisit it. It was a terrible DX.


docs: Fix typo in pyathena connection string

SUMMARY
The pyathena connection string is missing the schema name and the s3 staging dir parameter.

Thanks @ctSkennerton for the fix!


Airy function in entire complex plane

This PR gives dedicated routines for the airy functions instead of relying on the relations to Bessel functions. The first thing this allows is significantly more accuracy than we had before in whole domain (for airybi these are roughly 4 more digits accurate). I've updated the tests to reflect that.
The second thing is they are much faster now.
julia> @benchmark SpecialFunctions.airybi(z) setup=(z = 4.0)
BenchmarkTools.Trial: 10000 samples with 10 evaluations.
 Range (min … max):  1.257 μs …  2.107 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.267 μs              ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.271 μs ± 27.456 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

       ▂▄▄█▇▇▅▃▁
  ▁▂▄▅██████████▆▅▄▃▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▂▂▂▂▂▂▂▂▂▂▂▁▁▁▁▁ ▂
  1.26 μs        Histogram: frequency by time        1.31 μs <

 Memory estimate: 16 bytes, allocs estimate: 1.

julia> @benchmark airybi(z) setup=(z = 4.0)
BenchmarkTools.Trial: 10000 samples with 988 evaluations.
 Range (min … max):  46.224 ns … 79.481 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     47.749 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   48.534 ns ±  3.767 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▃  █▄                                                     ▁ ▁
  █▁▄██▄▄▅▇▇▇▆▅▆▅▆▆▆▇██▆▆▇▅▅▅▄▆▅▄▄▃▄▄▄▃▄▃▃▄▃▄▄▁▃▄▃▁▁▃▄▃▃▁▄▆▇█ █
  46.2 ns      Histogram: log(frequency) by time      69.3 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

So having dedicated routines allows for much better accuracy and speed. Of course, I have also added support for the entire complex plane. The motivation for this is some of the uniform asymptotic expansions for bessel functions require evalulation of the airy functions (https://dlmf.nist.gov/10.20) so this may come in handy as we look at those for complex plane.
Of course there are still some domains I'm not too happy with in the complex plane. I will add notes in the file on this. I chatted with @cgeoga and a potential use here is to use the hypergeometric functions or the exponentially improved functions to try and bridge domains. We can piece that together better in the future.
I still have a couple of things to finish up on this. Mainly tidying up and adding docs to this but also getting rid of some of those constant gamma calls. There is a lot of code overlap that could be simplified especially in the asymptotic expansions. However, what we want to overlap with will depend if we want to compute both airyai and airyaiprime or airyai and airybi. Just depends. It might be something we revisit in the future. Looking at the asymptotic expansions we will want airyai and airyprime so it might best to combine those....
Need to also go through and think more carefully about type promotions. I don't think we need to actually promote to float64 in a lot of the algorithms this is just left over from things I was trying.

Ok I've updated the docs and added the appropriate limits for infinite arguments therefore no error is return they will simply return the branches defined by the exponential or 1 / z.
There's one thing I'm still thinking about is how do we want to appropriately promote types in some of the series. For example the besseli series can always be computed without loss of precision but only for real arguments. For complex argument we will need to promote ComplexF32 to ComplexF64 and same happens here for the airy functions (airybi) which can be computed stably for real arguments but not so for imaginary arguments. Need to come up with a good structure to promote Float32 for these use cases.

Ok I think this is ready to go. I've added tests for Float32 arguments. This function oscillates around the zero axis for negative arguments so it is just easy to promote these to Float64. I'm open to other ideas the best way to do this but went with a structure like.
function airyai_power_series(x::ComplexOrReal{T}; tol=eps(T)) where T
    S = eltype(x)
    ...
   ...
    for i in 0:MaxIter
        ...
        abs(t) < tol * abs(ai1) && break
       ...
    end
    return ...
end
airyai_power_series(x::Float32) = Float32(airyai_power_series(Float64(x), tol=eps(Float32)))
airyai_power_series(x::ComplexF32) = ComplexF32(airyai_power_series(ComplexF64(x), tol=eps(Float32)))

Here is a plot of the relative errors in the entire complex plane compared to the special functions implementation.

@oscardssmith Ok with merging?


Different type narrowing behaviour between !Array.isArray and === false

Bug Report
I have a variable of type unknown that I want to narrow to either a string or any[].
If I use Array.isArray(variable) === false the type narrowing stays as unknown which I do not expect.
If I use !Array.isArray(variable) the type narrowing becomes string | any[] which I expected.
🔎 Search Terms
Array.isArray type narrowing unknown

🕗 Version & Regression Information
v4.8.2 - v3.3.3

This is the behavior in every version I tried.

⏯ Playground Link

Playground link with relevant code
💻 Code

const someConfigValue = (): unknown => {
    return [1, 2, 3];
}

// type is unknown, this is expected
const foo = someConfigValue();

// change the array check to !Array.isArray(foo) and foo narrows to string | any[] rather than unknown
if (typeof foo !== 'string' && Array.isArray(foo) === false) {
    throw new Error('Not a string or an array');
}

// foo is still unknown, this is not expected
// this is just here so you can easily see what they type is via the errors sidebar
console.log(foo.keys());

🙁 Actual behavior
Type narrowing does not work correctly when using Array.isArray(variable) === false vs !Array.isArray(variable)

🙂 Expected behavior
Type narrowing should work the same between Array.isArray(variable) === false and !Array.isArray(variable).
Array#isArray returns a boolean, it should not matter if I do a loose or strict check.

IIUC, typeof foo !== 'string' is not required to reproduce this behavior. Below is a more minimal reproduction (playground).
const foo = (arg: unknown) => {
  if (Array.isArray(arg) === true) {
    arg; // unknown
    //^?
  }
  if (Array.isArray(arg)) {
    arg; // any[]
    //^?
  }
};

Duplicate of #31105.

Also a duplicate of #9508 (much older), which was marked Won't fix.

I believed that there must be a duplicate because this behavior seemed easy to encounter, but I didn't have the time to investigate at the moment. Thanks for pointing out.

Interesting, thank you for the additional context. I wonder if the stance would be different now than it was in 2016, if this would be easier to implement or not.
I found the justification for the Won't Fix to be weak, in my opinion. I don't see how strictness is a code smell and should be a first class citizen when it comes to support.
However, I am not a contributor or maintainer of TypeScript so I am not privy to the complexity or design decisions around this. So happy with just knowing the context.

To me personally it's just more noise and adds no benefit. Makes the code harder to read as well.
You might be interested in the ESLint rule "strict-boolean-expressions": https://typescript-eslint.io/rules/strict-boolean-expressions/
It ensures that logical expressions are booleans. So if (myBool) is valid, but if (myString) or if (myNumber) is not.

Thanks, that might be a good substitute.


fix cloud gpu tutorial links

will this work? the tutorials are in /docs folder.

right, and so is index.md so the relative url should work i think. it works viewing it at https://github.com/mcohen01/course-v3/blob/tutorial_link_fixes/docs/index.md


Request Help : What is the best practice to store ES data for just 7 or 14 days?

Why do you submit this issue?
[x] Question or discussion

Question

What is the best practice to store ES data for just 7 or 14 days?
Is startup env setting SW_CORE_DAY_METRIC_DATA_TTL: 7 enough?
Should we remove all indices' alias before DELETE them?

POST /_aliases
{
    "actions": [
        { "remove": { "index": "segment-20200810", "alias": "segment" }}
    ]
}

Bug

docker image version oap : apache/skywalking-oap-server:7.0.0-es7
k8s deployment of oap :

      containers:
      - name: skywalking-oap
        image: "apache/skywalking-oap-server:7.0.0-es7"
        imagePullPolicy: IfNotPresent
        env:
        - name: JAVA_OPTS
          value: "-Xmx4g -Xms4g -Dmode=no-init"
        - name: SW_CLUSTER
          value: kubernetes
        - name: SW_CLUSTER_K8S_NAMESPACE
          value: "skywalking-prod"
        - name: SW_CLUSTER_K8S_LABEL
          value: "app=skywalking,release=skywalking-prod,component=skywalking-oap"
        - name: SKYWALKING_COLLECTOR_UID
          valueFrom:
            fieldRef:
              fieldPath: metadata.uid
        - name: SW_STORAGE
          value: elasticsearch7
        - name: SW_TRACE_SAMPLE_RATE
          value: "5000"
        - name: SW_CORE_DAY_METRIC_DATA_TTL
          value: "7"

We never recommended deleting the index by yourself. What is the background of your question?
TTL is the concept of the period of keeping data alive.


Please upload files for Nuscenes

Good work. Are you planning to upload a dataset_nuscenes and train_nuscenes files? I really want to test it.

Thanks for your interest in our work!
You can find our dataloader for nuScenes segmentation task in our PolarNet repo (https://github.com/edwardzhou130/PolarSeg/blob/master/dataloader/dataset_nuscenes.py). You will need to change the dataloader to load the panoptic nuScenes annotations rather than semantic annotations.
Our panoptic segmentation results on nuScenes in the paper were trained and evaluated based on our self-generated gt labels, which are different than the actual gt released officially by nuScenes (released after CVPR 2021). We currently have not plan to release those scripts and weights for this reason. However, You can find the reproduced results of our method on the panoptic nuscenes paper.

Thank You for your reply. But my concern is regarding preprocessing task. Can you upload instance_preprocess_nuscenes.pyfile? and I also want to point out for Panoptic you have used some augmentation so how can I use the code from Polarseg? can you guide me?

The process for nuScenes preprocessing is similar to SemanticKITTI. You will need to add the same function like this https://github.com/edwardzhou130/Panoptic-PolarNet/blob/3a72f2380a4e505e191b69da596f521a9d9f1a71/dataloader/dataset.py#L72-L116
to save all instance point clouds in a database. The differences between nuScenes and SemnticKITTI are:

The path to load and save the data.
The class label. Panoptic nuScenes saves the gt as 1000*semnatic + instance
You probably also need to create a similar yaml file for nuScenes.

@edwardzhou130  Thanks for your work. Is your generating panoptic label  same as 1000*semantic + instance?  And I also want to know if the pre-trained model is same as semantickitti. Thanks again.

@jwma0725 I used the same format in semanticKITTI (instance id is stored in the upper 16bits). The model is trained separately for semanticKITTI and nuscenes because their class and data distribution are different. But you can use the pretrained model in one dataset as the pretrained weight for the other (I did not try this though).

@edwardzhou130  Thanks for your reply. I still have some questions, as follows：

Can you tell me how to get pre-train model about Nuscenes? Or are you planning to upload the pre-train Nuscenes model?
Is there any difference between “Panoptic_SemKITTI.pt”model and “pretrained_weight/Panoptic_SemKITTI_PolarNet.pt”model？
Thanks!

We have no plan to upload our previous model because it is trained and evaluated on a different annotation than the official panoptic nuscenes.
pretrained_weight/Panoptic_SemKITTI_PolarNet.pt is the saved weight I got from running the training script. And it can be used to produce the result we reported in the paper.

Thank you for your response @edwardzhou130. I am trying as you told me but getting this error when I try to run dataloader.
 . I have changed lidarseg to panopticseg in line 124.


Encrypt Doesn't Work with URLs

It seems that the sls encrypt command doesn't store urls with certain characters in them. I tried making this a string by placing quotations around the value, but upon decrypting each time, the result is just an empty string. Is there a particular way to do this correctly? This would be to encrypt the postgresql database connection url for AWS

Would you have some example that fails to encrypt?


Can't run the requirements.txt

In this section,
-e git+https://git.pcl.ac.cn/ychuang/Mutual-Distillation.git@3b97956c34a2ba6ff49d71fc310a578aa03f6dab#egg=fairseq
i sign in the website
it said that he will send email in three hours,but can't receive the emial to valid it.

Hi, please delete this line of dependency in the requirements.txt.
I have updated the requirements.txt, thanks!

Hi, please delete this line of dependency in the requirements.txt.
I have updated the requirements.txt, thanks!

That's really nice,Thank you very much~~


Adding 2.15.0 release events to the website

Description
Adding 2.15.0 release events to the website
Issues Resolved
https://github.com/opensearch-project/opensearch-build/issues/4717
Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.

Hi @krisfreedain @nateynateynate could you help review this PR?
Thanks.


docs(material/stepper): add harness example for stepper

@annieyw it looks like this is somehow causing the stepper e2e test to time out. My guess is the e2e app page for stepper is just rendering all of the stepper examples, and this new one is probably causing some kind of exception on the page (pure guess with no actual investigation).

@annieyw it looks like this is somehow causing the stepper e2e test to time out. My guess is the e2e app page for stepper is just rendering all of the stepper examples, and this new one is probably causing some kind of exception on the page (pure guess with no actual investigation).

fixed e2e test timeout

fixed e2e test timeout


 For words with an apostrophe it seems to ignore the word break.
Repro steps;

Add the following to autocorrection_dict.txt

:htere        -> there
:cant         -> can't

Run the python file and compile firmware

Type " htere"
 Expected result: there
 Actual result: there
 Result: WORKING

Type "testhtere"
Expected result: testhtere
Actual result: testhtere
Result: WORKING

Type "  cant"
Expected result: can't
Actual result: can't
Result: WORKING

Type " testcant"
Expected result: testcant
Actual result: testcan't
Result: BROKEN

Thanks for the report! I appreciate the clear examples.
I haven't been able to reproduce the bug yet, unfortunately. I tried appending
:htere -> there
:cant -> can't

to my dictionary, and also tried a dictionary containing only those two entries, yet typing "testcant" isn't triggering autocorrection.
Would you mind to share your full dictionary? (You can make attachments in this thread.) The bug is conceivably a problem in trie encoding.
Do you have special QMK features like mod-taps, Tap Dance, Auto Shift, etc. active on the keys C, A, N, T? Interoperating with QMK's features can be tricky.


Cannot create channel after closing the test-network by ./network.sh down command.

Cannot create channel after close the test-network by ./network.sh down command.
Status: 405
{
        "error": "cannot join: channel already exists"
}

I checked the docker volume, found there were three volumes left after executing ./network.sh down command.

compose_orderer.example.com
compose_peer0.org1.example.com
compose_peer0.org2.example.com

I fixed the problem by changing line 343,
${CONTAINER_CLI} volume rm docker_orderer.example.com docker_peer0.org1.example.com docker_peer0.org2.example.com

to
${CONTAINER_CLI} volume rm compose_orderer.example.com compose_peer0.org1.example.com compose_peer0.org2.example.com

FYI.

Using the test-network in the current main branch, I can stop a running network with ./network.sh down and then start a new network with (for example using Fabric v3) ./network.sh up -bft && ./network.sh createChannel -bft, and I do not observe any failures.
I notice that the network.sh script still includes the lines:
${CONTAINER_CLI} volume rm docker_orderer.example.com docker_peer0.org1.example.com docker_peer0.org2.example.com

I'm not sure what changed since this issue was reported, or between my system and the one where the problem was observed.
Since I can't reproduce the problem, I am going to close this issue. Please comment with more details on how to reproduce if you still observe a problem.


fix: add support for typescript 2.6

Codecov Report

Merging #74 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master     #74      +/-   ##
=========================================
+ Coverage    99.6%   99.6%   +<.01%
=========================================
  Files           8       8
  Lines         251     254       +3
  Branches       10      10
=========================================
+ Hits          250     253       +3
  Misses          1       1

Impacted Files
Coverage Δ

test/test-kitchen.ts
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update daaa1a6...7f9a66c. Read the comment docs.

TBR= @kjin


[cpu-features] tools does not build on arm64-osx

It was fixed by https://github.com/google/cpu_features/commit/c5ece5e8ccd64f0da935fceb14f9ab8a3e2eefa0
We need revert this change after upstream release 0.9.0

Thanks!


COSBETA-184 Add pages to route users to bulk or manual journey

Description

Checklist:

[ ] Automated checks are passing locally.
[ ] CHANGELOG updated if change is worth telling users about.

General testing

[ ] Test without javascript
[ ] Test on small screen

Accessibility testing

[ ] Works keyboard only
[ ] Tested with one screen reader
[ ] Zoom page to 400% - content still visible
[ ] Disable css - does content make sense and appear in a logical order?
[ ] Passes automated checker (automated in build or manual)

Ah, I've accidentally included the commits from COSBETA-167 in here :( Please ignore them until that's merged


Simplify meteor models

Is your feature request related to a problem? Please describe.
The level of nesting in meteor models is a lot. Where multiple models are nested within each other. which makes it hard for extractor to serialise their data. Also, current models are in different lines compared to compass.
Describe the solution you'd like
We can simplify models into a more simplified approach. where asset is one models which can have table, topic and other models as one of its type.

@bsushmith fyi
We are using odpf/proton Asset model which just got updated recently in these two PRs respectively.
https://github.com/odpf/proton/pull/161

move from having multiple models (Table, Dashboard, etc) to one main Asset model that wrap the model (Table, Dashboard, etc) inside Data field.
All common fields across models is moved to Asset model.

https://github.com/odpf/proton/pull/174

Move Asset.Attributes to Asset.Data.Attributes


Add JSDoc comments to functions

This SDK has many functions that don't have comments, making the code more difficult to understand. Sometimes the comments don't make sense or have spelling errors.
Ideally we add comments to the functions.
Example: receiver_structured.js

I agree - JSDoc is super important. Ideally, I'd like to work to narrow and simplify the exposed API so that we can know best how to prioritize effort when documenting functions. Maybe we can start with the CloudEvent and HTTPReceiver classes, since those are the most accessible/easiest to use parts of the API at the moment.

Now that we have JSDoc everywhere and are publishing it, this can be closed.


You should only be using filter to make the new array wont pass.

Challenge filter-arrays-with-filter has an issue.
User Agent is: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36.
Please describe how to reproduce this issue, and include links to screenshots if possible.

var oldArray = [1,2,3,4,5,6,7,8,9,10];

// Only change code below this line.

var newArray = oldArray.filter(function(val){
return val < 6;
});

    ```

Thanks for the issue! Closing as duplicate of #10589


@material-ui/icons locations.

I was re-installing my OS for work over this weekend and when I installed this package with yarn and tried to run my projects I get the following errors
Module not found: Error: Can't resolve '@material-ui/icons/Close'
Module not found: Can't resolve './AcUnit' in '@material-ui/icons'
After further looking into this issue and looking at the repo here (in Github)  it looks like everything alphabetically from A - G got put into icons/esm/ in the published package.  Which all of those should be in icons/ correct?
Not sure if there was a mapping problem in the last push/build to the repo?  I've made sure to remove my node_modules and delete my yarn.lock file and re-install the package with the same results.
This last week sometime I re-installed my node_modules and everything worked v4.2.1  re-installing them today (same version)  I get the above errors when trying to run the projects.
If I change the path to the Close and add in the /esm/ it resolves fine after that.
Let me know if the docs need to be updated to use the new route?
Thank you!

@seth-seikosoft Noting has changed with the package.

@seth-seikosoft Something must be wrong in your env. The import you are mentioning resolves correctly, proof: https://unpkg.com/@material-ui/icons/Close.

Thank you both @mbrookes  @oliviertassinari for getting back to me.  After going over my system I ended up reverting to a backup and things are working correctly now.  Not sure what happened because I un-installed and reinstalled everything related to node.
thank you guys again for your quick responses and for putting out a stellar product!


GUI does not load correctly on KDE/Wayland

Current behavior
Starting the app does not load the GUI to the end. No layout. Can't click on anything. It happens in most cases here.
Version: aur/devtoys-bin <IP_ADDRESS>-1 (+7 3.55)
System Details:
Operating System: EndeavourOS
KDE Plasma Version: 6.1.2
KDE Frameworks Version: 6.4.0
Qt Version: 6.7.2
Kernel Version: 6.9.9-arch1-1 (64-bit)
Graphics Platform: Wayland
Processors: 16 × AMD Ryzen 7 PRO 4750U with Radeon Graphics
Memory: 30.6 GiB of RAM
Graphics Processor: AMD Radeon Graphics
Manufacturer: LENOVO
Product Name: 20UES00L00
System Version: ThinkPad T14 Gen 1
How to reproduce it (as minimally and precisely as possible)
Just start via Icon or from Terminal with DevToys
Expected behavior
show gui always correctly
Screenshots

Workaround
none. try several times.
Affected platforms
Linux
Affected DevToys kind
DevToys (app with GUI)
DevToys Version
Version 2.0-preview.3
Relevant Assets/Logs
2024-07-18T11:02:32.2635418+02:00	Information	[DevToys.Core.FileHelper]	[0]	Cleared temp files in 0.5974ms
2024-07-18T11:02:32.2719675+02:00	Information	[DevToys.Core.Mef.MefComposer]	[LogDiscoveringPlugin]	Discovering plugin in '/opt/devtoys/devtoys/Plugins/DevToys.Tools'...
2024-07-18T11:02:32.4113793+02:00	Information	[DevToys.Core.Mef.MefComposer]	[LogMefComposition]	MEF composed 112 parts from 6 assemblies in 141.3019ms
2024-07-18T11:02:32.4314566+02:00	Information	[DevToys.Linux.LinuxProgram]	[2]	MEF, services and logging initialized in 427.4289 ms
2024-07-18T11:02:32.4315292+02:00	Information	[DevToys.Linux.LinuxProgram]	[1]	App is starting...
2024-07-18T11:02:32.4656187+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'Base64ImageEncoderDecoder' tool instance manager.
2024-07-18T11:02:32.4656762+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'Base64TextEncoderDecoder' tool instance manager.
2024-07-18T11:02:32.4656808+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'CertificateDecoder' tool instance manager.
2024-07-18T11:02:32.4656833+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'ColorBlindnessSimulator' tool instance manager.
2024-07-18T11:02:32.4656865+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'CronParser' tool instance manager.
2024-07-18T11:02:32.4656895+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'DateConverter' tool instance manager.
2024-07-18T11:02:32.4656919+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'EscapeUnescape' tool instance manager.
2024-07-18T11:02:32.4656949+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'GZipEncoderDecoder' tool instance manager.
2024-07-18T11:02:32.4656974+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'HashAndChecksumGenerator' tool instance manager.
2024-07-18T11:02:32.4657006+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'HtmlEncoderDecoder' tool instance manager.
2024-07-18T11:02:32.4657041+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'ImageConverter' tool instance manager.
2024-07-18T11:02:32.4657068+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'JsonFormatter' tool instance manager.
2024-07-18T11:02:32.4657091+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'JSONPathTester' tool instance manager.
2024-07-18T11:02:32.4657123+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'JsonTableConverter' tool instance manager.
2024-07-18T11:02:32.4657148+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'JsonWebTokenEncoderDecoder' tool instance manager.
2024-07-18T11:02:32.4657172+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'JsonYamlConverter' tool instance manager.
2024-07-18T11:02:32.4657205+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'ListCompare' tool instance manager.
2024-07-18T11:02:32.4657234+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'LoremIpsumGenerator' tool instance manager.
2024-07-18T11:02:32.4657259+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'MarkdownPreview' tool instance manager.
2024-07-18T11:02:32.4657290+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'NumberBaseConverter' tool instance manager.
2024-07-18T11:02:32.4657315+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'PasswordGenerator' tool instance manager.
2024-07-18T11:02:32.4657339+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'QRCodeEncoderDecoder' tool instance manager.
2024-07-18T11:02:32.4657374+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'RegExTester' tool instance manager.
2024-07-18T11:02:32.4657399+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'SqlFormatter' tool instance manager.
2024-07-18T11:02:32.4657422+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'TextAnalyzerAndUtilities' tool instance manager.
2024-07-18T11:02:32.4657452+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'TextCompare' tool instance manager.
2024-07-18T11:02:32.4657475+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'UrlEncoderDecoder' tool instance manager.
2024-07-18T11:02:32.4657500+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'UUIDGenerator' tool instance manager.
2024-07-18T11:02:32.4657530+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'XmlFormatter' tool instance manager.
2024-07-18T11:02:32.4657556+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'XMLTester' tool instance manager.
2024-07-18T11:02:32.4664908+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'Extensions Manager' tool instance manager.
2024-07-18T11:02:32.4665084+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'Settings' tool instance manager.
2024-07-18T11:02:32.4669335+02:00	Information	[DevToys.Core.Tools.GuiToolInstance]	[LogInitialized]	Initialized 'NoSearchResults' tool instance manager.
2024-07-18T11:02:32.4689385+02:00	Information	[DevToys.Core.Tools.GuiToolProvider]	[LogToolInstancesCreated]	Instantiated 32 tools in 6.0628ms
2024-07-18T11:02:33.3740947+02:00	Information	[DevToys.Blazor.Core.Services.FontService]	[LogImportThirdPartyFontsAsync]	Loaded 2 font(s) in 22.2233 ms and injected them in 217.7792 ms
2024-07-18T11:02:36.8419088+02:00	Information	[DevToys.Core.Settings.SettingsProvider]	[LogSetSetting]	Setting 'DevToys.Blazor.UserPreferredNavBarState' changed to 'Expanded'
2024-07-18T11:02:36.8479024+02:00	Information	[DevToys.Core.FileHelper]	[0]	Cleared temp files in 0.0166ms

I confirm that problem in

Linux Mint 21.3
Cinnamon 6.0

Hello,
I do not reproduce the issue on Mint 21.3 on my side. Does the issue happen all the time or occasionally? You said "in most case", so I assume it's not all the time, right?

As the initial bug report states that the distribution is KDE based I wonder if there are perhaps some packages not installed which result in this kind of rendering?
Perhaps installing some other libadwaita app helps here (from the distro not flathub) to ensure there is at least one other libadwaita app running without problems.

I could reproduce it on my Gnome machine, too.
It's running Endeavour OS with Wayland and Gnome - all latest packages and same "visual misbehaviour".
First start, same look as in my first screenshot in post#1, second start okay. The ratio between works:does_not is better with Gnome than with Plasma.
But I cannot provide better logs that in post#1

As the initial bug report states that the distribution is KDE based I wonder if there are perhaps some packages not installed which result in this kind of rendering?
Perhaps installing some other libadwaita app helps here (from the distro not flathub) to ensure there is at least one other libadwaita app running without problems.

libadwaita is installed - and if the issue is a missing package then the error should be 100% reproduceable. But it works in 1 out of ~10 cases on Plasma and in 1 out of 2 under Gnome. When opening the program it seems as if loading of layout does not finish. I sometimes have seen this if a webpage does not load to the end (mayb be CSS missing or JavaScript sources) then it looks incomplete and like in my screenshot.
Can logging be set to a more verbose level to see this? Or may be in a next version to investigate further?

Yeah, it definitely sounds like a race condition

I am getting a similar issue after installing the latest version through the AUR running hyprland


CSV Example

I am following the CSV Example.  The schema includes a header of "header": "^DATE|HIGH TEMP C|LOW TEMP F|WIND DIR|WIND SPEED KMH|NOTE|LAT|LONG|UV INDEX$".
In package csv, decl.go's RowsBased (called from reader.go ReadAndMatch) function says if r.Header == nil then it's rows based but the line above makes the header not nil? So now it thinks my CSV is Header and Footer based.
What am I missing?

@bjordan2010
Good question. I assume you're following the example in Getting Started. If you look closely at the final schema in that doc page, you'll see there are two records under the file_declaration.records section: one for the header line and the other for the data lines.
In csv (actually csv2 to be more precise) schema, you can declare multiple (and nested if needed) records, each of which can be either rows based or header/footer based, independently. The schema and parser work together like a state machine, allowing it to read, match, process, transform, and transition between records.
In the example, we have two records, the first one:
            {
                "header": "^DATE|HIGH TEMP C|LOW TEMP F|WIND DIR|WIND SPEED KMH$",
                "min": 1, "max": 1
            }

It says, the record is header/footer based (in this case, no footer is specified, thus as long as header regex matches, the current line of data is identified with/matched to this record. Also this record has min/max both = 1, which means after a line matches to the record, the next line will automatically move on and attempt to match subsequent records: the state machine transition code for this is in HierarchyReader.recDone() function, where recDone() is called when the current data has been fully matched to the record and finished processing thus needing to move to the next record (by calling recNext() when the number of matched and processed instances of the current record has exceeded the declared max).
Once the state machine moved to the next record, which in the case of this example is:
            {
                "is_target": true,
                "columns": [
                    { "name": "DATE" },
                    { "name": "HIGH_TEMP_C" },
                    { "name": "LOW_TEMP_F" },
                    { "name": "WIND_DIR" },
                    { "name": "WIND_SPEED_KMH" }
                ]
            }

Lots of default omissions here (all defaults are explained in the file_declaration section):

rows omitted, thus defaulted to 1.
min/max omitted, thus defaulted to 0/-1 respectively.
Now the state machine will pretty much loop on this record forever, processing each data line as a record.

Hopefully this explains how the csv processing state machine transitions.
For detailed usage reference: CSV Schema in Depth (also linked from the root README.md).
For code inspection/review, I suggest you take a close look at the package flatfile, where we have two main interfaces: RecDecl and RecReader and the core state machine reader HierarchyReader. Once you understand how this package works, the rest (currently we only have csv and fixedlength) simply fall into place.

Thanks for responding.  I have narrowed my problem down to the min: 1 max:
1 setting.  Apparently I have something wrong in my header which I am
building dynamically based on the file.  I'm going to research the problem
this morning and I'll let you know if it's a bug on my end.  I also has to
specify is_target: false on the header row otherwise it would process it
but that may be related to me removing the min/max while troubleshooting.
I expect CSV to be processed as row based not header / footer based, is
that a correct assumption or will it be processed as a header / footer with
no footer?  If we get this working I'm going to ask my client to donate, if
they will not, then I will myself.  I appreciate the work you have put into
this project.
Thanks,
Jesse
On Wed, Nov 2, 2022 at 7:38 PM JF Technology @.***>
wrote:

@bjordan2010 https://github.com/bjordan2010
Good question. I assume you're following the example in Getting Started
https://github.com/jf-tech/omniparser/blob/master/doc/gettingstarted.md.
If you look closely at the final schema
https://github.com/jf-tech/omniparser/blob/master/doc/gettingstarted.md#the-schema
in that doc page, you'll see there are two records under the
file_declaration.records section: one for the header line and the other
for the data lines.
In csv (actually csv2 to be more precise) schema, you can declare
multiple (and nested if needed) records, each of which can be either rows
based or header/footer based, independently. The schema and parser work
together like a state machine, allowing it to read, match, process,
transform, and transition between records.
In the example, we have two records, the first one:
        {
            "header": "^DATE|HIGH TEMP C|LOW TEMP F|WIND DIR|WIND SPEED KMH$",
            "min": 1, "max": 1
        }

It says, the record is header/footer based (in this case, no footer is
specified, thus as long as header regex matches, the current line of data
is identified with/matched to this record. Also this record has min/max
both = 1, which means after a line matches to the record, the next line
will automatically move on and attempt to match subsequent records: the
state machine transition code for this is in HierarchyReader.recDone()
function
https://github.com/jf-tech/omniparser/blob/master/extensions/omniv21/fileformat/flatfile/hierarchyReader.go#L240,
where recDone() is called when the current data has been fully matched to
the record and finished processing thus needing to move to the next record
(by calling recNext() when the number of matched and processed instances
of the current record has exceeded the declared max).
Once the state machine moved to the next record, which in the case of this
example is:
        {
            "is_target": true,
            "columns": [
                { "name": "DATE" },
                { "name": "HIGH_TEMP_C" },
                { "name": "LOW_TEMP_F" },
                { "name": "WIND_DIR" },
                { "name": "WIND_SPEED_KMH" }
            ]
        }

Lots of default omissions here (all defaults are explained in the
file_declaration
https://github.com/jf-tech/omniparser/blob/master/doc/csv2_in_depth.md#csv-file_declaration
section):

rows omitted, thus defaulted to 1.
min/max omitted, thus defaulted to 0/-1 respectively.
Now the state machine will pretty much loop on this record forever,
processing each data line as a record.

Hopefully this explains how the csv processing state machine transitions.
For detailed usage reference: CSV Schema in Depth
https://github.com/jf-tech/omniparser/blob/master/doc/csv2_in_depth.md
(also linked from the root README.md
https://github.com/jf-tech/omniparser/blob/master/README.md#documentation
).
For code inspection/review, I suggest you take a close look at the package
flatfile
https://github.com/jf-tech/omniparser/tree/master/extensions/omniv21/fileformat/flatfile,
where we have two main interfaces: RecDecl and RecReader and the core
state machine reader HierarchyReader. Once you understand how this
package works, the rest (currently we only have csv and fixedlength)
simply fall into place.
—
Reply to this email directly, view it on GitHub
https://github.com/jf-tech/omniparser/issues/182#issuecomment-1301524179,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAFL6RGAPY5D6R6SR62Y7JLWGMCPBANCNFSM6AAAAAARVH5P3E
.
You are receiving this because you were mentioned.Message ID:
@.***>

I am getting the following error when processing file_format_type csv2:
input 'my_file' line 1: record/record_group '' needs min occur 1, but only
got 0
My header in schema:
""^FillDate,PharmacyID,PharmacyName,PHARMACY ZIP
CODE,PRESCRIPTIONSERVICEREFERENCENU,NDC11,NameNameExt,GenericDrugName,Brnd
Gen Compound,BG,Compound,OTC Indicator,MAC Price,Formulary,Mail
Retail,Channel,NUMBEROFREFILLSAUTHORIZED,Qty,Days,RxCount,ClaimStatus,INGREDIENTCOSTSUBMITTED,AWP100,UNC,IngredCost,DispFee,SalesTax,Gross,PlanPaidAmount,PatientPaidAmount,RebillAmtApplPerDedu,Mbr
Pay minus Ded,DAWCode$,"
My header row in the CSV file:
FillDate,PharmacyID,PharmacyName,PHARMACY ZIP
CODE,PRESCRIPTIONSERVICEREFERENCENU,NDC11,NameNameExt,GenericDrugName,Brnd
Gen Compound,BG,Compound,OTC Indicator,MAC Price,Formulary,Mail
Retail,Channel,NUMBEROFREFILLSAUTHORIZED,Qty,Days,RxCount,ClaimStatus,INGREDIENTCOSTSUBMITTED,AWP100,UNC,IngredCost,DispFee,SalesTax,Gross,PlanPaidAmount,PatientPaidAmount,RebillAmtApplPerDedu,Mbr
Pay minus Ded,DAWCode
My full file_declaration section:
"file_declaration": {
"delimiter": ",",
"records": [
{
"header": "^FillDate,PharmacyID,PharmacyName,PHARMACY ZIP
CODE,PRESCRIPTIONSERVICEREFERENCENU,NDC11,NameNameExt,GenericDrugName,Brnd
Gen Compound,BG,Compound,OTC Indicator,MAC
Price,Formulary,Mail/Retail,Channel,NUMBEROFREFILLSAUTHORIZED,Qty,Days,RxCount,ClaimStatus,INGREDIENTCOSTSUBMITTED,AWP100,UNC,IngredCost,DispFee,SalesTax,Gross,PlanPaidAmount,PatientPaidAmount,RebillAmtApplPerDedu,Mbr
Pay minus Ded,DAWCode$",
"min": 1, "max": 1, "is_target": false
},
{
"is_target": true,
"columns": [
{ "name": "FillDate" },
{ "name": "PharmacyID" },
{ "name": "PharmacyName" },
{ "name": "PHARMACY_ZIP_CODE" },
{ "name": "PRESCRIPTIONSERVICEREFERENCENU" },
{ "name": "NDC11" },
{ "name": "NameNameExt" },
{ "name": "GenericDrugName" },
{ "name": "Brnd_Gen_Compound" },
{ "name": "BG" },
{ "name": "Compound" },
{ "name": "OTC_Indicator" },
{ "name": "MAC_Price" },
{ "name": "Formulary" },
{ "name": "Mail Retail" },
{ "name": "Channel" },
{ "name": "NUMBEROFREFILLSAUTHORIZED" },
{ "name": "Qty" },
{ "name": "Days" },
{ "name": "RxCount" },
{ "name": "ClaimStatus" },
{ "name": "INGREDIENTCOSTSUBMITTED" },
{ "name": "AWP100" },
{ "name": "UNC" },
{ "name": "IngredCost" },
{ "name": "DispFee" },
{ "name": "SalesTax" },
{ "name": "Gross" },
{ "name": "PlanPaidAmount" },
{ "name": "PatientPaidAmount" },
{ "name": "RebillAmtApplPerDedu" },
{ "name": "Mbr_Pay_minus_Ded" },
{ "name": "DAWCode" }
]
}
]
},
If I put both of the values for the header in a file, and I search on the
first one without the double quotes, ^, and $, they match, so I'm not sure
why I'm getting the error.  I will continue to debug but following the
Getting Started example this should work even without "is_target": false.
The only differences I see is the example uses a pipe | delimiter and far
fewer fields.
Thanks,
Jesse
On Wed, Nov 2, 2022 at 7:38 PM JF Technology @.***>
wrote:

@bjordan2010 https://github.com/bjordan2010
Good question. I assume you're following the example in Getting Started
https://github.com/jf-tech/omniparser/blob/master/doc/gettingstarted.md.
If you look closely at the final schema
https://github.com/jf-tech/omniparser/blob/master/doc/gettingstarted.md#the-schema
in that doc page, you'll see there are two records under the
file_declaration.records section: one for the header line and the other
for the data lines.
In csv (actually csv2 to be more precise) schema, you can declare
multiple (and nested if needed) records, each of which can be either rows
based or header/footer based, independently. The schema and parser work
together like a state machine, allowing it to read, match, process,
transform, and transition between records.
In the example, we have two records, the first one:
        {
            "header": "^DATE|HIGH TEMP C|LOW TEMP F|WIND DIR|WIND SPEED KMH$",
            "min": 1, "max": 1
        }

It says, the record is header/footer based (in this case, no footer is
specified, thus as long as header regex matches, the current line of data
is identified with/matched to this record. Also this record has min/max
both = 1, which means after a line matches to the record, the next line
will automatically move on and attempt to match subsequent records: the
state machine transition code for this is in HierarchyReader.recDone()
function
https://github.com/jf-tech/omniparser/blob/master/extensions/omniv21/fileformat/flatfile/hierarchyReader.go#L240,
where recDone() is called when the current data has been fully matched to
the record and finished processing thus needing to move to the next record
(by calling recNext() when the number of matched and processed instances
of the current record has exceeded the declared max).
Once the state machine moved to the next record, which in the case of this
example is:
        {
            "is_target": true,
            "columns": [
                { "name": "DATE" },
                { "name": "HIGH_TEMP_C" },
                { "name": "LOW_TEMP_F" },
                { "name": "WIND_DIR" },
                { "name": "WIND_SPEED_KMH" }
            ]
        }

Lots of default omissions here (all defaults are explained in the
file_declaration
https://github.com/jf-tech/omniparser/blob/master/doc/csv2_in_depth.md#csv-file_declaration
section):

rows omitted, thus defaulted to 1.
min/max omitted, thus defaulted to 0/-1 respectively.
Now the state machine will pretty much loop on this record forever,
processing each data line as a record.

Hopefully this explains how the csv processing state machine transitions.
For detailed usage reference: CSV Schema in Depth
https://github.com/jf-tech/omniparser/blob/master/doc/csv2_in_depth.md
(also linked from the root README.md
https://github.com/jf-tech/omniparser/blob/master/README.md#documentation
).
For code inspection/review, I suggest you take a close look at the package
flatfile
https://github.com/jf-tech/omniparser/tree/master/extensions/omniv21/fileformat/flatfile,
where we have two main interfaces: RecDecl and RecReader and the core
state machine reader HierarchyReader. Once you understand how this
package works, the rest (currently we only have csv and fixedlength)
simply fall into place.
—
Reply to this email directly, view it on GitHub
https://github.com/jf-tech/omniparser/issues/182#issuecomment-1301524179,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAFL6RGAPY5D6R6SR62Y7JLWGMCPBANCNFSM6AAAAAARVH5P3E
.
You are receiving this because you were mentioned.Message ID:
@.***>

I may be wrong but here is what I found in debugging. In decl.go's
matchLine function, line.raw gets passed in to re.MatchString which calls
doMatch will a nil io.RuneReader which calls doExecute with the same nil r,
which second line is return nil if r == nil.  Meaning no match.  Is this a
bug in MatchString?
On Wed, Nov 2, 2022 at 7:38 PM JF Technology @.***>
wrote:

@bjordan2010 https://github.com/bjordan2010
Good question. I assume you're following the example in Getting Started
https://github.com/jf-tech/omniparser/blob/master/doc/gettingstarted.md.
If you look closely at the final schema
https://github.com/jf-tech/omniparser/blob/master/doc/gettingstarted.md#the-schema
in that doc page, you'll see there are two records under the
file_declaration.records section: one for the header line and the other
for the data lines.
In csv (actually csv2 to be more precise) schema, you can declare
multiple (and nested if needed) records, each of which can be either rows
based or header/footer based, independently. The schema and parser work
together like a state machine, allowing it to read, match, process,
transform, and transition between records.
In the example, we have two records, the first one:
        {
            "header": "^DATE|HIGH TEMP C|LOW TEMP F|WIND DIR|WIND SPEED KMH$",
            "min": 1, "max": 1
        }

It says, the record is header/footer based (in this case, no footer is
specified, thus as long as header regex matches, the current line of data
is identified with/matched to this record. Also this record has min/max
both = 1, which means after a line matches to the record, the next line
will automatically move on and attempt to match subsequent records: the
state machine transition code for this is in HierarchyReader.recDone()
function
https://github.com/jf-tech/omniparser/blob/master/extensions/omniv21/fileformat/flatfile/hierarchyReader.go#L240,
where recDone() is called when the current data has been fully matched to
the record and finished processing thus needing to move to the next record
(by calling recNext() when the number of matched and processed instances
of the current record has exceeded the declared max).
Once the state machine moved to the next record, which in the case of this
example is:
        {
            "is_target": true,
            "columns": [
                { "name": "DATE" },
                { "name": "HIGH_TEMP_C" },
                { "name": "LOW_TEMP_F" },
                { "name": "WIND_DIR" },
                { "name": "WIND_SPEED_KMH" }
            ]
        }

Lots of default omissions here (all defaults are explained in the
file_declaration
https://github.com/jf-tech/omniparser/blob/master/doc/csv2_in_depth.md#csv-file_declaration
section):

rows omitted, thus defaulted to 1.
min/max omitted, thus defaulted to 0/-1 respectively.
Now the state machine will pretty much loop on this record forever,
processing each data line as a record.

Hopefully this explains how the csv processing state machine transitions.
For detailed usage reference: CSV Schema in Depth
https://github.com/jf-tech/omniparser/blob/master/doc/csv2_in_depth.md
(also linked from the root README.md
https://github.com/jf-tech/omniparser/blob/master/README.md#documentation
).
For code inspection/review, I suggest you take a close look at the package
flatfile
https://github.com/jf-tech/omniparser/tree/master/extensions/omniv21/fileformat/flatfile,
where we have two main interfaces: RecDecl and RecReader and the core
state machine reader HierarchyReader. Once you understand how this
package works, the rest (currently we only have csv and fixedlength)
simply fall into place.
—
Reply to this email directly, view it on GitHub
https://github.com/jf-tech/omniparser/issues/182#issuecomment-1301524179,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAFL6RGAPY5D6R6SR62Y7JLWGMCPBANCNFSM6AAAAAARVH5P3E
.
You are receiving this because you were mentioned.Message ID:
@.***>

Meh it was a simple bug.  I had inadvertently placed the , for the header
line inside the quote when dynamically building the header line.  Sorry to
waste your time but I appreciate your prior response.
Thanks,
Jesse
On Thu, Nov 3, 2022 at 8:31 AM J. Brian Jordan @.***> wrote:

I may be wrong but here is what I found in debugging. In decl.go's
matchLine function, line.raw gets passed in to re.MatchString which calls
doMatch will a nil io.RuneReader which calls doExecute with the same nil r,
which second line is return nil if r == nil.  Meaning no match.  Is this a
bug in MatchString?
On Wed, Nov 2, 2022 at 7:38 PM JF Technology @.***>
wrote:

@bjordan2010 https://github.com/bjordan2010
Good question. I assume you're following the example in Getting Started
https://github.com/jf-tech/omniparser/blob/master/doc/gettingstarted.md.
If you look closely at the final schema
https://github.com/jf-tech/omniparser/blob/master/doc/gettingstarted.md#the-schema
in that doc page, you'll see there are two records under the
file_declaration.records section: one for the header line and the other
for the data lines.
In csv (actually csv2 to be more precise) schema, you can declare
multiple (and nested if needed) records, each of which can be either rows
based or header/footer based, independently. The schema and parser work
together like a state machine, allowing it to read, match, process,
transform, and transition between records.
In the example, we have two records, the first one:
        {
            "header": "^DATE|HIGH TEMP C|LOW TEMP F|WIND DIR|WIND SPEED KMH$",
            "min": 1, "max": 1
        }

It says, the record is header/footer based (in this case, no footer is
specified, thus as long as header regex matches, the current line of
data is identified with/matched to this record. Also this record has min/
max both = 1, which means after a line matches to the record, the next
line will automatically move on and attempt to match subsequent records:
the state machine transition code for this is in HierarchyReader.recDone()
function
https://github.com/jf-tech/omniparser/blob/master/extensions/omniv21/fileformat/flatfile/hierarchyReader.go#L240,
where recDone() is called when the current data has been fully matched
to the record and finished processing thus needing to move to the next
record (by calling recNext() when the number of matched and processed
instances of the current record has exceeded the declared max).
Once the state machine moved to the next record, which in the case of
this example is:
        {
            "is_target": true,
            "columns": [
                { "name": "DATE" },
                { "name": "HIGH_TEMP_C" },
                { "name": "LOW_TEMP_F" },
                { "name": "WIND_DIR" },
                { "name": "WIND_SPEED_KMH" }
            ]
        }

Lots of default omissions here (all defaults are explained in the
file_declaration
https://github.com/jf-tech/omniparser/blob/master/doc/csv2_in_depth.md#csv-file_declaration
section):

rows omitted, thus defaulted to 1.
min/max omitted, thus defaulted to 0/-1 respectively.
Now the state machine will pretty much loop on this record forever,
processing each data line as a record.

Hopefully this explains how the csv processing state machine transitions.
For detailed usage reference: CSV Schema in Depth
https://github.com/jf-tech/omniparser/blob/master/doc/csv2_in_depth.md
(also linked from the root README.md
https://github.com/jf-tech/omniparser/blob/master/README.md#documentation
).
For code inspection/review, I suggest you take a close look at the
package flatfile
https://github.com/jf-tech/omniparser/tree/master/extensions/omniv21/fileformat/flatfile,
where we have two main interfaces: RecDecl and RecReader and the core
state machine reader HierarchyReader. Once you understand how this
package works, the rest (currently we only have csv and fixedlength)
simply fall into place.
—
Reply to this email directly, view it on GitHub
https://github.com/jf-tech/omniparser/issues/182#issuecomment-1301524179,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAFL6RGAPY5D6R6SR62Y7JLWGMCPBANCNFSM6AAAAAARVH5P3E
.
You are receiving this because you were mentioned.Message ID:
@.***>

@bjordan2010
A couple thoughts/tips:

check out the "CSV Schema in Depth" doc which contains many scenarios - sometimes the easiest the way is to simply skip header row verification (https://github.com/jf-tech/omniparser/blob/master/doc/csv2_in_depth.md#use-case-simple-csv-with-header-but-header-verification-not-needed)

note the header match is regex based so you might find using tool like https://regex101.com/ to test your header pattern in your schema against the actual header in your data file to see if they match or not.

Appreciate you kindness! Let me know if any other problems you need help with.

@bjordan2010, Do you still have any issues? Can we close the ticket now?

@jf-tech All is well, thanks for your help.


Not working when not connected to Serial

The WiFi Manager doesn't work when powering the ESP32 over its VIN Port, but does when powering it through USB. I'm simply using the Code from the Async_AutoConnect_ESP32_minimal.ino example. I already remove the while (!Serial); delay(200); line. Is there a simple fix for this?

Turns out there is a more general problem with powering the ESP32 on the VIN Pins, which has nothing to do with the library.

I've just been back home and seen this and am happy you did your thorough research/experiment and found out the issue before I would inform you.
This is good experience for other new users, and will be definitely helpful. Don't hesitate to post any issue, correct or not, as they are all beneficial somehow to other users.
Good Luck with your project,


step4: add frontend, tests and circleci config

Codecov Report

Merging #7 (fdc0f3f) into main (9772d8c) will decrease coverage by 23.48%.
The diff coverage is 20.54%.

@@             Coverage Diff             @@
##             main       #7       +/-   ##
===========================================
- Coverage   63.33%   39.84%   -23.49%
===========================================
  Files           3        4        +1
  Lines          60      133       +73
  Branches        0       21       +21
===========================================
+ Hits           38       53       +15
- Misses         22       69       +47
- Partials        0       11       +11

Impacted Files
Coverage Δ

web/static/js/calculator.js
20.54% <20.54%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Django 2.0 support

Tox tests were successful.
Please check following:

tox.ini: I have changed tox environment from py 3.5 to 3.6, I have added environment for dj 1.11, dj 2.0.

There is problem with earlier url_patterns.regex.pattern, used on more places in generator, test_command and test_generator. I think it could be replaced with str(url_patterns.pattern) in dj 2.0+. Temporary I have used common function get_pattern() which I suggest to use as long as single writing is not possible. But maybe you have better solution.

Best regards,
Mirek

Codecov Report

Merging #1 into master will decrease coverage by 0.89%.
The diff coverage is 80%.

@@            Coverage Diff            @@
##           master       #1     +/-   ##
=========================================
- Coverage   96.23%   95.33%   -0.9%
=========================================
  Files           5        5
  Lines         186      193      +7
  Branches       24       25      +1
=========================================
+ Hits          179      184      +5
- Misses          6        7      +1
- Partials        1        2      +1

Impacted Files
Coverage Δ

django_smoke_tests/generator.py
94.69% <80%> (-1.31%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3ceac3c...8383dab. Read the comment docs.


Feature request: min-height and min-width

Imgix has a neat function to specify a minimum height and width.

That functionality would be awesome to see in imgproxy.
More about the feature here

@iaK
Hi, I'm just another imgproxy user here.
I took a look at the link you provided on minimum-width, and I think this feature is currently available in imgproxy under a different name.
imgix sample
Let me reiterate the example to make sure I got it right.
Minimum-Width

Given an original image that is
https://assets.imgix.net/unsplash/alley.jpg
3744×5616
When specifying fit=crop&h=200&min-w=300
you would expect the image to be cropped and then stretched to fit a width of 300x200
This seems to be different than what you described:

It enables you to resize smaller images, while bigger images are untouched.

Imgproxy Equivalent
I wonder if setting resize:fill provides you with the desired result.
https://docs.imgproxy.net/#/generating_the_url_advanced?id=resizing-type
Sample Resizer on imgproxy.net

Thanks for your answer @legopin!
What min-width and min-height brings to the table is that larger images will be unaffected. Say if you have one 100x100 image and a 500x500 image and set rs:fill:300:300 - it will resize both images. What i want is the 100x100 image to become 300x300, but the large 500x500 image should keep its size.

Ah I see, the documentation on imgix was confusing then. This is what you meant right?
Large image remaining the same.

Hey everyone!
min-width & min-height were added to v3.0.0.beta1.
You can try the OSS version of v3.0.0.beta1, or you can subscribe to our imgproxy Pro v3 Early Access program


[stable/prometheus-operator] [stable/granfana] grafana sidecar SSL: CERTIFICATE_VERIFY_FAILED

This seems the same issue as  https://github.com/helm/charts/issues/9118 which was closed after go stale without any reference to fixes.  I don't know how to reopen it.  This is a showstopper for grafana under prometheus or used with kubernetes, because the grafana is not working with the sidecar failures like this, thus we should not close this issue unless it is fixed.
I have hit a similar error using stable/prometheus-operator 1.8.0, which uses subchart stable/grafana 1.24.2.  This issue did not exist in coreos/prometheus-operator + coreos/kube-prometheus.
Apparently we need an option or instruction to supply the certificate to verify the ssl certificate of the h ttps server, the kubernetes apiserver to be exact, or turning off the verification.
A related issue on the upstream grafana sidecar image 0.0.6 can be found here: https://github.com/kiwigrid/k8s-sidecar/issues/18
Here is the log produced by one of the sidecar containers:
  Starting config map collector
Config for cluster api loaded...
2019-02-08 19:01:25,019 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:841)'),)': /api/v1/namespaces/monitoring/configmaps?watch=True
2019-02-08 19:01:25,026 WARNING Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:841)'),)': /api/v1/namespaces/monitoring/configmaps?watch=True
2019-02-08 19:01:25,032 WARNING Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:841)'),)': /api/v1/namespaces/monitoring/configmaps?watch=True
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 849, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 356, in connect
    ssl_context=context)
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 372, in ssl_wrap_socket
    return context.wrap_socket(sock)
  File "/usr/local/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/usr/local/lib/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
    for event in stream:
  File "/usr/local/lib/python3.6/site-packages/kubernetes/watch/watch.py", line 122, in stream
    resp = func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/apis/core_v1_api.py", line 11650, in list_namespaced_config_map
    (data) = self.list_namespaced_config_map_with_http_info(namespace, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/apis/core_v1_api.py", line 11753, in list_namespaced_config_map_with_http_info
    collection_formats=collection_formats)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 231, in GET
    query_params=query_params)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 205, in request
    headers=headers)
  File "/usr/local/lib/python3.6/site-packages/urllib3/request.py", line 68, in request
    **urlopen_kw)
  File "/usr/local/lib/python3.6/site-packages/urllib3/request.py", line 89, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/usr/local/lib/python3.6/site-packages/urllib3/poolmanager.py", line 322, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 667, in urlopen
    **response_kw)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 667, in urlopen
    **response_kw)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 667, in urlopen
    **response_kw)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 398, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='<IP_ADDRESS>', port=443): Max retries exceeded with url: /api/v1/namespaces/monitoring/configmaps?watch=True (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:841)'),))

We have a fix on the grafana sidecar https://github.com/kiwigrid/k8s-sidecar/pull/19
Would you please

update the grafana subchart to use the new sidecar version kiwigrid/k8s-sidecar:0.0.11 and add the new configurable env SKIP_TLS_VERIFY to both the values.yaml and the deployment.yaml

update the prometneus-operator chart to use the new grafana subchart

Hey? ISSUE IS NOT CLOSED ! It still persists , give us ability to pass environment variables to that image


status line show builds running?

we should have a function for this.

I've made a PR https://github.com/benekastah/neomake/pull/387 for this, not really hard.

Closing in favor of #915.


Week number not working

according to http://strftime.org when I use %U or %W the week number should be printed but instead some of the currently present output is removed. For example:
before: "%F %H:%M" -> 2018-05-04 16:35
after: "%F %W %H:%M" -> pią 16:35
Ubuntu 18.04

You’re looking at the wrong documentation. It should be:
https://developer.gnome.org/glib/stable/glib-GDateTime.html#g-date-time-format

Agreed with @da2x; thanks to his pull request we're now pointing at the correct documentation.  I think that the best you can do for week number is %V, but see the docs above.

@da2x I'm not using Gnome, so I was a bit loath to merge a thing that I haven't myself tested. (Docs updates are OK, though.) I'd be happy for someone (you?) to take over this extension...

Sure. I showed interest almost a year ago and I’m still here, so I guess that makes me qualified. 😛
Please give my account co-ownership of the extension and repository. My username on GNOME Extensions is .ext and my username on GitHub is da2x. You’ll find my email address used with both services in this repository’s git log, if you need it. In case you can’t transfer the extension to a new owner, would you consider transferring your account to me? I’d like to avoid forking the existing extension entry to maintain auto-update capabilities with the existing user base. There are too many forks already.
If at all possible, I’d also really appreciate it if you would write down your release and release testing steps in the repository wiki; and document and transfer any other knowledge I should have.

@da2x the extension doesn't seem to have been transferred to you on extensions.gnome.org; I'm waiting on that.

@stuartlangridge last step is to either go to https://github.com/stuartlangridge/gnome-shell-clock-override/settings and either transfer ownership to me or add me as a admin collaborator from https://github.com/stuartlangridge/gnome-shell-clock-override/settings/collaboration

@stuartlangridge I think I must have ran into a caching issue; couldn’t push before but it works fine now. Thanks!


Add compatibility matrix to README file

Similar to:
https://github.com/kubernetes/client-go#compatibility-matrix
Add a compatibility matrix in README file to define which kubernetes version is supported by which python client.

fixed in #178


Hot-Module-Replacement (HMR) NOT working with vue-cli-service build --watch

Version
3.1.3
Reproduction link
https://bitbucket.org/maxx/hrm-vue-cli-watch-bug
Node and OS info
Node 8.12.0 / Vagrant / Ubuntu 16.04 / VirtualBox / Windows 10
Steps to reproduce

Create project from https://cli.vuejs.org/guide/creating-a-project.html#vue-create
go to /app/test/hello-world
npm install
npm vue-cli-service build --watch --mode development OR sudo npm run watch
add any string in /src/components/HelloWorld.vue

What is expected?
rebuild dist folder at once any file in project changing
What is actually happening?
Dist folder didn't rebuilding when file is changed

Tried https://github.com/vuejs/vue-cli/issues/2051 - fail.
But vue-cli-service serve works fine and rebuildat once file changing

Does this help: https://github.com/webpack/webpack/issues/6262?

No. it doesn't

for me HMR for vue files working only for this app mounting method
window.appVue = new Vue({   router,   store,   render: (h) => h(App), }).$mount('#app');
and not working for this
window.appVue = new Vue({   el: '#app',   router,   store,   components: {App},   template: '<App/>' })

Both  doesn't work for me

Closing since not reproducible.

maybe this can help someone: https://webpack.js.org/configuration/watch/#changes-seen-but-not-processed


HLS-Code

I cannot make sense of this pragma:
#define CASSERT_DATAFLOW(x) ;

I searched the internet and did not find an answer to what this does exactly.
Thank you for your help.
Kind regards

Hi,
It is a no-op, it doesn't do anything. It used to be an assert for debugging, but now it expands to ignore any argument to it.
Ken

Thank you.