RDKDEV-112: Add support to allow a full DVB stack to be used in RDKV

Reason for change: Add DTV plugin
Test procedure: View a DVB service on a hybrid device.
Risks: None
Signed-off by: Steve Ford<EMAIL_ADDRESS>Change-Id: I3525c48f32f6039661caedbb02553e6b6abb3dcd

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

https://jenkins.code.rdkcentral.com/job/github-triggered/1934/ started

Copyright scan failure
Commit: 4b33278d53a5d884bcea0791ce350e7c4ca78c1d
Report detail: https://gist.github.com/e940ca996ede33ebb8e4da73d909fdc6

Hi Steve,
Copyright scan is complaining here because ThunderNanoServices is Apache licensed and our headers are RDK proprietary.  You need RDK Apache headers please.
Martin

https://jenkins.code.rdkcentral.com/job/github-triggered/1936/ started

Thanks, Martin :+1:

https://jenkins.code.rdkcentral.com/job/github-triggered/2008/ started


SMTP error when remove all messages with storage maildir

Hi,
First of all thank you for this great application!
I plan to integrate it on Neard but there is a problem during the removal of all messages with the parameter -storage maildir.
When i remove all messages via the WebUI the maildir is completely removed and next i've got an error when i want to send messages :
SMTP server response: 452 Unable to store message

Would be better to only remove the content of the folder and not the folder himself.
Thanks!

Btw it does not happen all the time and i'm on Windows 10 64bits

Cannot reproduce since i posted this issue


Add support for SQL UNION Operator

Describe the feature

Gorm currently has no support for SQL UNION Operator https://www.w3schools.com/sql/sql_union.asp
Motivation

This makes it possible to combine results from multiple tables and apply conditions/ordering them. More examples can be found on w3schools link above.
Related Issues
There are no related issues.

Hello @moneszarrugh
The following code is preferred for UNION support
DB.Raw("? UNION ?", DB.Select("name").Model(&user), DB.Select("age").Model(&user)).Scan(&newData)

Since Order() is ignored for raw queries, it seems that sorted unions are impossible for now.

The following workaround allows ordering:
.Table("(?) AS union_table", subQuery1).
Joins("UNION (?)", subQuery2)


What happens when new claim is added as draft arg reason

Handle first server side (will write client side code but that's not to be trusted!
Check the differences between the db version of the claim and the request version.
If there are any new 'reasons' / args then check if they exist - first use provided ID (there should be one) then use the description (if there isn't an ID).
If no matches are found, create a new draft and save it (also save it to the user's profile).  Once that returns (for every instance!), update / save the original draft and (if it's in the new route - add it to the user's profile too)

OR - when the user adds a new 'reason' client side - it's highlighted red until they confirm they want to save it - then it'll run through it's own save new process and return a full object that won't cause any crazyness in the update / new controller for the current draft  - good idea Iain, good idea!


Add basic validator.

This is the first version. For the moment, I prototyped using
datasets_format/datasets/titanic/metadata.json.
Design elements:

The entry is the script datasets_format/python/format/scripts/validate.py.
README in datasets_format/python/format/README.md.
Dataset represents a graph from a file.
In Dataset.__init__, we call the validator.
The validator builds an RDFLib graph, then converts it to a NetworkX
graph, then checks this graph.
At the moment, we only check that each subject has the proper
properties.
I put in place some unit tests, particularly in datasets_format/python/format/src/graphs_test.py.

Questions:

What should I expect on http://mlcommons.org/schema/dataType? Just that the dataType is a valid URI? Should we fetch the URI?

Todo:

Check the ID, and run a BFS to check that the tree is valid.
Launch tests in CI/CD.
Support more complex use cases (e.g., ml:RecordSet -> data in Titanic), and other datasets.
Make OpenML converter compatible (for instance, at the moment, I have the following errors):

format.src.errors.ValidationError: Found the following 6 error(s) during the validation:
  -  [dataset(Titanic)] Property "https://schema.org/citation" is mandatory, but does not exist. Existing properties: {rdflib.term.URIRef('https://schema.org/version'), rdflib.term.URIRef('https://schema.org/keywords'), rdflib.term.URIRef('https://schema.org/distribution'), rdflib.term.URIRef('https://schema.org/dateModified'), rdflib.term.URIRef('https://schema.org/description'), rdflib.term.URIRef('https://schema.org/url'), rdflib.term.URIRef('https://schema.org/creativeWorkStatus'), rdflib.term.URIRef('http://www.w3.org/1999/02/22-rdf-syntax-ns#type'), rdflib.term.URIRef('https://schema.org/@language'), rdflib.term.URIRef('https://schema.org/recordSet'), rdflib.term.URIRef('https://schema.org/isAccessibleForFree'), rdflib.term.URIRef('https://schema.org/name'), rdflib.term.URIRef('https://schema.org/dateCreated')}.
  -  [dataset(Titanic)] Property "https://schema.org/license" is mandatory, but does not exist. Existing properties: {rdflib.term.URIRef('https://schema.org/version'), rdflib.term.URIRef('https://schema.org/keywords'), rdflib.term.URIRef('https://schema.org/distribution'), rdflib.term.URIRef('https://schema.org/dateModified'), rdflib.term.URIRef('https://schema.org/description'), rdflib.term.URIRef('https://schema.org/url'), rdflib.term.URIRef('https://schema.org/creativeWorkStatus'), rdflib.term.URIRef('http://www.w3.org/1999/02/22-rdf-syntax-ns#type'), rdflib.term.URIRef('https://schema.org/@language'), rdflib.term.URIRef('https://schema.org/recordSet'), rdflib.term.URIRef('https://schema.org/isAccessibleForFree'), rdflib.term.URIRef('https://schema.org/name'), rdflib.term.URIRef('https://schema.org/dateCreated')}.
  -  [dataset(Titanic) > distribution(Titanic (arff))] Property "https://schema.org/contentSize" is mandatory, but does not exist. Existing properties: {rdflib.term.URIRef('https://schema.org/name'), rdflib.term.URIRef('https://schema.org/encodingFormat'), rdflib.term.URIRef('https://schema.org/contentUrl'), rdflib.term.URIRef('http://www.w3.org/1999/02/22-rdf-syntax-ns#type')}.
  -  [dataset(Titanic) > distribution(Titanic (arff))] At least one of these properties should be defined: [rdflib.term.URIRef('https://schema.org/sha256'), rdflib.term.URIRef('https://schema.org/md5')]. Existing properties: {rdflib.term.URIRef('https://schema.org/name'), rdflib.term.URIRef('https://schema.org/encodingFormat'), rdflib.term.URIRef('https://schema.org/contentUrl'), rdflib.term.URIRef('http://www.w3.org/1999/02/22-rdf-syntax-ns#type')}.
  -  [dataset(Titanic) > distribution(Titanic (parquet))] Property "https://schema.org/contentSize" is mandatory, but does not exist. Existing properties: {rdflib.term.URIRef('https://schema.org/name'), rdflib.term.URIRef('https://schema.org/encodingFormat'), rdflib.term.URIRef('https://schema.org/contentUrl'), rdflib.term.URIRef('http://www.w3.org/1999/02/22-rdf-syntax-ns#type')}.
  -  [dataset(Titanic) > distribution(Titanic (parquet))] At least one of these properties should be defined: [rdflib.term.URIRef('https://schema.org/sha256'), rdflib.term.URIRef('https://schema.org/md5')]. Existing properties: {rdflib.term.URIRef('https://schema.org/name'), rdflib.term.URIRef('https://schema.org/encodingFormat'), rdflib.term.URIRef('https://schema.org/contentUrl'), rdflib.term.URIRef('http://www.w3.org/1999/02/22-rdf-syntax-ns#type')}.

@benjelloun, thank you for the review!
I also addressed your question on reducing the number of test files. I think we have to keep a minimum of "end-to-end tests" with real data, but I added more unit tests to support other cases.

@pierrot0 Can you please review? Notably, I would like your input on the project's structure:
.
|-- python
|   |-- format
|   |   |-- __init__.py
|   |   |-- pyproject.toml
|   |   |-- README.md
|   |   |-- scripts
|   |   |   `-- validate.py
|   |   `-- src
|   |       |-- constants.py
|   |       |-- datasets.py
|   |       |-- datasets_test.py
|   |       |-- errors.py
|   |       |-- errors_test.py
|   |       |-- graphs.py
|   |       |-- graphs_test.py
|   |       |-- __init__.py
|   |       `-- tests
|   |           `-- graphs
|   |               ... *.json files for tests
|   `-- openmlconverter
|       ...
`-- README.md

I think:

We should drop the src folder.
We should have one datasets_format library instead of sub-folders (python/format).

I think:

We should drop the src folder.

What would be the gain of dropping the src folder?
It feels to me that it might be beneficiary to have a similar structure for all sub-projects (openmlconverter, ...)?

We should have one datasets_format library instead of sub-folders (python/format).

Do you mean that we may want to have multiple github repos?
Or something else?


feat: Add topic load-test

PR Description:
Add topic load-test
Notes

The test takes in two environment variables:

MOMENTO_AUTH_TOKEN: Token required to perform operations on a cache in a region
BROWSER_CONFIG='{"numberOfBrowserInstances": <number>, "numSubscriptions": <number>, "publishRatePerSecond": <number>}' where numberOfBrowserInstances in the number of browser instances that are required for this load-test, numSubscriptions are the number of subscriptions created by each browser, publishRatePerSecond is the number of publish requests to be sent in one second by each browser.

Currently, max number of publish requests is set to 100 which can be changed after calculating the numbers this load-test is required to run in order to complete the experiment

The program runs in a fargate container and finally uploads 3 files to s3:

Time taken by the publish hanler to publish the request
Time between when the request was published and  received
Context on successful/failed subscriptions/publishes

COMMAND used to deploy to fargate through terminal:
AWS_PROFILE=dev aws ecs run-task --task-definition topics-loadgen-test --cluster topics-loadgen-test-cluster --network-configuration "awsvpcConfiguration={subnets=[subnet-086a8ea0811f6ec8b],securityGroups=[sg-09e57fa7302e3620a],assignPublicIp=ENABLED}" --launch-type FARGATE --count 2

The program exits gracefully after 1 min (can be changed based on the calculations).

Testing
MOMENTO_AUTH_TOKEN=<token> npm run integration-test-subscribe

@rishtigupta I believe this one is superceded by https://github.com/momentohq/demos/pull/8 and we can close this PR, right?

yes, closing it


GEM_PATH broken when calling $(chef shell-init bash) on chefdk 0.13.21

Hello,
I encounter an issue when I call eval "$(chef shell-init bash)" with chefdk 0.13.21 on MacOSX 10.10.5 and GNU bash, version 4.3.42(1)-release (x86_64-apple-darwin14.5.0).
The GEM_PATH global variable seems broken :
export GEM_PATH="/Users/v.demonchy/.chefdk/gem/ruby/2.1.0:/opt/chefdk/embedded/lib/ruby/gems/2.1.0" _chef_comp local COMMANDS="exec env gem generate shell-init install update push push-archive show-policy diff provision export clean-policy-revisions clean-policy-cookbooks delete-policy-grou complete -F _chef_comp chefOMMANDS" -- ${COMP_WORDS[COMP_CWORD]} ))
Obviously, trying to eval that results in an error (syntax error near unexpected token...)
It was fine with chefdk 0.12.0 (previous stable release) :
export GEM_PATH="/Users/v.demonchy/.chefdk/gem/ruby/2.1.0:/opt/chefdk/embedded/lib/ruby/gems/2.1.0" _chef_comp() { local COMMANDS="exec env gem generate shell-init install update push push-archive show-policy diff provision export clean-policy-revisions clean-policy-cookbooks delete-policy-group delete-policy undelete verify" COMPREPLY=($(compgen -W "$COMMANDS" -- ${COMP_WORDS[COMP_CWORD]} )) } complete -F _chef_comp chef

👍

Is this still broken for you as of 0.15.15?

I had this problem with 0.15.15 until I deleted my .chefdk directory and had it recreated on OS X.

Looks like its resolved for me on Mac OS 10.11.5 and Chef Development Kit Version: 0.15.15

closing. please reopen if it is still occuring.

For the record, I was seeing this error with Max OS X 10.13 (High Sierra) and ChefDK 1.6.1.
$ eval $(chef shell-init bash)
-bash: syntax error near unexpected token `('

Surrounding the eval arg with quotes made the error go away:
$ eval "$(chef shell-init bash)"
$ echo $?
0


[stable/prometheus-adapter] unable to add custom rules using --set-file

Describe the bug
helm install my-release stable/prometheus-adapter --set-file rules.custom=customrules.yaml doesnt work
an extraneous "|" (pipe symbol) is added to the start of the new rules in the configmap resource and this causes the (prometheus adapter) pod to NOT start up.
Version of Helm and Kubernetes:
Helm Version:"v3.0.2"
Kubernetes Version: "1.14.9" (on EKS - Platform:"linux/amd64")
Which chart:
[stable/prometheus-adapter] version 2.0.1
What happened:
an extraneous "|" (pipe symbol) is added to the start of the new rules in the configmap resource and this causes the (prometheus adapter) pod to NOT start up.
What you expected to happen:
the extraneous "|" should not be added in the configmap resource and the prometheus adapter pod should startup.
How to reproduce it (as minimally and precisely as possible):
- seriesQuery: "mytest_tasks_per_minute"
  seriesFilters: []
  resources:
    overrides:
      kubernetes_namespace:
        resource: namespace
      kubernetes_pod_name:
        resource: pod
  name:
    matches: "mytest_tasks_per_minute"
    as: ""
  metricsQuery: <<.Series>>{<<.LabelMatchers>>,container_name!="POD"}

store the above rule in a customrules.yaml and install the helm chart as:
helm install my-release stable/prometheus-adapter --set-file rules.custom=customrules.yaml
Anything else we need to know:
changing the templates/custom-metrics-configmap.yaml file to the below works for me
{{- if .Values.rules.custom }}
{{ tpl .Values.rules.custom . | indent 4 }}
{{- end -}}

I assume the same problem would occur with the rules.external and rules.resource parameters as well!

the problem is still present for extraScrapeConfigs


Convert log

Hi,
I'm using hapi-pino for logging but I don't know how I can map the field contain IP with location to new field (or overwrite IP field).
In other words, I want to customize log and maybe use middleware (for mapping IP to locaiton) on process,
Current
{
   "ip": "<IP_ADDRESS>"
}

What I want
{
   "ip": "<IP_ADDRESS>",  // May be removed
   "location": "USA"
}

Thanks

See https://getpino.io/#/docs/api?id=formatters-object


Feat: Add procedure to find the next active player

@Created-for-a-purpose where are the test for check procedure ?
On a very highlevel in the test we should evaluate

consecutive check counter is increased
we found the next active player and updated the turn


Use thread registry from the consensus layer in Subscription/Common.hs

Need to decide if we want to do this or not. This will be significantly affected by whether we choose to use the p2p governor for the first shelley release.

Subscription/Common.hs is part of the old non-p2p code and won't be updated.


Event Grid Sample issues

Section Link:

Reason:
The number of parameters does not match.
Suggestion:
Delete the parameter 1
@jongio for notification.

This PR updated both the EventGridEvent ctor and the readme sample code to take 4 params.  https://github.com/Azure/azure-sdk-for-java/commit/d43a520cd41044a2518ede6a51e65e6a2f66747b.  Closing this issue as I don't believe the inconsistency exists in the latest build.
Though looking at the EventGridEvent public ctor in C# and Python shows that the parameter ordering is different across the different SDK languages (/cc @ellismg).  I'll follow up on that offline.


Fix updating of the last_modified value

The last_modified value was in fact not updated when modifying a prefix, in most cases at least. This was due to conditions on the trigger that performs the update making it run only when the vrf_id, type or prefix was changed. The conditions themselves are there to increase performance by avoiding to run a large trigger function when it's not necessary.
By moving the conditional part from the trigger definition into the trigger function, the last_modified value can always be updated but the lions share of the function avoided when it is not needed.

:thumbsup:


Sync up README and doc/apn.markdown

Small docs fix

Coverage remained the same at 100.0% when pulling 76c3b10124e55d58a875779e3f62ca93fb855ae6 on schottkey7:master into f7456dfe249905ce11a27b0217991d9cc36929a1 on node-apn:master.


[PER-16]: When todo status changes, we now properly update updated_at field

#4  👈
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @rafaelrene and the rest of your teammates on  Graphite


UI: A11y: chromecast dialog is not accessible

This should be re-tested.  It may have changed in Chrome.  If not, this should be a Chrome bug, since we don't control the Chromecast dialog.

Can't reproduce on Chrome 71.


test: 100% Coverage pkg/runner/logging

Signed-off-by: Shubham Gupta<EMAIL_ADDRESS>

Awesome thanks !


Enabling PWM support for STM32 F207ZG Platform

Enabled PWM support for STM32 F207ZG target platform.
Verified PWM signals on timer1 using test application tests/drivers/pwm

@sarthsmart Thanks for this. Please fix warnings reported by Compliance checks

@sarthsmart Thanks for this. Please fix warnings reported by Compliance checks

Fixed warnings by following https://docs.zephyrproject.org/latest/contribute/#coding-style


Unable to Connect to Twitch

Looking over DEBUG, I never see it send the extra option "PASS":
17 Apr 13:43:03 - Sending irc NICK/USER
17 Apr 13:43:03 - SEND: NICK ircusername
17 Apr 13:43:04 - SEND: USER ircusername 8 * ircusername
17 Apr 13:43:13 - Connection got "end" event
17 Apr 13:43:13 - Connection got "close" event
17 Apr 13:43:13 - Disconnected: reconnecting
17 Apr 13:43:13 - Waiting 10000ms before retrying
CyclingPingTimer 0: server silent for too long, let's send a PING
17 Apr 13:43:19 - SEND: PING 1
17 Apr 13:43:19 - Network error: Error: This socket has been ended by the other party
17 Apr 13:43:23 - Sending irc NICK/USER
17 Apr 13:43:24 - SEND: NICK ircusername
17 Apr 13:43:24 - SEND: USER ircusername 8 * ircusername
I am specifying the option in the config like so:
[
{
"nickname": "ircusername",
"server": "irc.twitch.tv",
"discordEmail": "email@address",
"discordPassword": "password",
"autoSendCommands": [ // Commands that will be sent on connect
["PASS", "oauth:password"]
],
"channelMapping": { // Maps each Discord-channel to an IRC-channel, used to$
"#general": "#ircchannel" // Add channel keys after the chan$
},
"ircOptions": { // Optional node-irc options
"debug": "true",
"showErrors": "true",
"retryDelay": "10000"
}
}
]
I am able to connect fine over telnet, however, you must send USER, PASS, then NICK:
[user@host ~]$ telnet irc.twitch.tv 6667
Trying <IP_ADDRESS>...
Connected to irc.twitch.tv.
Escape character is '^]'.
USER ircusername 8 * ircusername
PASS oauth:password
NICK ircusername
:tmi.twitch.tv 001 ircusername :Welcome, GLHF!
:tmi.twitch.tv 002 ircusername :Your host is tmi.twitch.tv
:tmi.twitch.tv 003 ircusername :This server is rather new
:tmi.twitch.tv 004 ircusername :-
:tmi.twitch.tv 375 ircusername :-
:tmi.twitch.tv 372 ircusername :You are in a maze of twisty passages, all alike.
:tmi.twitch.tv 376 ircusername :>
How can this connection be accomplished using this library? Thanks in advance.

https://github.com/reactiflux/discord-irc/issues/42


Implement dependency injection for Dialogs in Enterprise Bot Template

Description
There has been a lot of talk and wishes for improvement around dependency injection of Dialogs.

https://github.com/Microsoft/botbuilder-dotnet/issues/860
https://github.com/Microsoft/botbuilder-dotnet/issues/357
https://github.com/Microsoft/botbuilder-dotnet/issues/1188
https://stackoverflow.com/questions/52549850/how-do-you-do-dependency-injection-in-a-class-that-inherits-dialogcontainer-bot

I believe Enterprise Bot Template should set a good example. I'm trying to do that with this PR without changes to the bot framework.
Architecture
This PR builds the existing ASP.Net Core DI code and follows the approach from https://github.com/Microsoft/BotBuilder-Samples/blob/master/samples/csharp_dotnetcore/42.scaleout/Startup.cs

Dialogs are registered in a separate method RegisterDialogs in Startup.cs
The MainDialog is injected in Bot
Redundant constructor parameters are removed:

Some dialogs do not need UserState or ConversationState anymore
UserState and ConversationState can be injected directly because they are registered
OnboardingDialog generate its own state accessor based on the injected UserState

Testing Steps
Run the bot as before
Checklist

[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have added or updated the appropriate unit tests
[ ] I have updated related documentation

[ ] A duplicate issue is filed to track future  work.

Thanks! We totally agree and actually have this almost ready for our Build release, would love your feedback on whether it hits the Mark - it's a big step forward we feel. It's all in the 4.4 branch and we should have a new vsix today/tomorrow if you'd like to test?
At the same time you'll see the new skill work!

@darrenj
That's really nice to hear, and I'm curious to see the results of that!
At the same time I would really appreciate it then, if you could maintain a kind of roadmap with a clear indication of what is being worked on and where that can be found.
That would allow people to collaborate if possible, and it would prevent people from spending effort and time on PRs that are redundant...
E.g. @johnataylor briefly mentioned that there would be worked on dependency injection but did not give any details and did not reply to questions anymore. This made it appear to be a longer-term thing that might even not be released, so I came up with this PR based on the mentioned sample.
However if I had known that you are concretely working on an implementation that is more thorough, maybe I would not have made this PR.

Closing since this logic is being handled in the 4.4 branch.

@lauren-mills @darrenj if I look now in the 4.4 branch for the Enterprise Bot Template I do not see any dependency injection of dialogs in the MainDialog for example:
https://github.com/Microsoft/AI/blob/4.4/templates/Enterprise-Template/src/csharp/EnterpriseBotTemplate/EnterpriseTemplate/Dialogs/Main/MainDialog.cs
Am I looking in the wrong place?

Virtual Assistant template replaces the Enterprise Template: https://github.com/Microsoft/AI/tree/4.4/templates/Virtual-Assistant-Template

OK, thank you for the clarification!


Create Overview of main points of DROID Ops Tools at RevelTek in the form of .md file

Completed.
Thank you


Creating a project as a cluster member throws forbidden error but the project is still created

Rancher Server Setup

Rancher version: v2.6-head 3506e43
Installation option (Docker install/Helm Chart): Docker install

If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc): k3s

Proxy/Cert Details:

Information about the Cluster

Kubernetes version: v1.24.2+k3s1
Cluster Type (Local/Downstream): Downstream Infrastructure provider 3 nodes all roles.

If downstream, what type of cluster? (Custom/Imported or specify provider for Hosted/Infrastructure Provider):

User Information

What is the role of the user logged in? (Admin/Cluster Owner/Cluster Member/Project Owner/Project Member/Custom)

If custom, define the set of permissions: Cluster Member

Describe the bug
Project creation errors with forbidden message for cluster members. But the project is still created.
To Reproduce

Create a downstream cluster with the spec mentioned above.
On rancher server 2.6-head, create a standard user.
Add the standard user in the downstream cluster as a cluster member.
Login as a cluster member, from projects/namespaces create a project - P1.

Result
Project creation returns an error
"projects.management.cattle.io \"p-rf8jj\" is forbidden: User \"u-2cdmn\" cannot get resource \"projects\" in API group \"management.cattle.io\" in the namespace \"c-5s6dj\""

Some errors from the rancher logs seen when a new project is created as a cluster member.
2022/07/14 22:26:14 [INFO] [mgmt-project-rbac-create] Creating creator projectRoleTemplateBinding for user u-2cdmn for project p-rf8jj
2022/07/14 22:26:14 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=p-rf8jj, err=Operation cannot be fulfilled on namespaces "p-rf8jj": the object has been modified; please apply your changes to the latest version and try again

Expected Result
Expected the project creation to work without any errors.
Additional context
When we try to create a project from v3 API https://<rancherIP>/v3/clusters/<clusterID>/projects , no errors are seen and the project creation goes through without any error.
Even though the project creation from UI throws error, the project is created without any errors.
Not seen on v2.6.5.
Creating a project as a cluster owner goes through without any error.

I tried to reproduce this on v2-6 head https://github.com/rancher/dashboard/issues/6464 but it looks like it works. I created a cluster member, added them to a downstream cluster, logged in as the cluster member, and created projects successfully.
While doing that, I did run into this other error https://github.com/rancher/dashboard/issues/6377. There is a PR for that that is in review.

Verified on v2.6-head b66dd0f
Issue is not seen anymore. Creating projects a cluster member and no errors seen. Closing the issue and will re open if seen in future

Still seeing the issue on v2.6-head 78017e6
Creating a project as a cluster member results in the error "projects.management.cattle.io \"p-rf8jj\" is forbidden: User \"u-2cdmn\" cannot get resource \"projects\" in API group \"management.cattle.io\" in the namespace \"c-5s6dj\"" but the project is still created

Moving to the Review column because incoming changes aren't merged yet.

Since we're in a weird Zenhub/Zube hybrid state, moved both states to To Test.

Verified on v2.6-head 2f85f56
Creating projects a cluster member works and no errors seen.

I am running into this issue on v2.6.8 (single-node Rancher instance in Docker).

Hey @maxsokolovsky are you seeing this issue on rke1 or rke2 downstream clusters? we recently had a fix for rke2 downstream node driver clusters here .

@anupama2501, I am seeing this on RKE1 downstream.

I am also able to reproduce this in Rancher 2.6.8 with an RKE1 cluster

Reopened the issue to try to repro with RKE1

@catherineluse to validate if this is still happening with the latest.

I'm able to reproduce the issue on rancher v2.7-head 8062f1e

Install rancher server on v2.7-head
Create an rke1 downstream node driver - 1 node all roles
Enabled openldap auth and added a standard user from the auth to the downstream as a cluster member.
Logged in as a standard user - cluster member from the openldap auth
Created a project and it immediately errors out with

projects.management.cattle.io "p-lldfj" is forbidden: User "u-x5skpj2fjg" cannot get resource "projects" in API group "management.cattle.io" in the namespace "c-bw8wz"

Navigate back to projects/namespaces page and see the project created

The following POST request fails:
xhr.js:178          POST https://<redacted>/v3/projects/c-bw8wz:p-lldfj?action=setpodsecuritypolicytemplate 403 (Forbidden)
xhr.js:178          POST https://<redacted>/v3/projects/c-bw8wz:p-lldfj?action=setpodsecuritypolicytemplate 403 (Forbidden)

✅ PASSED
Reproduction Environment

Component
Version / Type

Rancher version
v2.7.1

Installation option
Helm (high availability)

If Helm Chart k8s cluster
v2.16.8-rancher2

Cert Details
Let's Encrypt / traefik

k3s ha external db
Aurora MySQL 5.7.mysql_aurora.2.11.1

Helm version
v1.24.7+k3s1

Downstream cluster type
RKE1 Linode

Downstream K8s version
v1.24.9

Authentication providers enabled
OpenLDAP

Logged in user role
admin & standard user

Browser type
Google Chrome

Browser version
v2.16.8-rancher2

Bug doesn't happen 100% of the time. See the chart below:

Frequency
Selected Option

Consistent (10/10) - A consistent bug can be replicated every time the test is executed.

Frequent (7/10)- A frequent bug can be replicated like 7 out of 10 times when a test is replicated
✅

Occasional (5/10) - Occasional bug is more hard to replicate than a frequent bug.

Once thus far, twice thus far - Few bugs cannot be reproduced every time you run the test.

🚨 Additional Reproduction Setup Details: Click to Expand

K3s install command for node one. Replace values in <>

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=<K3S_VERSION> sh -s - server --token=SECRET --datastore-endpoint=<DB_ENDPOINT> --tls-san <TLD> --node-external-ip <IP>

K3s install command for node two. Replace values in <>. Main difference here is the --token

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=<K3S_VERSION> sh -s - server --token=<TOKEN_FROM_NODE_ONE> --datastore-endpoint=<DB_ENDPOINT> --tls-san <TLD> --node-external-ip <IP>

Reproduction steps

Setup Rancher 2.7.1
Enable OpenLDAP
As admin create a downstream RKE1 Linode cluster
Assign an OpenLDAP user as a cluster member on the downstream
Log out of Rancher as admin ===> login as the standard user assigned to cluster member
As the cluster member try creating a project in the downstream cluster
Intermittently getting the following error message when creating a project:

Also see the attached screenshot

cannot get resource "projects" in API group "management.cattle.io" in the namespace

Additional Info
RESULTS
✅ Expected
For a cluster member to be able to create projects without intermittently getting an error message
❌ Actual
Cluster member is intermittently getting an error message when creating projects
Screenshots

Validation Environment

Component
Version / Type

Rancher version
v2.7-c24fb8b0869a0b445f55b3307c6ed4582e147747-head

Rancher commit link
https://github.com/rancher/rancher/commit/c24fb8b0869a0b445f55b3307c6ed4582e147747

Installation option
Helm (high availability)

If Helm Chart k8s cluster
v1.24.7+k3s1

Cert Details
Let's Encrypt / traefik

k3s ha external db
Aurora MySQL 5.7.mysql_aurora.2.10.2

Helm version
v2.16.8-rancher2

Downstream cluster type
RKE1 Linode

Downstream K8s version
v1.25.6

Authentication providers enabled
OpenLDAP

Logged in user role
admin & standard user

Browser type
Google Chrome

Browser version
(Official Build) (x86_64)

🚨 Additional Reproduction Setup Details: Click to Expand

K3s install command for node one. Replace values in <>

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=<K3S_VERSION> sh -s - server --token=SECRET --datastore-endpoint=<DB_ENDPOINT> --tls-san <TLD> --node-external-ip <IP>

K3s install command for node two. Replace values in <>. Main difference here is the --token

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=<K3S_VERSION> sh -s - server --token=<TOKEN_FROM_NODE_ONE> --datastore-endpoint=<DB_ENDPOINT> --tls-san <TLD> --node-external-ip <IP>

Validation steps

Setup Rancher 2.7-head
Enable OpenLDAP
As admin create a downstream RKE1 Linode cluster
Assign an OpenLDAP user as a cluster member on the downstream
Log out of Rancher as admin ===> login as the standard user assigned to cluster member
As the cluster member try creating a project in the downstream cluster
Intermittently getting the following error message when creating a project:

Also see the attached screenshot

Additional Info
RESULTS
✅ Expected
No error messages when cluster member is creating projects
✅ Actual
No error messages when cluster member is creating projects
📋 QA Test Plan

Scenario
Steps
Expected
Actual & Pass/Fail

Start with 2.7.1 upgrade to 2.7-head

Start with Rancher 2.7.1
See the bug by following repo/valid steps
Upgrade Rancher to 2.7-head with Helm command
Follow the repo/valid steps again
Verify the bug is no longer presenting itself

Expected: No error messages when cluster member is creating projects
Actual: No error messages when cluster member is creating projectsPass/Fail: ✅PASS FAIL

Start with 2.7.1 upgrade to 2.7-head then create 1,100 users & role bindings

Start with Rancher 2.7.1
See the bug by following repo/valid steps
Upgrade Rancher to 2.7-head with Helm command
Follow the repo/valid steps again
Verify the bug is no longer presenting itself
Create 1,100 users and role bindings using the Terraform below
https://github.com/brudnak/rancher-tf-users
Verify there is no error in the UI when creating projects after creating 1,100 users and role bindings

Expected: No error messages when cluster member is creating projects
Actual: No error messages when cluster member is creating projectsPass/Fail: ✅PASS FAIL

/backport v2.6-next1


Add rsconnect json files and folium tests

Add tests for render.display for shiny express apps.
Also, enable deploy tests to shinyapps.io in addition to connect

@wch The deploy of express apps to Connect fails to start with the logs in Connect stating the following
2023/12/31 12:40:58 PM: Loading code from "shiny.express.app:app_2e_py"
2023/12/31 12:40:59 PM: Detected Shiny Express app. please note that Shiny Express is still in development and the API is subject to change!
2023/12/31 12:40:59 PM: Unexpected error while running Python API: cannot import name 'ui' from 'shiny.express' (/opt/rstudio-connect/mnt/app/python/env/lib/python3.10/site-packages/shiny/express/__init__.py)
2023/12/31 12:40:59 PM: Traceback (most recent call last):
2023/12/31 12:40:59 PM:   File "/opt/rstudio-connect/python/connect_fastapi_runtime.py", line 489, in
2023/12/31 12:40:59 PM:     main(sys.argv)
2023/12/31 12:40:59 PM:   File "/opt/rstudio-connect/python/connect_fastapi_runtime.py", line 418, in main
2023/12/31 12:40:59 PM:     user_app = import_app(entrypoint, app_mode)
2023/12/31 12:40:59 PM:   File "/opt/rstudio-connect/python/connect_common_runtime.py", line 167, in import_app
2023/12/31 12:40:59 PM:     app = getattr(app, name_part)
2023/12/31 12:40:59 PM:   File "/opt/rstudio-connect/mnt/app/python/env/lib/python3.10/site-packages/shiny/express/app.py", line 13, in __getattr__
2023/12/31 12:40:59 PM:     return wrap_express_app(Path(name))
2023/12/31 12:40:59 PM:   File "/opt/rstudio-connect/mnt/app/python/env/lib/python3.10/site-packages/shiny/express/_run.py", line 44, in wrap_express_app
2023/12/31 12:40:59 PM:     app_ui = run_express(file)
2023/12/31 12:40:59 PM:   File "/opt/rstudio-connect/mnt/app/python/env/lib/python3.10/site-packages/shiny/express/_run.py", line 98, in run_express
2023/12/31 12:40:59 PM:     exec(
2023/12/31 12:40:59 PM:   File "/opt/rstudio-connect/mnt/app/app.py", line 2, in
2023/12/31 12:40:59 PM:     from shiny.express import input, ui
2023/12/31 12:40:59 PM: ImportError: cannot import name 'ui' from 'shiny.express' (/opt/rstudio-connect/mnt/app/python/env/lib/python3.10/site-packages/shiny/express/__init__.py)

Shinyapps.io deploys seem to be unaffected.

@karangattu Do you know how how the deployment script determines which version of Shiny to use? When I deploy locally, I'm able to reproduce the problem if I have this as my requirements.txt:
shiny

This will install Shiny from PyPI, and that version of Shiny does not have a shiny.express.ui module (it only has shiny.express.layout).
However, if I change the requirements.txt to this, the app deploys and runs just fine:
git+https://github.com/posit-dev/py-shiny

This installs Shiny from GitHub, and that version of Shiny does have shiny.express.ui.
In the Connect logs, if you look at "Python environment restore" and search for shiny, what version do you see? If you have shiny==<IP_ADDRESS>00, then it's using the GitHub version; if you see something else, then it's not using the correct version.
I don't know why this would be different between shinyapps.io and a Connect server, though.

@karangattu Do you know how how the deployment script determines which version of Shiny to use? When I deploy locally, I'm able to reproduce the problem if I have this as my requirements.txt:
shiny

This will install Shiny from PyPI, and that version of Shiny does not have a shiny.express.ui module (it only has shiny.express.layout).
However, if I change the requirements.txt to this, the app deploys and runs just fine:
git+https://github.com/posit-dev/py-shiny

This installs Shiny from GitHub, and that version of Shiny does have shiny.express.ui.
In the Connect logs, if you look at "Python environment restore" and search for shiny, what version do you see? If you have shiny==<IP_ADDRESS>00, then it's using the GitHub version; if you see something else, then it's not using the correct version.
(One more note: even if it has <IP_ADDRESS>00, it is possible that it's using an older commit, before the shiny.express.ui stuff was added. If that's the case, then I think it's likely that Connect's package caching scheme is using just the version number, and not looking at the commit hash.)
I don't know why this would be different between shinyapps.io and a Connect server, though.

The requirements.txt contains the following for each express app
git+https://github.com/posit-dev/py-shiny.git#egg=shiny
git+https://github.com/posit-dev/py-htmltools.git#egg=htmltools

As for the version the deploy script uses, it runs the setup on the dev version, as show in this GitHub actions workflow step so I assume it is using the develop version.
I ran pip install -e ".[dev,test]"  on my local and the deploys still don't go through. :(
I suspect shinyapps.io might be caching and the deploys might not be happening and the tests are reading the older version of the app. I have yet to test that theory though.

The python environment restore in connect logs show the following:
2024/01/03 10:05:36 PM: Packages in the environment: aiofiles==23.2.1, annotated-types==0.6.0, anyio==3.7.1, appdirs==1.4.4, asgiref==3.7.2, click==8.1.7, exceptiongroup==1.2.0, fastapi==0.105.0, h11==0.14.0, htmltools @ git+https://github.com/posit-dev/py-htmltools.git@3b6fa5e4d0fab58b6bd79deae09367ec0c1a69bf, idna==3.6, linkify-it-py==2.0.2, markdown-it-py==3.0.0, mdit-py-plugins==0.4.0, mdurl==0.1.2, packaging==23.2, prompt-toolkit==3.0.36, pydantic==2.5.2, pydantic_core==2.14.5, python-multipart==0.0.6, questionary==2.0.1, shiny @ git+https://github.com/posit-dev/py-shiny.git@ad20db166b220c0866488ddf72706ffabf2e1362, sniffio==1.3.0, starlette==0.27.0, typing_extensions==4.9.0, uc-micro-py==1.0.2, uvicorn==0.24.0.post1, watchfiles==0.21.0, wcwidth==0.2.12, websockets==12.0, wsproto==1.2.0,

Looking at the SHA, the connect env had the shiny version cached from 13 days ago.. when I delete the apps hosted on Connect and then deploy them again, things work fine. Needs more investigating why Connect did not pick the latest shiny, but for now to unblock the deploy tests, I will delete each of the hosted apps and then run the tests.

Fixed it by removing .git#egg=shiny from git+https://github.com/posit-dev/py-shiny.git#egg=shiny listed in the requirements.txt in the directory for each express app that we deploy to both connect and Shinyapps.io


Provide a SecureStorage

Provide an additional storage API to store data in a trusted environment on the native client. On iOS this would be the iOS KeyChain. The API should be the same as localStorage.
This cordova plugin may be considered as an alternative.

This is implemented on iOS for 1.9.

There is a global object secureStore available on iOS. It's still not documented and considered internal.

We agreed to declare secureStore public API. Documentation has been added. Fixed with b0bc79d.


google-cloud-sdk: 155.0.0 -> 159.0.0

Motivation for this change
google-cloud-sdk: 155.0.0 -> 159.0.0
Things done

[ ] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] Linux

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Thanks!


Some tests and changes to increase coverage in the code.

I added some tests to increase code coverage.  I also removed an unused method since GitHub requires urls in project description to contain http or https.

Current coverage is 99.03% (diff: 100%)

Merging #534 into master will increase coverage by 0.76%

@@             master       #534   diff @@
==========================================
  Files            40         40
  Lines          1038       1032     -6
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           1020       1022     +2
+ Misses           18         10     -8
  Partials          0          0

Powered by Codecov. Last update 85510b0...ac9438b


Compliance Report D | Output Table adjustments |1|

Task 1:
Bold the "Total" and "Carbon Intensity" number values to align with the bolded labels (see red box in attached screenshot).
Task 2:
Allow negative values for all input values within the output table. https://trello.com/c/z54uJT88/1515-compliance-report-d-output-table-adjustments-1

 Compliance Report D | Output Table adjustments |1|

Completed in Sprint Quasar


disambiguate Watir usage of the term 'parent'

This addresses #445

Except for the indentation, looks good!


Fix jenkins_build.sh not building the base image

The gcloud beta container builds submit was given the wrong
directory.  That directory didn't contain a Dockerfile, and thus
the build would fail.

FYI, you can remove the beta from that command now (since it went GA recently) :)

@nkubala Awesome.  I will remove the ß in a follow-up PR.


Headless Task Failed error message upon trying to open workspace with my private repo

Error message:
sudo -E /app/bob build
}; exit
15c3f72d-f296-4fb8-9497-ae938bd8812a:/workspace$ {
> sudo -E /app/bob build
> }; exit
{"file":"builder.go:226","func":"StartBuildkit","level":"debug","message":"buildkitd started","serviceContext":{"service":"bob","version":""},"severity":"DEBUG","stderr":"/tmp/buildkitd_stderr3333616279","stdout":"/tmp/buildkitd_stdout1790700550","time":"2023-09-26T21:21:48.136304167Z"}
{"attempt":0,"file":"builder.go:262","func":"connectToBuildkitd","level":"debug","message":"attempting to connect to buildkitd","serviceContext":{"service":"bob","version":""},"severity":"DEBUG","time":"2023-09-26T21:21:48.136526707Z"}
{"attempt":1,"file":"builder.go:262","func":"connectToBuildkitd","level":"debug","message":"attempting to connect to buildkitd","serviceContext":{"service":"bob","version":""},"severity":"DEBUG","time":"2023-09-26T21:21:49.137521544Z"}
{"file":"export.go:69","func":"Info","level":"info","message":"building base image","serviceContext":{"service":"bob","version":""},"severity":"INFO","time":"2023-09-26T21:21:49.152963023Z"}
{"file":"export.go:69","func":"Info","level":"info","message":"waiting for build context","serviceContext":{"service":"bob","version":""},"severity":"INFO","time":"2023-09-26T21:21:49.153019793Z"}
#1 [internal] load build definition from .gitpod.Dockerfile
#1 transferring dockerfile: 819B done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/library/ubuntu:20.04
#2 DONE 1.0s

#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [internal] load build context
#4 transferring context: 2B done
#4 DONE 0.0s

#5 [ 5/16] RUN apt-get install -yqq ssh
#5 CACHED

#6 [ 6/16] RUN apt-get install -yqq git
#6 CACHED

#7 [ 8/16] RUN mkdir ~/scripts
#7 CACHED

#8 [ 3/16] RUN apt-get install -yqq wget
#8 CACHED

#9 [ 4/16] RUN apt-get install -yqq ca-certificates
#9 CACHED

#10 [ 7/16] RUN apt-get clean && rm -rf /var/lib/apt/lists/*
#10 CACHED

#11 [ 9/16] COPY ./scripts/dl-graalvm.sh /root/scripts/dl-graalvm.sh
#11 ERROR: failed to calculate checksum of ref r4xtx9mgaw9pstaz5hy8qttw2::yt6jsbef4e62foedbel1m26lp: failed to walk /tmp/buildkit-mount3090133466/scripts: lstat /tmp/buildkit-mount3090133466/scripts: no such file or directory

#12 [10/16] COPY scripts/.shrc /root/.shrc
#12 ERROR: failed to calculate checksum of ref r4xtx9mgaw9pstaz5hy8qttw2::yt6jsbef4e62foedbel1m26lp: failed to walk /tmp/buildkit-mount3283856763/scripts: lstat /tmp/buildkit-mount3283856763/scripts: no such file or directory

#13 [ 1/16] FROM docker.io/library/ubuntu:20.04@sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e3b2026b4fc2faba
#13 resolve docker.io/library/ubuntu:20.04@sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e3b2026b4fc2faba done
#13 sha256:edaedc954fb53f42a7754a6e2d1b57f091bc9b11063bc445c2e325ea448f8f68 0B / 27.51MB
#13 ...

#14 [ 2/16] RUN apt-get update --yes &&     apt-get install --yes --no-install-recommends
#14 CANCELED
------
 > [ 9/16] COPY ./scripts/dl-graalvm.sh /root/scripts/dl-graalvm.sh:
------
------
 > [10/16] COPY scripts/.shrc /root/.shrc:
------
.gitpod.Dockerfile:19
--------------------
  17 |     RUN mkdir ~/scripts
  18 |     COPY ./scripts/dl-graalvm.sh /root/scripts/dl-graalvm.sh
  19 | >>> COPY scripts/.shrc /root/.shrc
  20 |
  21 |     RUN ssh-keyscan github.com
--------------------
error: failed to solve: failed to compute cache key: failed to calculate checksum of ref r4xtx9mgaw9pstaz5hy8qttw2::yt6jsbef4e62foedbel1m26lp: failed to walk /tmp/buildkit-mount3283856763/scripts: lstat /tmp/buildkit-mount3283856763/scripts: no such file or directory
{"@type":"type.googleapis.com/google.devtools.clouderrorreporting.v1beta1.ReportedErrorEvent","command":"build","error":"exit status 1","file":"build.go:44","func":"func1","level":"error","message":"build failed","serviceContext":{"service":"bob","version":""},"severity":"ERROR","time":"2023-09-26T21:21:50.239331331Z"}
exit

Error: headless task failed: exit status 1

This path has been fixed in #6 and we are deprecating the use of private forks as a needless complication. @de02E try starting a GitPod workspace on the class monorepo URL and re-open if this continues to be a problem.
https://github.com/TheEvergreenStateCollege/upper-division-cs


Does this work in a browser?

I was using electron-machine-id and updated to this thinking it would work in chrome?
Does it still require electron or NW?
ERROR in ./node_modules/node-machine-id/dist/index.js
Module not found: Error: Can't resolve 'child_process' in 'f:\Github\electron-store\node_modules\node-machine-id\dist'
@ ./node_modules/node-machine-id/dist/index.js 1:82-106 1:18348-18372
@ ./src/js/helper.js
@ ./src/purchase-entry.js
@ multi (webpack)-dev-server/client?http://localhost:8080 webpack/hot/dev-server ./src/purchase-entry.js

@sgehrman module can't work with browser, but have full support for electron and NW.
In your case it seems to me is webpack configuration issue


missing a few features: editable metadata,  json/map export,  slideshows with live coding

I have a very intensive use of notebook environments (Ipython, Sage, etc.) for my work, and I'm (also) a long-time lisper learning clojure  !   I really love the simplicity of Gorilla-repl but I miss a few features regarding my own workflow. These features are available in e.g. the Ipython notebooks:

the possibility to annotate the cells (blocks)  with editable (JSon or other EDN) metadata.
For example, slide annotations for slideshows,  skipme tag for skipping when producing
a pdf,   answers when I am authoring exercises, etc.

the possibility to export to an easily parsable and scriptable file format (think JSon, or clojure map EDN)
I find it nice that a gorilla worksheet is also a clojure source file,  but I would still enjoy a more easily parsable export for later processing (especially  markdown generation for pandoc).  The metadata would play a significant role for the processing of course. For example to skip some blocks...

a  slidehow view in which the blocks remain editable  (think reveal.js but maybe something more "zen", in the spirit of gorilla as I see it).
Slideshows & live coding is for me a "killer feature"... excellent  for courses (it seems the students enjoy this) and not-too-boring presentations...

(sorry for the noice, these are not actual issues).

Hi Fred,
thanks for the comments - not noise at all, always like to hear people's views. It raises an interesting point as well, which I've been thinking about lately since one of the IPython devs questioned why not write a Clojure kernel for IPython, rather than Gorilla REPL? I think my answer to that question, which touches upon the points you mentioned, is that I don't really see Gorilla REPL as quite the same as IPython. Rather I think it makes its own design choices.
So, to address your particular points:

I've strenuously resisted the addition of cell- or worksheet-level metadata, and think I want to continue to do so! Not because it's not a good idea, and I appreciate that would enable a lot of different other good ideas. But just because I want to keep the core of Gorilla as simple as is possible. I think this is one of the points where I see Gorilla as taking a different path than IPython.

Interesting idea. Given the explicit "read" step in LISP I wonder whether this already exists though? I've never looked closely at tools.reader, but it might already do precisely what is wanted here - convert a source file into a data structure.

See 1 :-p . Again, I don't think this is a bad idea at all, just one that I don't see as fitting with how I want Gorilla REPL to be.

I hope I don't seem too disagreeable - once again, I don't mean to suggest these are bad ideas, just that I want to keep a clear vision of what Gorilla should be. I should say too, for the record, I think it would be a great thing to have a well supported IPython kernel for Clojure :-)
Jony

So, there's been some further discussion of cell-level metadata on #192 . Might be worth reconsidering these ideas once that has shaken out.
I'm going to close this one for the moment, but feel free to comment/reopen.

I ran across this issue when Googling for a converter to use on a project of mine. I ended up writing one - I've put it in a gist.
I'd be happy to bundle this into a library and throw it onto clojars if there isn't already a solution (or plans for one). Since I wrote it as a one-off I think it needs a little work to make it more robust; I'd welcome any feedback on that front as well.


google.protobuf error

(test) D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master>python aleappGUI.py
Traceback (most recent call last):
  File "aleappGUI.py", line 1, in <module>
    import aleapp
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\aleapp.py", line 9, in <module>
    from scripts.ilap_artifacts import *
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\scripts\ilap_artifacts.py", line 53, in <module>
    from scripts.artifacts.usagestats import get_usagestats
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\scripts\artifacts\usagestats.py", line 4, in <module>
    import scripts.artifacts.usagestats_pb.usagestatsservice_pb2 as usagestatsservice_pb2
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\scripts\artifacts\usagestats_pb\usagestatsservice_pb2.py", line 14, in <module>
    from . import configuration_pb2 as configuration__pb2
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\scripts\artifacts\usagestats_pb\configuration_pb2.py", line 14, in <module>
    from . import window_configuration_pb2 as window__configuration__pb2
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\scripts\artifacts\usagestats_pb\window_configuration_pb2.py", line 14, in <module>
    from . import rect_pb2 as rect__pb2
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\scripts\artifacts\usagestats_pb\rect_pb2.py", line 14, in <module>
    from . import privacy_pb2 as privacy__pb2
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\scripts\artifacts\usagestats_pb\privacy_pb2.py", line 15, in <module>
    from . import descriptor_pb2  as descriptor__pb2
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\scripts\artifacts\usagestats_pb\descriptor_pb2.py", line 2010, in <module>
    'Location' : _reflection.GeneratedProtocolMessageType('Location', (_message.Message,), {
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\test\test\lib\site-packages\google\protobuf\internal\python_message.py", line 189, in __init__
    _AttachFieldHelpers(cls, field)
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\test\test\lib\site-packages\google\protobuf\internal\python_message.py", line 295, in _AttachFieldHelpers
    field_descriptor.GetOptions().packed)
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\test\test\lib\site-packages\google\protobuf\descriptor.py", line 147, in GetOptions
    from google.protobuf import descriptor_pb2
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\test\test\lib\site-packages\google\protobuf\descriptor_pb2.py", line 112, in <module>
    _sym_db.RegisterEnumDescriptor(_FIELDDESCRIPTORPROTO_TYPE)
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\test\test\lib\site-packages\google\protobuf\symbol_database.py", line 107, in RegisterEnumDescriptor
    self.pool._AddEnumDescriptor(enum_descriptor)
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\test\test\lib\site-packages\google\protobuf\descriptor_pool.py", line 263, in _AddEnumDescriptor
    self._CheckConflictRegister(enum_desc, enum_desc.full_name, file_name)
  File "D:\Forensic_Challenge\digitalcorpora-Android 10\ALEAPP-master\test\test\lib\site-packages\google\protobuf\descriptor_pool.py", line 191, in _CheckConflictRegister
    raise TypeError(error_msg)
TypeError: Conflict register for file "google/protobuf/descriptor.proto": google.protobuf.FieldDescriptorProto.Type is already defined in file "descriptor.proto". Please fix the conflict by adding package name on the proto file, or use different name for the duplication.

Got 3 emails stating this is an issue. Going to look into it.
Thoughts @ydkhatri?

Seems to be a rehash of the one we had earlier (last year). There are problems with google's protobuf. It does not perform the same way on every platform and every version of python. I recall the last time we had narrowed this down to a specific version of protobuf (on one particular platform) that did not work. Seems we need to do that again.
Can you ask whoever emailed you to provide-

the version of protobuf installed
version of python installed
platform (os)

Doing so now.
On Thu, Dec 10, 2020, 3:26 PM Yogesh Khatri (@swiftforensics) <
<EMAIL_ADDRESS>wrote:

Seems to be a rehash of the one we had earlier (last year). There are
problems with google's protobuf. It does not perform the same way on every
platform and every version of python. I recall the last time we had
narrowed this down to a specific version of protobuf (on one particular
platform) that did not work. Seems we need to do that again.
Can you ask whoever emailed you to provide-

the version of protobuf installed
version of python installed
platform (os)

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/abrignoni/ALEAPP/issues/79#issuecomment-742777494,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AG3DPC3RZZHLRMWTMOGRFY3SUEVH3ANCNFSM4ULREV6A
.

Python 3.9 (32bit, 64bit)
Windows 10 (Version 2004) @ydkhatri

Python 3.9 (32bit, 64bit)
Windows 10 (Version 2004) @ydkhatri
It works normally in 3.8.

I get the same with python 3.9. Works with earlier version (3.7, 3.8). For now, just avoid python 3.9

OK, this seems to fix it for python 3.9 (and protobuf 3.14.0) on windows 10.
In file \scripts\artifacts\usagestats_pb\privacy_pb2.py
Change
from . import descriptor_pb2  as descriptor__pb2
to
from google.protobuf import descriptor_pb2  as descriptor__pb2
Please run it and see if it parses usagestats. I will later test with older versions and see if that works too.

nice :) @ydkhatri

Good evening!
I hate to be the bearer of bad news, but I'm afraid this issue extends beyond Python 3.9, so I'd like to see if we can re-open the issue.
When installing aleapp on a barebones installation of Ubuntu 20.04 (which comes with Python 3.8 by default), I get the following:
root@09ce0a2b6800:/aleapp# python3 aleapp.py
Traceback (most recent call last):
  File "aleapp.py", line 9, in <module>
    from scripts.ilap_artifacts import *
  File "/aleapp/scripts/ilap_artifacts.py", line 53, in <module>
    from scripts.artifacts.usagestats import get_usagestats
  File "/aleapp/scripts/artifacts/usagestats.py", line 4, in <module>
    import scripts.artifacts.usagestats_pb.usagestatsservice_pb2 as usagestatsservice_pb2
  File "/aleapp/scripts/artifacts/usagestats_pb/usagestatsservice_pb2.py", line 14, in <module>
    from . import configuration_pb2 as configuration__pb2
  File "/aleapp/scripts/artifacts/usagestats_pb/configuration_pb2.py", line 14, in <module>
    from . import window_configuration_pb2 as window__configuration__pb2
  File "/aleapp/scripts/artifacts/usagestats_pb/window_configuration_pb2.py", line 14, in <module>
    from . import rect_pb2 as rect__pb2
  File "/aleapp/scripts/artifacts/usagestats_pb/rect_pb2.py", line 14, in <module>
    from . import privacy_pb2 as privacy__pb2
  File "/aleapp/scripts/artifacts/usagestats_pb/privacy_pb2.py", line 17, in <module>
    from . import descriptor_pb2  as descriptor__pb2
  File "/aleapp/scripts/artifacts/usagestats_pb/descriptor_pb2.py", line 2010, in <module>
    'Location' : _reflection.GeneratedProtocolMessageType('Location', (_message.Message,), {
  File "/usr/local/lib/python3.8/dist-packages/google/protobuf/internal/python_message.py", line 189, in __init__
    _AttachFieldHelpers(cls, field)
  File "/usr/local/lib/python3.8/dist-packages/google/protobuf/internal/python_message.py", line 295, in _AttachFieldHelpers
    field_descriptor.GetOptions().packed)
  File "/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py", line 147, in GetOptions
    from google.protobuf import descriptor_pb2
  File "/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor_pb2.py", line 112, in <module>
    _sym_db.RegisterEnumDescriptor(_FIELDDESCRIPTORPROTO_TYPE)
  File "/usr/local/lib/python3.8/dist-packages/google/protobuf/symbol_database.py", line 107, in RegisterEnumDescriptor
    self.pool._AddEnumDescriptor(enum_descriptor)
  File "/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor_pool.py", line 263, in _AddEnumDescriptor
    self._CheckConflictRegister(enum_desc, enum_desc.full_name, file_name)
  File "/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor_pool.py", line 191, in _CheckConflictRegister
    raise TypeError(error_msg)
TypeError: Conflict register for file "google/protobuf/descriptor.proto": google.protobuf.FieldDescriptorProto.Type is already defined in file "descriptor.proto". Please fix the conflict by adding package name on the proto file, or use different name for the duplication.

root@09ce0a2b6800:/aleapp# python3 -V
Python 3.8.5

Means to duplicate:
apt install git -y
git clone https://github.com/abrignoni/aleapp
cd aleapp/
pip3 install -r requirements.txt
python3 aleapp.py

Since this actually works for python3.8.5 as well, any chance you can fix this so it covers this version?

When installing aleapp on a barebones installation of Ubuntu 20.04 (which comes with Python 3.8 by default), I get the following:

You are correct, just tested with linux and macOS (neither works with 3.8), but work with 3.9 now! I believe that one line if statement I added needs more fine-tuned logic. Need to test py 3.7, 3.8, 3.9 across Win/Linux/macOS and tweak it to make the right choice.

@digitalsleuth , try now. I changed the code, and tested that it works on ubuntu 3.8 and 3.9.

Closing, as it seems to be resolved.

Thank you so much. Been working with folks over email. It is working now.
On Mon, Dec 14, 2020, 10:17 AM Yogesh Khatri (@swiftforensics) <
<EMAIL_ADDRESS>wrote:

Closing, as it seems to be resolved.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/abrignoni/ALEAPP/issues/79#issuecomment-744509433,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AG3DPC3UEFLMA5JRSTKRAHDSUYUCFANCNFSM4ULREV6A
.


 It will not be cached if it is empty. Is it necessary to do it?
https://github.com/thephpleague/flysystem-cached-adapter/blob/master/src/CachedAdapter.php#L251
regards

Fixed in 93eec679c

Just tagged 1.0.6 for this.


Installation: Low cost installation

What would you like to be added?
Karpor currently only support ElasticSearch, but the installation cost of ElasticSearch is relatively high and we need to consider a low-cost installation method, sush as support ZincSearch, SQLLite etc (#230 ).
If we completely replace etcd and es with sqlite (which should be technically feasible), we can even use a binary to launch karpor independently, independent of kubernetes and docker, or even install it with one click using package management tools such as homebrew. This is so cool.
Why is this needed?
No response

I check the code and didn't find etcd is used for what.

@solarhell karpor-server is essentially a kubernetes apiserver, so it needs etcd as storage behind it.


Garbled screen

Hi, as far as I can see, I have everything working on the software side, i.e. the screen comes on when a song is played but unfortunately it is full of white dots 8-P (totally garbled).
I have done a clean install (Volumio, Docker, pydPiper as stated in another post here) with no effect. The screen is OK because I got it to work normally with a python script provided by Adafruit.
Any help would be muuuuuch appreciated. Thanks.
Configuration is:
Raspberry Pi3, Oled SSD1306 via i2c on 0x3D, Volumio 2.368, Allo Kali, Allo Piano2.1 DAC.
/etc/systemd/system/pydpiper.service is
ExecStart=/usr/bin/docker run --network=host --privileged -v /var/log:/var/log:rw  dhrone/pydpiper:v0.31-alpha python /app/pydPiper.py --volumio --driver ssd1306_i2c  --width 128 --height 64 --timezone 'paris/france' --temperature celsius --pages pages_ssd1306.py --i2caddress 0x3D --i2cport 1

The library that pydPiper uses to display to the ssd1306 is Luma.  https://pypi.python.org/pypi/luma.oled.  See if you can get any of their demo programs running on the display.

I have successfully tested my SSD1301 Oled with the Adafuit scripts which are based on the luma scripts I think. Looking though the luma-oled.readthedocs.io site I found this article describing the issue I'm having. It seems to be related to the display initiation.
https://luma-oled.readthedocs.io/en/latest/troubleshooting.html
Basically, it says to use the SH1106 driver instead of the SSD1301 driver. I did try to install luma but got multiple errors and gave up. I'm not a developer, so for me its pretty hard to navigate and comprehend all this information. But as said, the display works with the Adafruit packages.
Where is the snippet that initiates the display ? Maybe I could try and get it out of one of the Adafruit scripts and paste it somewhere in a pydpiper script ?
thanks

update: finally got luma to work. All demos I tried work with the sh1106 driver ;-)))

The SH1106 driver is already integrated into pydPiper.  Try the following start line...
ExecStart=/usr/bin/docker run --network=host --privileged -v /var/log:/var/log:rw dhrone/pydpiper:v0.31-alpha python /app/pydPiper.py --volumio --driver luma_i2c --devicetype sh1106  --width 128 --height 64 --timezone 'paris/france' --temperature celsius --pages pages_ssd1306.py --i2caddress 0x3D --i2cport 1

hi, no unfortunately the screen doesn't even come on.

hmm, not sure what's going on but it only says "NONE". I restarted the daemon before testing and cut the DC to the screen before restarting pydpiper.

hmm, unfortunately nothing or snow on the screen. I will reinstall everything and try later.

after clean install, still not working. Reinstalled luma as well and it works.

It's not clear to me from your message.  Are you saying that the Luma demo programs are working but that using pydPiper with --driver luma_i2c --devicetype sh1106 does not?

yes, correct. It works with luma but not with the pydPiper and luma_i2c driver

Interesting.  You can run the luma_i2c driver in a test mode without running the rest of pydPiper.
Try...
sudo /usr/bin/docker run -ti --network=host --privileged -v /var/log:/var/log:rw dhrone/pydpiper:v0.31-alpha /bin/bash
This will start a bash shell inside the docker container.  Once you are at the bash prompt inside the container...
cd displays
python luma_i2c.py -r 64 -c 128 --devicetype sh1106 --i2c_address 61 --i2c_port 1
Note that the command line argument parser is slightly different than what pydPiper.py uses with the biggest potential confusion being that the i2c_address needs to be specified in decimal.  This example should work though if your address is 0x3D.

hi dhrone, sorry, all I get is "snow" (same as in my first pic) and a lengthly output.

root@volumio:/app/displays# python luma_i2c.py -r 64 -c 128 --devicetype sh1106 --i2c_address 61 --i2c_port 1

LUMA OLED Display Test
ROWS=64, COLS=128, DEVICETYPE=sh1106, I2C_ADDRESS=61, I2C_PORT=1
Loading font from /app/displays/fonts/latin1_5x8_fixed.fnt
2018-03-09 22:19:13,519:DEBUG:STREAM 'IHDR' 16 13
2018-03-09 22:19:13,521:DEBUG:STREAM 'gAMA' 41 4
2018-03-09 22:19:13,523:DEBUG:STREAM 'pHYs' 57 9
2018-03-09 22:19:13,524:DEBUG:STREAM 'iTXt' 78 1061
2018-03-09 22:19:13,526:DEBUG:STREAM 'IDAT' 1151 3193
2018-03-09 22:19:13,596:DEBUG:Error closing: 'NoneType' object has no attribute 'close'
2018-03-09 22:19:16,395:DEBUG:Loading ../pages_ssd1306.py as page file
2018-03-09 22:19:16,397:DEBUG:Loading font small
Loading font from /app/displays/fonts/latin1_5x8.fnt
2018-03-09 22:19:16,436:DEBUG:STREAM 'IHDR' 16 13
2018-03-09 22:19:16,437:DEBUG:STREAM 'gAMA' 41 4
2018-03-09 22:19:16,438:DEBUG:STREAM 'pHYs' 57 9
2018-03-09 22:19:16,439:DEBUG:STREAM 'iTXt' 78 1061
2018-03-09 22:19:16,440:DEBUG:STREAM 'IDAT' 1151 3193
2018-03-09 22:19:16,510:DEBUG:Error closing: 'NoneType' object has no attribute 'close'
2018-03-09 22:19:16,511:DEBUG:Loading font large
Loading font from /app/displays/fonts/Vintl01_10x16.fnt
2018-03-09 22:19:16,551:DEBUG:STREAM 'IHDR' 16 13
2018-03-09 22:19:16,552:DEBUG:STREAM 'gAMA' 41 4
2018-03-09 22:19:16,553:DEBUG:STREAM 'pHYs' 57 9
2018-03-09 22:19:16,554:DEBUG:STREAM 'iTXt' 78 1062
2018-03-09 22:19:16,555:DEBUG:STREAM 'IDAT' 1152 7058
2018-03-09 22:19:16,659:DEBUG:Error closing: 'NoneType' object has no attribute 'close'
2018-03-09 22:19:16,661:DEBUG:Loading font tiny
Loading font from /app/displays/fonts/upperasciiwide_3x5_fixed.fnt
2018-03-09 22:19:16,677:DEBUG:STREAM 'IHDR' 16 13
2018-03-09 22:19:16,679:DEBUG:STREAM 'gAMA' 41 4
2018-03-09 22:19:16,680:DEBUG:STREAM 'pHYs' 57 9
2018-03-09 22:19:16,681:DEBUG:STREAM 'iTXt' 78 1058
2018-03-09 22:19:16,682:DEBUG:STREAM 'IDAT' 1148 726
2018-03-09 22:19:16,708:DEBUG:Error closing: 'NoneType' object has no attribute 'close'
2018-03-09 22:19:16,709:DEBUG:Loading font DejaVuSans28
2018-03-09 22:19:16,717:DEBUG:Loading image splash
2018-03-09 22:19:16,718:DEBUG:STREAM 'IHDR' 16 13
2018-03-09 22:19:16,719:DEBUG:STREAM 'gAMA' 41 4
2018-03-09 22:19:16,721:DEBUG:STREAM 'pHYs' 57 9
2018-03-09 22:19:16,722:DEBUG:STREAM 'iTXt' 78 1060
2018-03-09 22:19:16,723:DEBUG:STREAM 'IDAT' 1150 295
2018-03-09 22:19:16,724:DEBUG:Loading image progbar
2018-03-09 22:19:16,726:DEBUG:STREAM 'IHDR' 16 13
2018-03-09 22:19:16,727:DEBUG:STREAM 'gAMA' 41 4
2018-03-09 22:19:16,728:DEBUG:STREAM 'pHYs' 57 9
2018-03-09 22:19:16,729:DEBUG:STREAM 'iTXt' 78 1057
2018-03-09 22:19:16,730:DEBUG:STREAM 'IDAT' 1147 107
2018-03-09 22:19:16,732:DEBUG:Loading widget xofy
2018-03-09 22:19:16,734:DEBUG:Loading widget weather
2018-03-09 22:19:16,737:DEBUG:Adding effect scroll to weather which is currently size (40, 16)
2018-03-09 22:19:16,739:DEBUG:Scroll added with widget size (60, 16)
2018-03-09 22:19:16,740:DEBUG:Loading widget temp
2018-03-09 22:19:16,743:DEBUG:Loading widget artist
2018-03-09 22:19:16,747:DEBUG:Adding effect scroll to artist which is currently size (34, 16)
2018-03-09 22:19:16,749:DEBUG:Scroll added with widget size (54, 16)
2018-03-09 22:19:16,750:DEBUG:Loading widget songprogress
2018-03-09 22:19:16,754:DEBUG:Loading widget title
2018-03-09 22:19:16,760:DEBUG:Adding effect scroll to title which is currently size (86, 8)
2018-03-09 22:19:16,762:DEBUG:Scroll added with widget size (106, 8)
2018-03-09 22:19:16,763:DEBUG:Loading widget elapsed
2018-03-09 22:19:16,766:DEBUG:Loading widget volume
2018-03-09 22:19:16,771:DEBUG:Loading widget length
2018-03-09 22:19:16,774:DEBUG:Loading widget ampm
2018-03-09 22:19:16,781:DEBUG:Loading widget volumelarge
2018-03-09 22:19:16,786:DEBUG:Loading widget temphilow
2018-03-09 22:19:16,789:DEBUG:Loading widget time
2018-03-09 22:19:16,797:DEBUG:Loading widget volumebar
2018-03-09 22:19:16,802:DEBUG:Loading widget bitdepth
2018-03-09 22:19:16,805:DEBUG:Loading widget samplerate
2018-03-09 22:19:16,809:DEBUG:Loading widget ttime
2018-03-09 22:19:16,821:DEBUG:Loading widget tempsmall
2018-03-09 22:19:16,823:DEBUG:Loading widget stoptimetemp_popup
2018-03-09 22:19:16,826:DEBUG:Loading widget volume_changed
2018-03-09 22:19:16,828:DEBUG:Loading widget stoptime
2018-03-09 22:19:16,829:DEBUG:Loading widget playing
2018-03-09 22:19:16,831:DEBUG:Loading sequence seqPlay
2018-03-09 22:19:16,832:DEBUG:Loading sequence seqStop
2018-03-09 22:19:16,833:DEBUG:Loading sequence seqVolume

any idea why the screen works with lumina but not with pydpiper/docker ? I have tried all sorts of combinations in the pydpiper.service file with no success. The only thing that works is that the screen comes on (covered in random dots) when pydpiper is activated.

Still looks like a driver issue.  I don't have a SH1106 to test with but placed one on order so should be able to test it soon.

Hi, thanks. I’ve ordered a couple of screens from different manufacturers.
all the best
rupert

On 13 Mar 2018, at 02:40, dhrone<EMAIL_ADDRESS>wrote:
Still looks like a driver issue. I don't have a SH1106 to test with but placed one on order so should be able to test it soon.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/dhrone/pydPiper/issues/25#issuecomment-372517676, or mute the thread https://github.com/notifications/unsubscribe-auth/ARGs2VHXCQEQPF4HVA1jyVdQSu5Ng65uks5tdyNxgaJpZM4ScczP.

I'm beginning to think it is a "reset" issue. I am taking the signals from a Piano 2.1 DAC header and ma not using RST. Just wondering out loud after having read this ..
https://github.com/adafruit/Adafruit_SSD1306/issues/111

Are you still experiencing display problems?

hi Dhrone,
sorry, but I gave up and havn’t had time to revive the project… if / when I will (revive it), I’ll def let you know via github..
all the best and greetings
rupert

On 25 May 2018, at 21:46, dhrone<EMAIL_ADDRESS>wrote:
Are you still experiencing display problems?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/dhrone/pydPiper/issues/25#issuecomment-392161433, or mute the thread https://github.com/notifications/unsubscribe-auth/ARGs2Sc1ivPdXMbP8E6jNV-YlkB_f4Pnks5t2F95gaJpZM4ScczP.

Hi, rebooted this issue with a 3$ sh1106 display from Aliexpress. I got this display to work with cava/antiprism package. But with your solution, I get horizontal lines with garbled text. But at least, the display seems to work :-) I used the service parameters referenced in for the sh1106 driver.
ExecStart=/usr/bin/docker run --network=host --privileged -v /var/log:/var/log:rw dhrone/pydpiper:v0.31-alpha python /app/pydPiper.py --volumio --driver luma_i2c --devicetype sh1106 --width 128 --height 64 --timezone 'paris/france' --temperature celsius --pages pages_ssd1306.py --i2caddress 0x3D --i2cport 1


✨ Example pr 12

I'm opening this pull request for the  branch, pushed by @AnandChowdhary with 1 commits.

CLA Assistant Lite bot ✍ All contributors have signed the CLA


[BUG] Multiple parsers don't generate statements.csv

statements.csv is not generated in the output folder when selecting both revolut and T212. Individual parsers generate the file with no issues. I'm using the exe file, if the output folder is the same as exe there is no statement.csv in root folder, if I select a different sub-folder for results then the csv file is there.

Please, confirm you're using the directory structure required. Keep in mind that with multiple parsers selected, statements.csv files for each parser are organized into sub-directories of the output directory.
Please, provide any errors that you're receiving and follow the BUG template.

I found the statements.csv per parser in the sub-folders. Is it possible to combine them in root folder similar to xml and other csv output structures?

Thank you for following up on this issue.No. That behavior is intentional. I find it easier to have them separated in directories for each parser. In the future, there might be more files per parser, so I don't want to introduce a complex naming convention for each file type.


[C++][FS][Azure] CopyFile doesn't work with Azure hierachical namespace support

Describe the enhancement requested
[ RUN      ] TestAzureHierarchicalNSGeneric.CopyFile
/home/kou/work/cpp/arrow.kou/cpp/src/arrow/filesystem/test_util.cc:531: Failure
Failed
'fs->CopyFile("AB/abc", "def")' failed with IOError: Failed to copy a blob. (https://XXX.blob.core.windows.net/a2giu5a9ij1szn7tsx4l0z6194h5zqww/AB/abc -> https://clearcodearrow.blob.core.windows.net/a2giu5a9ij1szn7tsx4l0z6194h5zqww/def) Azure Error: [CannotVerifyCopySource] 401 Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
RequestId:2c08ee06-f01e-0062-12d2-807ca0000000
Time:2024-03-28T05:40:28.1650700Z
Request ID: 2c08ee06-f01e-0062-12d2-807ca0000000
[  FAILED  ] TestAzureHierarchicalNSGeneric.CopyFile (1007 ms)
[       OK ] TestAzureHierarchicalNSGeneric.OpenInputFileAsync (394 ms)
[ RUN      ] TestAzureHierarchicalNSGeneric.SpecialChars
/home/kou/work/cpp/arrow.kou/cpp/src/arrow/filesystem/test_util.cc:1178: Failure
Failed
'fs->CopyFile("Blank Char/Special%Char.txt", "Special and%different.txt")' failed with IOError: Failed to copy a blob. (https://XXX.blob.core.windows.net/armiaz82i30wliew7324v0l421a4uhop/Blank%20Char/Special%25Char.txt -> https://XXX.blob.core.windows.net/armiaz82i30wliew7324v0l421a4uhop/Special%20and%25different.txt) Azure Error: [CannotVerifyCopySource] 401 Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
RequestId:dbfb99e1-801e-008c-4ad2-80d689000000
Time:2024-03-28T05:40:38.3406267Z
Request ID: dbfb99e1-801e-008c-4ad2-80d689000000
[  FAILED  ] TestAzureHierarchicalNSGeneric.SpecialChars (475 ms)

Component(s)
C++

https://learn.microsoft.com/en-us/rest/api/storageservices/data-lake-storage-gen2
It seems that Copy API doesn't exist...?

We use CopyFromUri [1]. The issue here doesn't seem to be HNS, but authorizing the source with the same credentials used to create the arrow::fs::ArrowFileSystem.

Server failed to authenticate the request. Please refer to the information in the www-authenticate header.

[1]:
  Azure::Response<Models::CopyBlobFromUriResult> BlobClient::CopyFromUri(
      const std::string& sourceUri,
      const CopyBlobFromUriOptions& options,
      const Azure::Core::Context& context) const

What is XXX?

I wanted to mask the account but I missed one substitution...

Can you report the issue to Microsoft? The function is provided by the SDK and it doesn't seem right that it's failing to authorize the access.

I couldn't find whether should we report this... So I've opened a question issue in Azure/azure-sdk-for-cpp: https://github.com/Azure/azure-sdk-for-cpp/issues/5542

https://github.com/Azure/azure-sdk-for-cpp/issues/5542#issuecomment-2062908150

it seems the source URL doesn't contain any authentication information.
You can use SAS token to authenticate the copy source.

It's a bit disappointing that Azure doesn't support this without SAS toek, though it does match up with my memory https://github.com/apache/arrow/pull/39058#issuecomment-1838159705

Issue resolved by pull request 41276
https://github.com/apache/arrow/pull/41276


how to load the pre-trained word vectors of histwords using gensim?

Thank you for your great work. I am wondering can I load the pre-trained word vectors of histwords using gensim?

Hmm, do you mean to load the pre-trained vectors into Gensim to do more training? Or just to use the gensim API for KeyedVectors (https://radimrehurek.com/gensim/models/keyedvectors.html)?
If you just want to load them into a KeyedVectors vector you need to convert them to a format GenSim understands. For example, after loading in an Embedding object in HistWords you could run something like
with open("gensim_compat_file.txt", "w") as fp:     for word in embeddings.iw:         fp.write(word + " ".join(list(embeddings[word])) + "\n")
That should output the embeddings in "C text" format, which gensim can read.

Thank you very much. This solution works perfectly.
from representations.embedding import Embedding

embeddings = Embedding.load("../data/english_all_sgns/1900")

with open("../data/english_all_sgns_gensim_1900.txt", "w") as fp:
    for word in embeddings.iw:
        fp.write(word + " ".join(map(str, (embeddings[word]) )) + "\n")


Allow Pipelines to be used with different Resources

This change makes it possible to reuse a Pipeline with different Resources without having to change the Pipeline itself. It does this by moving the linking of the Resources a Task requires from the Pipeline into the PipelineRun.
The relationship between Resources and the Tasks that are expected to be executed on them is still expressed in the Pipeline (via providedBy).
ResourceBindings move from Pipeline to PipelineRun and become ResourceDependencies. No longer calling these "bindings" in the API or using the term "source" fixes + the additional docs in using.md fixes #139.
Removing version b/c we had been hoping to use version to cover two cases:

To allow a user to override the version of the Resource they are using (e.g. for a particular git resource, use a different revision)
For the case where a Task modifies a resource then provides it to the next Task, we wanted to express the modification in the version

But now a user can "change" a Resource by creating a new one and specifying the new one in teh PipelineRun, and they can do the same thing via a TaskRun.
For the second case, in #124 @shashwathi is designing how the output of one task will be passed to the next, and it is likely going to be via PVCs and will not need the version field.
This interface is still probably not in its final form and hopefully we can iterate on it!

Fixes #200

@pivotal-nader-ziada I re-added the test case that I (shouldn't have) removed for the branch case, now updated to get the branch from the resource instead, PTAL

@pivotal-nader-ziada I re-added the test case that I (shouldn't have) removed for the branch case, now updated to get the branch from the resource instead, PTAL

I should make sure this is clear in the docs too!

Okay added some more docs :D

pkg/reconciler/v1alpha1/pipelinerun/pipelinerun_test.go:439:18: undefined: "github.com/knative/build-pipeline/pkg/apis/pipeline/v1alpha1".TaskRunResourceVersion

whoops! should have ran the tests after rebasing :D

/lgtm
/meow space


Bump to use latest released cardano-base and cardano-prelude packages

The release branch is using an unreleased commit of cardano-base which contains the SECP primitives but is before the heapwords split. This makes it incompatible with either of the released versions:

The previous version doesn't have the SECP primitives
The next version has the heapwords split

In order to make the release branch usable via CHaP, I think the simplest thing is to backport the changes for the newer cardano-base packages, i.e. the heapwords split. This is a bit tedious but not too bad overall.
Afterwards I'll release another version from this branch.

Hmm, I'm just going to double check this works, do not merge for now.

thank you @michaelpj !

@michaelpj why did you end up closing this PR and not pursuing it to be merged into 1.1.x?

Because I was told that 1.1.x was not a real release and anyway I was trying to fix<EMAIL_ADDRESS>which was using 1.0.x not 1.1.x so I was doing completely the wrong thing anyway.

We'll use this as the basis for the 1.1.1 release, for use in https://github.com/input-output-hk/cardano-node/pull/4608.
Todo:

[ ] CI green
[ ] bump version
[ ] release 1.1.1 to CHaP

Sigh, so this is green, except for a busted cicero call.


fix: request-validator plugin cannot be created when using +json suffix

https://github.com/Kong/go-apiops/issues/174

Codecov Report
Attention: Patch coverage is 60.00000% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 67.79%. Comparing base (8ffe079) to head (f848b3d).

Files
Patch %
Lines

openapi2kong/validator.go
60.00%
2 Missing and 2 partials :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #175      +/-   ##
==========================================
- Coverage   67.79%   67.79%   -0.01%
==========================================
  Files          23       23
  Lines        2481     2490       +9
==========================================
+ Hits         1682     1688       +6
- Misses        641      642       +1
- Partials      158      160       +2

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


python311Packages.datasets: 2.19.0 -> 2.19.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python311Packages.datasets is: Open-access datasets and evaluation metrics for natural language processing
meta.homepage for python311Packages.datasets is: https://github.com/huggingface/datasets
meta.changelog for python311Packages.datasets is: https://github.com/huggingface/datasets/releases/tag/2.19.1
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 2.19.1 with grep in /nix/store/x21skvrdq49g5pwmnf1pgiinzl5awfr6-python3.11-datasets-2.19.1
found 2.19.1 in filename of file in /nix/store/x21skvrdq49g5pwmnf1pgiinzl5awfr6-python3.11-datasets-2.19.1

Rebuild report (if merged into master) (click to expand)

100 total rebuild path(s)

100 package rebuild(s)

First fifty rebuilds by attrpath

khoj
libretranslate
open-interpreter
pentestgpt
python311Packages.accelerate
python311Packages.anthropic
python311Packages.argos-translate-files
python311Packages.argostranslate
python311Packages.chromadb
python311Packages.cleanlab
python311Packages.cohere
python311Packages.ctranslate2
python311Packages.dalle-mini
python311Packages.datasets
python311Packages.diffusers
python311Packages.dm-sonnet
python311Packages.evaluate
python311Packages.farm-haystack
python311Packages.fastembed
python311Packages.faster-whisper
python311Packages.gradio
python311Packages.gradio-client
python311Packages.gradio-pdf
python311Packages.instructor
python311Packages.k-diffusion
python311Packages.langchain
python311Packages.langchain-community
python311Packages.langchain-core
python311Packages.langchain-text-splitters
python311Packages.langsmith
python311Packages.libretranslate
python311Packages.litellm
python311Packages.llama-index
python311Packages.llama-index-cli
python311Packages.llama-index-embeddings-huggingface
python311Packages.llama-index-llms-openai-like
python311Packages.llama-index-vector-stores-chroma
python311Packages.manga-ocr
python311Packages.manifest-ml
python311Packages.open-clip-torch
python311Packages.open-interpreter
python311Packages.openllm
python311Packages.optimum
python311Packages.outlines
python311Packages.peft
python311Packages.sentence-transformers
python311Packages.spacy-transformers
python311Packages.stanza
python311Packages.tensorflow-datasets

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/x21skvrdq49g5pwmnf1pgiinzl5awfr6-python3.11-datasets-2.19.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python311Packages.datasets https://github.com/r-ryantm/nixpkgs/archive/65e766bdd1e361bbe0c9a40d09ed947499212e8a.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/65e766bdd1e361bbe0c9a40d09ed947499212e8a#python311Packages.datasets

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/x21skvrdq49g5pwmnf1pgiinzl5awfr6-python3.11-datasets-2.19.1
ls -la /nix/store/x21skvrdq49g5pwmnf1pgiinzl5awfr6-python3.11-datasets-2.19.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
nixpkgs-review took longer than 120m and timed out

Maintainer pings

txtai fails on hydra, open-clip-torch freezes in pytestCheckPhase, and rclip depends on open-clip-torch