Book 1, Chapter 2, page 35 typo

In Section <IP_ADDRESS> "Independence of events",

We say that event $A$ is conditionally independent of event $B$ if $\Pr(A,B)=\Pr(A)\Pr(B)$

should be

We say that event $A$ is independent of event $B$ if $\Pr(A,B)=\Pr(A)\Pr(B)$

if I'm not mistaken.

oops! Fixed.


Add highest_score field to HackProjectScoreCategory model

S08 - Judging submissions
Score Categories can have different score ranges (e.g. 1-10, 1-15)
Adding this field sets the upper end of the scale, so the admin can define for each scoring category the scale by providing the highest possible score for that given category.
I'm just realising that this field was indeed on the original schema (with name score), but wasn't implemented in the models. In this respect it can be treated as a fix.
This might affect other user stories so I'm submitting this PR without any other changes to S08, so others can pull it and start working with this fix.
Tests:
Added the field to the test case but no assertion is added for it. I run the tests and all pass.
Know bugs/errors:
None
In my version, the field is named highest_score, but I can rename it, if you guys deem necessary.

Added min_score field with default=1 and renamed highest_score to max_score, added assertions for both fields to the tests and run them. All tests passing


feat: documents and evicence

PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines
[ ] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

Issue Number: #12
PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

What is the new behavior?
Added DocumentsAndEvidence controller and endpoints. Package 'controller' was renamed to 'controllers'

Other information (Optional)

PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines
[ ] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

Issue Number: #12
PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

What is the new behavior?
Now, the controller is complete. All packages and classes have been added, and the application is working.

Other information (Optional)

PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines
[ ] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

Issue Number: #12
PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

What is the new behavior?
Now the controller is complete, with all its endpoints. All packages and classes have been added, the application is working, and all conflicts have been resolved.

Other information (Optional)


Metrics for GRPC server

Reason for This PR
[Author TODO: add issue # or explain reasoning.]
Description of Changes
Follow up PR for changes in https://github.com/roadrunner-server/grpc/pull/62
In this PR I added missing metrics requests_queue, request_total and request_duration_seconds (The same exists for HTTP plugin).

@rustatian I added replace to have possibility to test it, feel free to remove it afterwards.
License Acceptance
By submitting this pull request, I confirm that my contribution is made under
the terms of the MIT license.
PR Checklist
[Author TODO: Meet these criteria.]
[Reviewer TODO: Verify that these criteria are met. Request changes if not]

[x] All commits in this PR are signed (git commit -s).
[x] The reason for this PR is clearly provided (issue no. or explanation).
[x] The description of changes is clear and encompassing.
[ ] Any required documentation changes (code and docs) are included in this PR.
[x] Any user-facing changes are mentioned in CHANGELOG.md.
[x] All added/changed functionality is tested.

@rustatian version updated


Isn't there a better way to notify pulls about data arrival?

Currently I call tick() at the end of every write and in the middle of every push. This tick method contains the following code:
        tick: function () {
            while (this.callbackQueue.length && this.dataQueue.length) {
                var callback = this.callbackQueue.shift();
                var data = this.dataQueue.shift();
                callback(data);
            }
            if (this.isDry())
                this.emit("dry", this);
        }

The problem is by the last write.
Currently I have 2 workarounds:
adding a write callback:
            var pointer = 0;
            this.on("pulled", function (flow) {
                setTimeout(function () {
                    if (pointer < items.length)
                        flow.write(items[pointer++]);
                }, 1);
            });
            this.on("tick", function (flow) {
                if (pointer == items.length)
                    flow.exhaust();
            });

adding a before tick event
            var pointer = 0;
            this.on("pulled", function (flow) {
                setTimeout(function () {
                    if (pointer < items.length)
                        flow.write(items[pointer++]);
                }, 1);
            });
            this.on("tick", function (flow) {
                if (pointer == items.length)
                    flow.exhaust();
            });

I like neither of them. Another possible solution to defer tick, so I will be able to exhaust() before ticking.
            var pointer = 0;
            this.on("pulled", function (flow) {
                setTimeout(function () {
                    flow.write(items[pointer]);
                    if (++pointer == items.length)
                        flow.exhaust();
                }, 1);
            });

So the real problem here that one part of the code is sync, while the other part of the code is async.

I need to separate the sync code from the async code. I'll use 2 classes:

var source = new DataSource();
source.write(data);
var data = source.read();
source.isDry();
source.isExhausted();
source.exhaust();

var flow = new DataFlow();

flow.push(function (source) {
    source.write(1);
    source.write(2);
    source.exhaust();
});

// after push, before tick, permanent
flow.on("pushed", function (source) {
    console.log(source.read());
});

// on tick after the source was written, after the pushed event
flow.await(function (source) {
    console.log(source.read());
});

// on tick after the source was written
flow.pull(function (source) {
    console.log(source.read());
});

// after pull, before tick, permanent
flow.on("pulled", function (source) {
    source.write(1);
    source.write(2);
    source.exhaust();
});

After every flow callback there is a tick. Not sure how to implement this by the event handlers, but it is easy by the others.

With the new design:
var ArrayReader = DataFlow.extend({
    prototype: {
        init: function (items) {
            DataFlow.prototype.init.call(this);
            if (!items.length) {
                this.source.exhaust();
                return;
            }
            var flow = this;
            var pointer = 0;
            this.on("pulled", function (source) {
                setTimeout(function () {
                    source.write(items[pointer]);
                    if (++pointer == items.length)
                        source.exhaust();
                    flow.tick();
                }, 1);
            });
        }
    }
});

var reader = new ArrayReader([1, 2, 3]);

reader.pull(function pump(source) {
    display.data(source.read());
    if (!source.isExhausted())
        reader.pull(pump);
});

I still need to call the tick manually, or another option to wrap the timeout callback with an auto tick caller.

It would be better to use a different branch for this, but I decided to use this implementation instead of the current one. I think this will be much better.

Actually we can use the same by other events, so we can put them into transactions.

Should a rewrite unlock a data source from exhaustion? That will be another question.

The new design works properly. I'll keep writing more advanced examples.


error: git cmd = '/snap/microk8s/4175/usr/bin/git fetch --depth=1 origin master': exit status 128

$ k version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24+", GitVersion:"v1.24.8-2+1dda18a15eea38", GitCommit:"1dda18a15eea38645814deffa547fe1c30141ef2", GitTreeState:"clean", BuildDate:"2022-11-10T18:01:19Z", GoVersion:"go1.18.8", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24+", GitVersion:"v1.24.8-2+1dda18a15eea38", GitCommit:"1dda18a15eea38645814deffa547fe1c30141ef2", GitTreeState:"clean", BuildDate:"2022-11-10T17:58:39Z", GoVersion:"go1.18.8", Compiler:"gc", Platform:"linux/amd64"}
$ k apply -k "github.com/kubernetes-sigs/aws-ebs-csi-driver/deploy/kubernetes/overlays/stable/?ref=master"
error: git cmd = '/snap/microk8s/4175/usr/bin/git fetch --depth=1 origin master': exit status 128

Summary

What Should Happen Instead?

Reproduction Steps

...
...

Introspection Report

Can you suggest a fix?

Are you interested in contributing with a fix?

I'm unable to reproduce this on either 1.25 or 1.24:
# 1.25
ubuntu@uk8s:~$ sudo microk8s kubectl apply -k "github.com/kubernetes-sigs/aws-ebs-csi-driver/deploy/kubernetes/overlays/stable/?ref=master"
serviceaccount/ebs-csi-controller-sa created
serviceaccount/ebs-csi-node-sa created
clusterrole.rbac.authorization.k8s.io/ebs-csi-node-role created
clusterrole.rbac.authorization.k8s.io/ebs-external-attacher-role created
clusterrole.rbac.authorization.k8s.io/ebs-external-provisioner-role created
clusterrole.rbac.authorization.k8s.io/ebs-external-resizer-role created
clusterrole.rbac.authorization.k8s.io/ebs-external-snapshotter-role created
clusterrolebinding.rbac.authorization.k8s.io/ebs-csi-attacher-binding created
clusterrolebinding.rbac.authorization.k8s.io/ebs-csi-node-getter-binding created
clusterrolebinding.rbac.authorization.k8s.io/ebs-csi-provisioner-binding created
clusterrolebinding.rbac.authorization.k8s.io/ebs-csi-resizer-binding created
clusterrolebinding.rbac.authorization.k8s.io/ebs-csi-snapshotter-binding created
deployment.apps/ebs-csi-controller created
poddisruptionbudget.policy/ebs-csi-controller created
daemonset.apps/ebs-csi-node created
csidriver.storage.k8s.io/ebs.csi.aws.com created

And on 1.24:
ubuntu@uk8s:~$ microk8s kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24+", GitVersion:"v1.24.8-2+1dda18a15eea38", GitCommit:"1dda18a15eea38645814deffa547fe1c30141ef2", GitTreeState:"clean", BuildDate:"2022-11-10T18:01:19Z", GoVersion:"go1.18.8", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24+", GitVersion:"v1.24.8-2+1dda18a15eea38", GitCommit:"1dda18a15eea38645814deffa547fe1c30141ef2", GitTreeState:"clean", BuildDate:"2022-11-10T17:58:39Z", GoVersion:"go1.18.8", Compiler:"gc", Platform:"linux/amd64"}
ubuntu@uk8s:~$ microk8s kubectl apply -k "github.com/kubernetes-sigs/aws-ebs-csi-driver/deploy/kubernetes/overlays/stable/?ref=master"
serviceaccount/ebs-csi-controller-sa unchanged
serviceaccount/ebs-csi-node-sa unchanged
clusterrole.rbac.authorization.k8s.io/ebs-csi-node-role unchanged
clusterrole.rbac.authorization.k8s.io/ebs-external-attacher-role unchanged
clusterrole.rbac.authorization.k8s.io/ebs-external-provisioner-role unchanged
clusterrole.rbac.authorization.k8s.io/ebs-external-resizer-role unchanged
clusterrole.rbac.authorization.k8s.io/ebs-external-snapshotter-role unchanged
clusterrolebinding.rbac.authorization.k8s.io/ebs-csi-attacher-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/ebs-csi-node-getter-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/ebs-csi-provisioner-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/ebs-csi-resizer-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/ebs-csi-snapshotter-binding unchanged
deployment.apps/ebs-csi-controller configured
poddisruptionbudget.policy/ebs-csi-controller configured
daemonset.apps/ebs-csi-node configured
csidriver.storage.k8s.io/ebs.csi.aws.com unchanged

Could this have been a temporary outage error? Can you try again and see if the issue persists?

I decided to apply -k from my local cloned repo.

I decided to apply -k from my local cloned repo.

That is a proper workaround. Are you still having troubles with the original command?

No

Thanks, closing this. Please re-open the issue if it occurs again.

microk8s.kubectl apply -k "github.com/kubernetes-sigs/aws-efs-csi-driver/deploy/kubernetes/overlays/stable/?ref=release-1.4.6" > private-ecr-driver.yml
/open
/reopen

@khteh Are you sure the release-1.4.6 reference is correct?
$ git clone https://github.com/kubernetes-sigs/aws-efs-csi-driver -b release-1.4.6
Cloning into 'aws-efs-csi-driver'...
fatal: Remote branch release-1.4.6 not found in upstream origin

Perhaps v1.4.6 or release-1.4 is what you are after?

i'm having a similar issue...
# kubectl apply --server-side -k "https://github.com/piraeusdatastore/piraeus-operator//config/default?ref=v2.3.0"
error: git cmd = '/snap/microk8s/6236/usr/bin/git fetch --depth=1 origin v2.3.0': exit status 128

# microk8s version
MicroK8s v1.26.11 revision 6236

@zeph see https://github.com/canonical/microk8s/issues/3988#issuecomment-1630330167


update 1.2.1 LED matrix is flashing

Hi, I have tried the update 1.2.1, but the LED matrix is flashing. there is a solution?

Is data connected to GPIO2? Are you using ESP8266/ESP32?

yes. i use data line connected to rx/gpio3. i used esp8266.


Use a general Eval function for expressions in templates.

One of the proposals in #23328
This PR introduces a simple expression calculator (templates/eval/eval.go), it can do basic expression calculations.
Many untested template helper functions like Mul Add can be replaced by this new approach.
Then these Add / Mul / percentage / Subtract could all use this Eval.
And it provides enhancements for Golang templates, and improves readability.
Some examples:

Before: {{Add (Mul $glyph.Row 12) 12}}
After: {{Eval $glyph.Row "*" 12 "+" 12}}

Before: {{if lt (Add $i 1) (len $.Topics)}}
After: {{if Eval $i "+" 1 "<" (len $.Topics)}}

Codecov Report

Merging #23927 (5c74fc6) into main (f521e88) will decrease coverage by 0.16%.
The diff coverage is 39.32%.

:exclamation: Current head 5c74fc6 differs from pull request most recent head c94f64a. Consider uploading reports for the commit c94f64a to get more accurate results

@@            Coverage Diff             @@
##             main   #23927      +/-   ##
==========================================
- Coverage   47.14%   46.98%   -0.16%
==========================================
  Files        1149     1161      +12
  Lines      151446   153689    +2243
==========================================
+ Hits        71397    72209     +812
- Misses      71611    72962    +1351
- Partials     8438     8518      +80

Impacted Files
Coverage Δ

cmd/dump.go
0.66% <0.00%> (-0.01%)
:arrow_down:

cmd/mailer.go
0.00% <0.00%> (ø)

cmd/manager.go
0.00% <0.00%> (ø)

cmd/manager_logging.go
0.00% <0.00%> (ø)

cmd/migrate_storage.go
5.76% <0.00%> (-0.12%)
:arrow_down:

cmd/restore_repo.go
0.00% <0.00%> (ø)

cmd/web.go
0.00% <0.00%> (ø)

models/actions/run.go
1.63% <0.00%> (-0.10%)
:arrow_down:

models/actions/runner.go
1.44% <ø> (ø)

models/packages/package.go
45.45% <0.00%> (-1.13%)
:arrow_down:

... and 74 more

... and 85 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Add code to AAD provider and REGO for 7.6 through 7.9 to support PIM for Groups

🗣 Description
Updated AAD Provider to add unified rules for PIM Groups associated with Privileged roles.  Updated AAD 7.6 - 7.9 to account for multiple rules (from directory role and from PIM Groups).
closes #919
TODO: #944  - Need to add new functional test cases.
💭 Motivation and context
PIM Groups provide mechanism to give users privileged access.  This enhancement allows to make sure notifications/alerts are produced appropriately.
🧪 Testing

Updated and ran REGO Unit test
Ran ScubaGear on tenant with PIM Group - manually verified results.

✅ Pre-approval checklist

[X] This PR has an informative and human-readable title.
[X] PR targets the correct parent branch (e.g., main or release-name) for merge.
[X] Changes are limited to a single goal - eschew scope creep!
[X] Changes are sized such that they do not touch excessive number of files.
[X] All future TODOs are captured in issues, which are referenced in code comments.
[X] These code changes follow the ScubaGear content style guide.
[X] Related issues these changes resolve are linked preferably via closing keywords.
[X] All relevant type-of-change labels added.
[X] All relevant project fields are set.
[X] All relevant repo and/or project documentation updated to reflect these changes.
[X] Unit tests added/updated to cover PowerShell and Rego changes.
[X] Functional tests added/updated to cover PowerShell and Rego changes. Except covered in #944
[X] All relevant functional tests passed.
[X] All automated checks (e.g., linting, static analysis, unit/smoke tests) passed.

✅ Pre-merge checklist

[ ] PR passed smoke test check.

[ ] Feature branch has been rebased against changes from parent branch, as needed
Use Rebase branch button below or use this reference to rebase from the command line.

[ ] Resolved all merge conflicts on branch

[ ] Notified merge coordinator that PR is ready for merge via comment mention

✅ Post-merge checklist

[ ] Feature branch deleted after merge to clean up repository.
[ ] Verified that all checks pass on parent branch (e.g., main or release-name) after merge.

Testing Strategy
To test this PR, I'm using the following setup and expected result scenarios to exercise the potential permutations of compliant and non-compliant states. Each scenario assumes that a specific PIM group used for the test is actively assigned to one or more highly privileged roles - in the case of 7.6 the group is assigned to Global Admin.
Scenario 1 Pass - Both the role configurations and the PIM group configurations are compliant with the policy.
Scenario 2 Fail - The role configuration is compliant but the group configuration is non-compliant.
Scenario 3 Fail - The group configuration is compliant but the role configuration is non-compliant
Scenario 4 Fail - Both the role and the group configurations are non-compliant.

Test Results

Policy
Test results

7.6 Activation of the Global Administrator role SHALL require approval.
Tested all four scenarios and the report displayed the correct output.

7.7 Eligible and Active highly privileged role assignments SHALL trigger an alert.
col2

7.8 User activation of the Global Administrator role SHALL trigger an alert.
col2

7.9 User activation of other highly privileged roles SHOULD trigger an alert.
col2

Regression Testing Results
Modified the test configurations for policies 7.4 and 7.5.
Ran ScubaGear version 1.1.1 versus the code in this PR to compare the outputs for those policies.
Policies 7.4 and 7.5 produce the same output for both versions. This matches what I expected.

Code review of JSON
I reviewed the new provider JSON structure and observed that now the PIM group configurations were added to the respective roles where groups are assigned. The new fields Rule.RuleSource, Rule.RuleSourceType were added and referenced in the Rego. No questions about these specific updates.

Ran on G3 tenant and got the warning to terminal: "WARNING: Omitting calls to Get-PrivilegedRole and Get-PrivilegedUser."
The report showed "Error" in the Result column of the AAD-7 group policies. Wondering if there is a missing check to for tenants without the PIM capabilities.

can you post a screenshot? i will take a look
sure, here is a screenshot for G3:

Shanti - Crutch is looking into why the code is giving errors for G3.

Update the message for 7.6 to say "0 role(s) or group(s) allowing activation without approval found" without the bold.

Inserted activation in details

Ran on G3 tenant and got the warning to terminal: "WARNING: Omitting calls to Get-PrivilegedRole and Get-PrivilegedUser."
The report showed "Error" in the Result column of the AAD-7 group policies. Wondering if there is a missing check to for tenants without the PIM capabilities.

can you post a screenshot? i will take a look
sure, here is a screenshot for G3:

Shanti - Crutch is looking into why the code is giving errors for G3.

Issue is directly related to expired license.  When retrieving Service Plans we check for provisioning status of 'Success'. In our case it is 'Disabled' since the license has expired.  Question is how to handle as was ask in another thread.  I believe this is out of scope of this issue.

Ran on G3 tenant and got the warning to terminal: "WARNING: Omitting calls to Get-PrivilegedRole and Get-PrivilegedUser."
The report showed "Error" in the Result column of the AAD-7 group policies. Wondering if there is a missing check to for tenants without the PIM capabilities.

can you post a screenshot? i will take a look
sure, here is a screenshot for G3:

Shanti - Crutch is looking into why the code is giving errors for G3.

This is directly related to expired license.  We look for service plans with a provisioning status of 'Success'.  In this case it is 'Disabled'.  The code then indicate getting privileged roles & users will be omitted.  This begs the previously ask question, do we alert the user that the service plans are disabled?  I believe resolution is out of scope of this issue.  WIll open another issue to track.

@tkol2022  Added code to cache check for PIM Groups Looks like a good number of hits. With a large number of assignments the hit/miss ration would just get better.

@tkol2022 I fixed IsPimGroup per our discussion

@tkol2022 Update caching to use privileged users functions determination of an Id as a user (i.e. non-group) to reduce future API calls in privileged role, in particular PIM Groups,  Also cleared cache after processing so would be clean if executed again.

This is blocked waiting the reactivation of the G3 tenant.

@nanda-katikaneni ready for merge


story location, mentions, links(swipe up) and hashtags not showing on instagram stories

story metadata (location, mentions, links and hashtags) not showing on instagram after successful upload
from instagrapi.story import StoryBuilder, StoryMention
from pathlib import Path
from instagrapi import Client
from instagrapi.types import Location, StoryMention, StoryLocation, StoryLink, StoryHashtag

cl = Client()
cl.login(username, password)

user = cl.user_info_by_username('obaapa2087')

buildout = StoryBuilder(
    'alonso.jpg',
    'Credits @obaapa2087',
    [StoryMention(user=user, x=0.49892962, y=0.703125, width=0.8333333333333334, height=0.125)]
)

loc = cl.location_complete(Location(name='Test', lat=42.0, lng=42.0))
ht = cl.hashtag_info('dhbastards')

upload = cl.photo_upload_to_story(
    buildout.path,
    "Credits @obaapa2087",
    mentions=buildout.mentions,
    locations=[StoryLocation(location=loc, x=0.33, y=0.22, width=0.4, height=0.7)],
    links=[StoryLink(webUri='https://github.com/adw0rd/instagrapi')],
    hashtags=[StoryHashtag(hashtag=ht, x=0.23, y=0.32, width=0.5, height=0.22)],
)

print(upload)

instagrapi: latest version

@alosoft You don't understand what a story is, it consists of two parts:

Video/photo editor in the Instagram application
Markup story (events and coordinates: hashtags, location, stickers, mentions, etc)

1 - you have to do it yourself, StoryBuilder can help you in part
2 - photo / video_upload_to_story is provided markup for you story

@alosoft links works when you account have more 10k followers (instagram limitation)


Decide on mentor engagement and involvement

Option under consideration:

Require Project teams to field up to 10 mentors to support their projects, along with up to five breakout room hosts.

https://docs.google.com/document/d/172hZ0BbsTsKvNP7GNxMZ_ihPsHOB_OkUHFc66tOUINw/edit?usp=sharing

Project teams will have the opportunity to bring up to 15 mentors to support the education and collaboration from attendees during OSD.


refactor: close prompt after succesful payment

This fixes: #217
For now I called the action weblnPay.
What about when a user has set an allowance. Should the extension show some loading state in a different way, or is it up to the publisher?

What about when a user has set an allowance. Should the extension show some loading state in a different way, or is it up to the publisher?

I would say it is up to the publisher but maybe we can indicate that with some icon state (color, or a different character then)

Something is still broken with the events. The payments do not get persisted.
this is because of:
https://github.com/getAlby/lightning-browser-extension/pull/253/files#r727028434

What do you think about moving the new sendPaymentOrPrompt.js to a new actions/webln folder? Then the ln folder contains all actions that directly talk to the lightning connector and these more "highlevel" actions get their own folder?

I think it's a good idea to make this distinction.


ENSCORESW-2742: keep existing descriptions. Only for merged species

Requirements

Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
Review the contributing guidelines for this repository; remember in particular:

do not modify code without testing for regression
provide simple unit tests to test the changes
if you change the schema you must patch the test databases as well, see Updating the schema
the PR must not fail unit testing

Description
Currently, the code deletes all gene descriptions in the core database prior to loading the xrefs, then writes new descriptions based on the xref mappings. The change proposed leaves existing descriptions in place but overwrites them with descriptions assigned by the xref mapping.
Use case
Gene descriptions are set by the xref pipeline. For species with manual annotation, we use the description from the xref chosen as the display xref. Not all genes will get a valid display_xref (the clone name is used in that case), and not all genes with a valid display_xref have a useful description. By leaving the existing descriptions in, we increase the percentage of genes with a useful description.
For example, see http://e92.ensembl.org/Homo_sapiens/Gene/Summary?g=ENSG00000283537;r=7:143620943-143645675 which does not have any description. With the proposed change, this gene would get a meaningful description from Havana, ie 'novel TRPM8 channel-associated factor family pseudogene'
Benefits
Increased number of genes with descriptions.
The change only applies to species with manual annotation, as it is in the set_display_xrefs_from_stable_table method which is only used for human, mouse, rat, zebrafish and pig. By default, species use the set_display_xrefs method.
Possible Drawbacks
Some descriptions from Havana might not be meaningful ('novel transcript'). If old descriptions are kept in the database, they will not be cleaned up by the new xref update.
Testing
Have you added/modified unit tests to test the changes?
NA
If so, do the tests pass/fail?
NA
Have you run the entire test suite and no regression was detected?
NA

Coverage increased (+0.01%) to 82.45% when pulling ac34a0e403453375611c78a248d45043661720e0 on feature/havana_desc into 15a799e60a996cf5e1f730ad31a9db5034813ce9 on master.

Build failure is because one test is accessing the public Ensembl mysql, which was overloaded at that moment. Otherwise it seem reasonable. I have no idea how we'd remove stale items after...years...but hopefully more blanks will be filled in in the sources to overwrite them.

There we go, build passing now.


add filter to list items which related given tenant

just adding tenant-id on listing  group-policy-rules & ruls-sets, classifiers, actions and l2/l2 policy

Thank you for contributing to stackforge/group-based-policy-ui!
stackforge/group-based-policy-ui uses Gerrit for code review.
Please visit
http://docs.openstack.org/infra/manual/developers.html#development-workflow
and follow the instructions there to upload your change to Gerrit.


Save sorting options

We should save the options users select for sorting their lists. This should be on a per-game basis.

Done.


Changes to certain GTFS features

After a final round of reviews, we have made the following changes to the GTFS feature mapping:

move stop_times.shape_dist_traveled to the Shapes component. Even if stop_times.shape_dist_traveled can be used without Shapes (https://github.com/google/transit/issues/394), it seems like it's the most common use case.
Add documentation links in HTML report per feature as defined in features.md

Acknowledging that this has already been added to the validator and might not be part of the next release.
Current validator implementation here: https://github.com/MobilityData/gtfs-validator/blob/master/docs/FEATURES.md

As part of this issue, we should also link each individual feature in the HTML report to its associated feature heading in https://gtfs.org/getting_started/features/overview/. In the tooltip summary it should link to https://gtfs.org/getting_started/features/overview/

I created a summary table with all the changes needed based on this version of the features list vs what we currently implemented in GTFS.org
For further reference check GTFS.org's features page and this list

Feature group (old)
Feature name
Changes

Accessibility
Text-to-speech
N/A

Accessibility
Wheelchair accessibility
Split into two (also split corresponding rules):"Stops Wheelchair Accessibility" & "Trips Wheelchair Accessibility"

Accessibility
Route Colors
Classify under new group: Base add-ons

Accessibility
Bike Allowed
Classify under new group: Base add-ons

Accessibility
Translations
Classify under new group: Base add-ons

Accessibility
Headsigns
Classify under new group: Base add-ons

Fares
Fare Products
N/A

Fares
Fare Media
N/A

Fares
Route-Based Fares
N/A

Fares
Time-Based Fares
N/A

Fares
Zone-Based Fares
N/A

Fares
Transfer Fares
Renamed: “Fares Transfers”

Fares
Fares V1
N/A

Pathways
Pathways (basic)*
Rename: “Pathway Connections”

Pathways
Pathways (extra)*
Rename: “Pathway Details”

Pathways
Levels
N/A

Pathways
In-station traversal time
N/A

Pathways
Pathways directions
Rename: “Pathway Signs”

Pathways
Location types
Classify under new group: Base add-ons

Metadata
Feed Information
Classify under new group: Base add-ons

Metadata
Attributions
Classify under new group: Base add-ons

Flexible Services
Continuous Stops
N/A

Shapes
Shapes
Classified under new group: Base add-ons

Transfers
Transfers
Classified under new group: Base add-ons

Frequency-based Services
Frequencies
Classified under new group: Base add-ons

Finally, just for visibility there are still 4 Features pending to be added as part of the Flexible services group:

Booking Rules
Predefined Routes with Deviation
Zone-based Demand Responsive Services
Fixed-Stops Demand Responsive Services

Tasks:

[ ] review the feature names table and apply the changes
[ ] add tests
[ ] ask spec team members to review the PR

Added documentation links to HTML report as a task for this issue


bug(CDK scrolling): Inconsistent scrolling with the mouse wheel

Reproduction
Just look at the examples in different browsers: https://material.angular.io/cdk/scrolling/overview#virtual-scrolling
Steps to reproduce:
scroll with the mouse wheel in the examples

in Chrome/Opera it jumps from Item 3 at the bottom to Item 7 at the top, never displaying items 4 to 6.
in Edge(Chromium) it just scrolls 2 items.
only in FF it scrolls as expected from Item 3 at the bottom to Item 4 at the top.

Expected Behavior
Scroll behavior as in FF.
Environment

Angular:
CDK/Material: 10.2.0 , 9.2.4
Browser(s): Chrome, Opera, Edge(Chromium)
Operating System: Windows 10

This sounds like probably just differences in how browsers handle mouse wheel scrolling. Normalizing this behavior across browsers is outside the scope of Angular Material.


conda package

Hi, I have created a recipe for a conda package of fmjpeg2koj. Do you plan to create a Github tag so that we can refer to that tag to build the package?
Here is the recipe PR (https://github.com/conda-forge/staged-recipes/pull/22639)

Like a version tag?  Will 'v1.0.3' work for the current master?

Yes, that should work.

Thanks for creating the tag. I have updated my conda PR with the new tag. Hopefully this will get into the conda-forge package list within a week or two.


Correct link to Apache Camel in README.md

Now http://apache.camel.org/, should be http://camel.apache.org/

Ah well spotted, you are welcome to work on a PR :+1:

Thanks Dmitry! Fixed.


API Version 2019-07 pagination

Hi,
The 2019-07 API version changed the pagination for few resources as described here.
The Product is one of the resources impacted.
I don't see any Breaking change notice, but I think it should be, because the 'page' parameter no longer works and returns an error.
Also, it would be nice to know if you plan to release API support for this feature.
Thank you

Hi,
this feature has already been implemented and merged into master with #594
Great work @garethson  👍
Why is the cursor based pagination targeted to the unstable API version?
As already mentioned some endpoints support this pagination already since 2019-07.
It would be great to to get this feature into a release soon.

@jiri-hacmac Thanks! I've since updated the readme (see https://github.com/Shopify/shopify_api/pull/599) to denote that relative cursor navigation is now available on all endpoints using the 2019-10 and later API versions.
Since only some of the endpoints were available in Shopify on 2019-07, we didn't want to add partial support in this library, but now that every endpoint is available, you can simply use 2019-10 (or unstable, which is not recommended for production use).

Closing this issue, but feel free to drop in any questions regarding this library's support for this type pagination here.


Smart Home Intents mixed with other Intents

This is more of a question, but I was thinking of creating an app that could service Smart Home Intents (/ha) and also Google Action intents (/ga).  (https://github.com/futbolpal/harmony-home).
I  have a home-automation branch that adds Smart Home functionality.
This way I could use the smart home style (which is less verbose, no "tell to so-and-so to do X"), but also support things that aren't supported by smart home (since there is only two schemas).
In the repo, I'm using some scripts to generate action.json based on environment variables (from Heroku for example).  I was hoping to use one action.json with intents for action.intents.devices and others, however, it appears that a smart home intent must be the only intent in the action.json.
Any thoughts on this?  It's kind of a bummer, but at the minute, I could basically create two apps and two sets of action.js files that I could push out in a deploy pipeline.

Yeah for right now Smart Home and AoG projects need to be separated.

Thanks :/


Replace Server-side events with Server-sent events in documentation

The acronym SSE actually means "Server-sent events" (see https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events)
To help everyone out, please make sure your PR does the following:

[ ] Update the first line to point to the ticket that this PR fixes
[x] Add a message that clearly describes the fix
[x] If applicable, add a test that would fail without this fix
[ ] Make sure the unit and integration tests pass locally with pnpm run tests and cd integration && pnpm run tests
[x] Includes a changeset if your fix affects the user with pnpm changeset

nice catch! thanks


Google Maps ContentProvider

For issue #273
https://github.com/davidfowl/JabbR/issues/273

Are you using tabs or spaces? Look at the guidelines on the readme

And I just jumped online to make this happen.


Document integration with s-c-netflix-{ribbon,zuul}

documents？

@Keets2016 not done yet, that's why this issue is still open.

+1


Expose texture IDs as an escape-hatch for non-standard texture sources

This will allow binding of non-standard texture sources (e.g. HtmlVideoElement, HtmlImageElement, and VideoFrame on web) by calling the lower-level context operations outside of the texture struct's implementation, but keeping the management of textures consistent with the mid-level API.

@seancribbs I had a hard time figuring out what would be the nicest approach, but ended on the change in #388. Then the texture needs to be created manually by raw context calls and then it can be used in a program using the proposed method. What do you think?


Add libheap GDB library

I needed to build GDB with the python from the ctftools virtualenv, otherwise the libheap is not available... Is it a problem?
Also, travis is failing on my tests, but I think it's still because of pyvmmonitor's link that is not reliable at all :(

I actually like it better that way -- easier to install python plugins for it (such as in this case).
Would you mind checking if peda and gef still work after this change? If so, I'm all for it!

Peda works... I can't really test gef because GDB segfaults when I try to load it, but i think it's a problem in gdb/gef more than their installations

Sweet. Just to be sure, let's ping @f0rki. He added the gef support (and, now that I think about it, the pwndbg support), and I'd hate for him to be mad at us for breaking it :-)
Unless f0rki points out some fundamental issue with the gdb stuff related to this change, I'll merge it in soon. Thanks for the contribution!

It really is far nicer this way :) pwndbg install is currently broken anyway because it needs python dependencies, which are searched in the system python paths. This way we can just install them in the virtualenv.

Yeah it's probably better to use ctf-tools-venv-activate. I didn't know about it because it's a recent (nice) addition ;)

Yep, I agree on all counts. I'll go ahead and fix it up after merging; I have a few minutes of free time and want to play around with libheap ;-)

Ops, thanks a lot!

I just pushed the changes we discussed!
Thank you for the contribution :-)


Fixing the padding when using collapse button

@jdayssol, your change is causing the body of the collapsible to shift on expanding/collapsing the sidebar. The body should not change, i.e., remain stagnant on expansion/collapse of the sidebar.

If the body don t change when expanding, then some elements of the body may become then hidden by the expanded menu. That the problem I wanted to solve.


Updated summary spec, made all required

[Current coverage][cc-pull] is 79.61%

Merging [#223][cc-pull] into [develop][cc-base-branch] will not change coverage

@@            develop       #223   diff @@
==========================================
  Files            24         24
  Lines          3566       3566
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits           2839       2839
  Misses          727        727
  Partials          0          0

Powered by Codecov. Last updated by [f421cae...7f3a418][cc-compare]
[cc-base-branch]: https://codecov.io/gh/kbase/data_api/branch/develop?src=pr
[cc-compare]: https://codecov.io/gh/kbase/data_api/compare/f421cae3c52f2b7bc64e57432dbc87254e6f92ff...7f3a418a00a1453e557e2b03c73d8825600c9574
[cc-pull]: https://codecov.io/gh/kbase/data_api/pull/223?src=pr


drop dead/unused symlink

/assign @carlisia

/lgtm
/approve

/retest
https://github.com/knative-sandbox/net-contour/pull/710, which should fix the CI issue, was merged just now.


Why there is no change in status of my PR

My PR: https://github.com/algorandfoundation/xGov/pull/55
Has been there for weeks and there is no change and no merger! May I ask for the reasons?

It might be better to comment on your PR and ask for clarifications there @emg110 instead of opening an issue :)
Nothing happens in between voting sessions, unless you actively engage the community, sharing your link and asking for feedback.

Thank you so much, this was an old one I forgot to close in fact! all good!


⚠ Fetchers has degraded performance

In 1a4d93b, Fetchers ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 356 ms

Resolved: Fetchers performance has improved in 0f4f7f5.


Git commit message as oneline

Thanks for the great GitHub Action! I was able to get it up and running immediately on the first try!
From the docs I know that we can get the git commit message as ${{ github.event.head_commit.message }}, but I was wondering if there's a way that we can get just the first line of the commit message?
Also, as an FYI, the head_commit.message property only exists on some workflow payloads. You might have to access it differently depending on how the workflow is triggered. Might be worth adding at link to the Workflow Payload doc in the README?

@ndench These are some of the functions that are available in GitHub Actions but I don't think there is one that can help you solve your problem.
You can write a bash script to get the first line of the commit message ${{ github.event.head_commit.message }}, store it in a variable and then reference it in the notify-slack action message.

Great, thanks for the tip @ravgeetdhillon. For anyone else looking for this, I've managed to get it working with the following.
NOTE: I'm triggering the workflow using the workflow_run event, not pull_request.
            - name: Get commit message
              if: always()
              id: commit
              run: echo "::set-output name=message::$(echo '${{ github.event.workflow_run.head_commit.message }}' | head -n 1)"
            - name: Send slack notification
              uses: ravsamhq/notify-slack-action@v2
              if: always()
              with:
                  message_format: '{emoji} ${{ steps.commit.message }}'
                  footer: "<{commit_url}|{repo}> | <{run_url}|{workflow}>"
                  token: ${{ secrets.GITHUB_TOKEN }}
              env:
                  SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

@ndench Thanks for adding your advice for other users.


See if the names in the map Legend change in Portuguese

For example, if (E)nemy is called Inimigo, the same letter must be used to represent it. In this way, the weight label must be rendered as Inimigo (E).

Yes, they do.


Android TextInput setting selection props beyond 0 on mount crash

When I set selection props on <TextInput /> that is above 0 (e.g. { start: 1, end: 1 }), it will crash at setSpan()…
e.g.
<TextInput multiline={true} selection={{ start: 1, end: 1 }}>
  <Text>abc</Text>
</TextInput>

It seems like the text are not set before selection is applied, thereby causing the crash. Crash log stack trace provided below.
Environment
Environment:
  OS: macOS High Sierra 10.13.3
  Node: 9.2.0
  Yarn: 1.3.2
  npm: 5.5.1
  Watchman: 4.9.0
  Xcode: Xcode 9.2 Build version 9C40b
  Android Studio: 3.0 AI-171.4443003

Packages: (wanted => installed)
  react: 16.2.0 => 16.2.0
  react-native: 0.53.2 => 0.53.2

Expected Behavior
I would expect it to not crash and set the selection correctly.
Actual Behavior
It crashes with the following stack trace…

    setSpan (1 ... 1) ends beyond length 0

Error while updating property 'selection' of a view managed by: AndroidTextInput

null

setSpan (1 ... 1) ends beyond length 0
updateViewProp
    ViewManagersPropertyCache.java:92
setProperty
    ViewManagerPropertyUpdater.java:129
updateProps
    ViewManagerPropertyUpdater.java:48
updateProperties
    ViewManager.java:34
createView
    NativeViewHierarchyManager.java:233
execute
    UIViewOperationQueue.java:153
dispatchPendingNonBatchedOperations
    UIViewOperationQueue.java:1010
doFrameGuarded
    UIViewOperationQueue.java:981
doFrame
    GuardedFrameCallback.java:31
doFrame
    ReactChoreographer.java:136
doFrame
    ChoreographerCompat.java:107
run
    Choreographer.java:856
doCallbacks
    Choreographer.java:670
doFrame
    Choreographer.java:603
run
    Choreographer.java:844
handleCallback
    Handler.java:739
dispatchMessage
    Handler.java:95
loop
    Looper.java:148
main
    ActivityThread.java:5417
invoke
    Method.java
run
    ZygoteInit.java:726
main
    ZygoteInit.java:616

Steps to Reproduce
https://snack.expo.io/@lxcid/textinput-selection-crash
Open the snack in Android and uncomment the line with selection={{ start: 1, end: 1 }}…
<TextInput
  // selection={{ start: 0, end: 0 }} // Uncomment and it does not crash
  // selection={{ start: 1, end: 1 }} // Uncomment and it crash in android
  style={styles.paragraph}
  multiline={true}>

Previous discussion: https://github.com/facebook/react-native/issues/17236

I have a workaround in https://github.com/akalin-keybase/rn-text-input-bugs , which is to wait for the next onSelectionChange event and set the selection there. Per the bug I linked above, this crash also happens if you set the text and selection at the same time, and the selection is out of bounds of the old text.
If the text box has focus, Android seems to fire onSelectionChange after the new text is applied, so that's why the workaround works. If the text box doesn't have focus (e.g., you're changing the text programmatically) an onSelectionChange isn't fired, but the selection isn't visible anyway, so it's safe to wait until the user focuses (which triggers an onSelectionChange) to set the new selection.

Pretty sure this is still crashing.

+1

The root cause is due to setSelection and setText are handled in different mechanism. Seletion is by set property and Text is handled by extra data update. And extra data update will be hanled after all properties are set.

Closing, as #22723 was recently merged and will be available in 0.60 release. Thanks @shenyute!

still have same issue in react native 0.60.5.

Closing, as #22723 was recently merged and will be available in 0.60 release. Thanks @shenyute!

Still not working in 0.61.3 release. Any solution for same?

@thymikee This is still very much an issue in 0.61.5 release. The workaround at https://github.com/akalin-keybase/rn-text-input-bugs/blob/master/App.js also doesn't work.
We tried some workarounds with setNativeProps for text and selection, but still getting crashes.
It looks like if you set the selection prop, or call setNativeProps({selection: {...}) then you'll always get 2 onSelectionChanged events for the price of one when you type. The first with the accurate numbers then the second with the last values set with  selection or setNativeProps({selection: {...}). Causes cursor flicker/misplacement while typing, and crashes if you hit backspace.
Our use case is inserting emoji and mentions into a message input, while maintaining the correct selection.
I'll try to make a simple repro when I have a minute.
Arguably trying to set selection outside of the available range shouldn't crash the entire app anyway. Sure I can see arguments for both sides but IMO it should select from the start point to the end of the text if the end is out of range, or move the cursor to the end of the text if the entire selection is out of range.

Doesn't look finished but https://github.com/facebook/react-native/commit/7ab5eb4cafdea695c4c53ce2a737f6302afd6380 might solve this one day as it adds a setTextAndSelection method.

Repro: https://github.com/mjmasn/SelectionCrash

@russelRajitha @Anjy123 see https://github.com/mjmasn/SelectionCrash/pull/1 for a new workaround. Haven't noticed any issues with it yet but feel free to try it and let me know how it goes.

@russelRajitha @Anjy123 see mjmasn/SelectionCrash#1 for a new workaround. Haven't noticed any issues with it yet but feel free to try it and let me know how it goes.

Experiencing the same issues (only on Android) with a use case similar to yours, this fix did it for me. Thanks!


upgrades Julia 0.5.1 - Plotly 0.5.2

this will allow to use plotly 0.5.2
I branched off in our fork of PlotlyJS.jl
but I wouldn't merge that in master so that we can track updates and mirror
them in the Dockerfile (and hence the SHA) of this repo.
It seems we can only use checkout with a real branch and not a tag
with Julia package library.
what do you think, any thoughts on the naming convention?

ok, wait to merge this!
we have a better solution to track the tag/commit of our fork of the plotly js package,
in the dockerfile

yes we tried that with bertram, it seems it works like we did in the Dockerfile

ah no, ok you mean a commit instead of the tag : nj1-0.5.2 for instance? @mk

@zampino yes, a commit it immutable, so preferred to a tag or branch.


Handle EnvoyProxy Image version upgrades

thanks @cnvergence !
Thinking out loud, an outcome of this issue could be a E2E where all client requests are successful while 2 replicas of Envoy Proxy are undergoing a rolling restart.

@arkodg coming back to this after a while, could you please point me to where should I start?
I did check the upgrade and it seems like it is handled, but I may be wrong.
As for the E2E, I assume I should add a new scenario to the e2e test suite :)

hey I know @chauhanshubham was looking into a similar test for control plane upgrades which would invariably also upgrade envoy proxy, should we just collapse those two e2e tests into one where we perform an upgrade with a last known EG minor version, and ensure that

there is no config churn in the data plane during an upgrade
there is no traffic drop while this upgrade this happens (assuming we always have 2 replicas of control plane and data plane running)

I'm concerned that a hitless in-place upgrade of envoy is not trivial.
A graceful termination of envoy and may require:

Failing LB/Kubelet probes to stop new connection from being established to terminating pods
Triggering envoy to drain listeners
Delaying pod termination until all connections are terminated
Additional factors to consider:

IaaS LBs behave differently in response to targets becoming unhealthy (e.g. AWS will reset all existing connections, while GCP/Azure will stop establishing new connections but retain existing connections) and have varying levels of configurability for HCs.
The number of envoy pods per node and LB's ExternalTrafficPolicy impact the correctness of LB HCs.

It's also important to avoid race conditions where a new instance of envoy is receiving traffic before it was configured (e.g. due to order of component restart, failures in new control plane version, etc.).
Some prior art:

Contour: https://github.com/projectcontour/contour/blob/main/design/envoy-shutdown.md
Gloo Edge: https://docs.solo.io/gloo-edge/latest/operations/advanced/zero-downtime-gateway-rollout/

@arkodg
I executed a naive test:

Environment: kind, metallb, EG quickstart.yaml
envoy proxy replicas: 2
upgrade: 0.6.0 => 0.0.0-latest using helm upgrade
load simulation during upgrade: hey -c 100 -q 10 -z 300s -host www.example.com http://<IP_ADDRESS>/

The upgrade caused some client-facing failures during the test:
Error distribution:
  [8]	Get "http://<IP_ADDRESS>/": EOF
  [32]	Get "http://<IP_ADDRESS>/": dial tcp <IP_ADDRESS>:80: connect: connection refused
  [1]	Get "http://<IP_ADDRESS>/": read tcp <IP_ADDRESS>:55220-><IP_ADDRESS>:80: read: connection reset by peer
  [1]	Get "http://<IP_ADDRESS>/": read tcp <IP_ADDRESS>:55260-><IP_ADDRESS>:80: read: connection reset by peer

It's probably possible to tune some of the parameters mentioned in my previous comment to achieve a hitless upgrade under certain test conditions (RPS, connection reuse, HTTP version, ...). But, I'm not sure that we can claim to have a hitless upgrade in general, based on such test.
So, I propose that for the GA scope, we focus on an upgrade test that ensures request convergence to successful execution after the upgrade. A limited hitless upgrade test can be a stretch-goal.
In the future, we can explore:

Implementing a graceful envoy shutdown feature and providing guidance on configuring envoy for hitless in-place upgrades
Supporting canary deployments

WDYT?

hey @guydc I was hoping we could have some test for hitless upgrade in v1.0, with caveats, that can hopefully we removed over time post GA
do agree, we can split this up, and make it a stretch goal for v1.0

this should be fixed with #2633, keeping this open so that it can be validated with a e2e

fixed with https://github.com/envoyproxy/gateway/pull/2839


Add Apache Hive 3.1.2

Cherry pick of https://github.com/trinodb/docker-images/pull/79
Co-authored-by: Guy Cohen<EMAIL_ADDRESS>On top of the original PR from trino, additional changes are made to

Run SOCKS proxy
Add "hive.metastore.disallow.incompatible.col.type.changes" set to 'false' in hive-site.xml. This was changed from false to true from Hive versions < 3. It was causing AbstractTestHiveClient#testDropColumn to fail.
Added user hive. This is the user we run tests as.

@aweisberg : This PR contains changes to build Apache Hive 3.1.2 docker images. I get some errors while building images locally using the instructions in the README.md.  I'm using Docker For Mac 3.3.1 (Docker Engine V20.10.5). Specifically the errors relate to the "docker history" command run by bin/label-parent.sh which now doesn't get the actual IMAGE ID but an empty string.

For me it fails here https://pastebin.com/uAfuk4Bz
cd prestodb/centos7-oj8 && time /bin/sh -c "( tar -czh . | docker build   -t prestodb/centos7-oj8:latest --label com.teradata.git.hash=d5c4ce0c1ea22d21bdb6c48aa23cdae289c4ef6c - )"
[+] Building 0.1s (2/2) FINISHED
 => [internal] load remote build context                                                                                0.0s
 => ERROR copy /context /                                                                                0.1s
------
 > copy /context /:
------
failed to solve with frontend dockerfile.v0: failed to read dockerfile: Error processing tar file(gzip: invalid header):

I just have a question about the SSH daemon and some requests for cleanup and clarity.
Can you mention the PR description and second commit what the additional changes you are layering on top of the original PR are? Looks like you are trying to enable impersonation.

I just have a question about the SSH daemon and some requests for cleanup and clarity.
Can you mention the PR description and second commit what the additional changes you are layering on top of the original PR are? Looks like you are trying to enable impersonation.

The core-site.xml impersonation changes were a mistake, not required. I will remove them.

We should build and upload this before merging.

@highker can you merge this? The PR build doesn't work at all right now, but the image from this PR is built and uploaded.

@aweisberg, @nmahadevuni, so do we want a rebase and force push?

@aweisberg, @nmahadevuni, so do we want a rebase and force push?

No PRs were merged after I created this one. I don't think a rebase is required.


FEATURE - Refresh mockdata on preview if mockdata is changed

As a Fiori developer,
I want to see updated mockdata during preview without stopping/restarting the preview,
so that I can achieve my test results faster
Description
While a preview is running with mockdata and default configuration, changes to the mockdata are not reflected on preview reload until the mockserver is stopped and started again. The refresh function can be enabled with the parameter watch: true. The proposal is to enable this feature by default in the ui5-mock.yaml file.
Technical Design
Enhance ui5-mock.yaml template to include watch: true for all mocked services.
Acceptance Criteria
Given I have added mockserver support to my Fiori app
when I preview the app and later change the mockdata
then on reload of the preview the new mockdata is loaded.
Notes
@tobiasqueck @nlunets @zdravko-georgiev
Tasks

[ ]
[ ] Unit Tests added
[ ] Test Cases defined

i support this


Feat/multi orgs

This PR adds support for multiple orgs and scoped teams, along with some other minor improvements:

orgs can be provided as foo,bar,baz
teams must be provided with scoped orgs: foo:alpha,foo:beta,bar:beta,*:theta
the * wildcard can be used to say all orgs (and so the first org that the user has membership to will be used)
scopes can be provided from environment, as space-delimited values via GITHUB_SCOPES

@roleyfoley feel free to merge this once you've cut the right jobs across


Feature request: Lifecycle hooks

Is there an existing issue for this?

[X] I have searched the existing issues

Is your feature request related to a problem?
Jetbrains plugin not working for some time now. User @shochdoerfer suggested a passive approach not involving the IDE at all.
What if people could develop their own solutions to their problems, by hooking into the DDEV lifecycle, like only first-start / start / stop / delete? Does not matter if I need to do additional configuration, do an rsync, if a user can hide it in an executable, DDEV can stay clean.
For this problem, I could write a simple Go binary to take DDEV configuration values given to me and rebuild/reconfigure the .idea/ folder, if present, with the correct details, like new db connection details.
Describe your solution
Define additional nodes within .ddev/config.yaml like
hook:
  first-start:
   # option with all parameters
    - { name: full variant, os: linux, path: ./path-relative-to-dot-ddev/firstrun.sh }
   # option without name
    - { os: windows, path: ./path-relative-to-dot-ddev/firstrun.bat }
  start:
    # shortest variant, will execute on all OSes
    - ./start-vpn.sh
    - ./configure-ide.sh
  stop:
    - ./stop-vpn.sh
  delete:
    - ./delete-vcloud-instance.sh
    - ./stop-vpn.sh

os will compare against GOOS, and only execute when matching.
name can be defined by or omitted, not really important.
path will be relative to .ddev/

Every hook will do a OS execute on the given paths, injecting data available within ddev describe  as ENV variables to use within the script, within the order given. Failure to execute a hooked script will warn about it, but not further influence the DDEV flow.
Describe alternatives
I have no alternatives.
Additional context
I've search through other issues which like #5557 and #5448, which were on per-command and not that helpful. This is not on per command, this is about the container lifecycle which otherwise would need additional extra commands.

DDEV current has quite a few hooks, perhaps you missed the page - https://ddev.readthedocs.io/en/stable/users/configuration/hooks/
Perhaps you could a provide an specific example of others places you would like to see new hooks.

@tyler36 You are right, I've totally missed that. Seems to work, post-start in combination with parsed ddev describe -j could actually solve the problem, thanks very much!

@adrianrudnik
Glad you found what you needed and thank you for updating the issue.

Thanks again, got a basic go based CLI tool working that configures the PHPStorm DB config based on that. Hopefully the IDE plugin itself can sort out the publishing issues soon.


Security warning from cargo deny concerning tokio-rustls

Running a cargo deny check on a project containing reqwest results in:
cargo deny check
error[RUSTSEC-2020-0019]: tokio-rustls reads may cause excessive memory usage

     ┌── Cargo.lock:166:1 ───
     │
 166 │ tokio-rustls 0.13.0 registry+https://github.com/rust-lang/crates.io-index
     │ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ security vulnerability detected
     │
     = `tokio-rustls` does not call `process_new_packets` immediately after `read`,
       so the expected termination condition `wants_read` always returns true.
       As long as new incoming data arrives faster than it is processed
       and the reader does not return pending, data will be buffered.

       This may cause DoS.
     = URL: https://github.com/tokio-rs/tls/pull/14
     = tokio-rustls v0.13.0
       ├── hyper-rustls v0.20.0
       │   └── reqwest v0.10.6
       │       └── collector v0.1.3
       └── reqwest v0.10.6 (*)

2020-06-30 23:17:11 [ERROR] encountered 1 errors

It seems this has already been fixed upstream with https://github.com/tokio-rs/tls/pull/14

I believe this is fixed with #969


Support for stm32h725/735

Hi,
I'd like for a project to use the stm32h725. It has ECC RAM, something the other stm32h7 boards don't have. Well except for the stm32h747/757, but I'd rather not pay the premium for features I won't use anyway.
I noticed those boards aren't supported by this library.
Is it because nobody has the μControler to test? Or is it because of an incompatibility with the hardware?
If it's because of testing or minor incompatibility I'd be willing to help. I have never worked on a HAL like this though, so if it's ok with you I'd still want instructions about how to proceed.

I do not recall any reason for us to specifically exclude stm32h725/735
As to how you could proceed:
You have to make sure stm32h7 the PAC crate supports stm32h725, this does not seem to be the case, at least I cannot see it in this file: https://github.com/stm32-rs/stm32-rs-nightlies/blob/master/stm32h7/Cargo.toml (that is probably the reason we didn't include it yet since the PAC is really the basics of the basics required to get going) The ZIP archive (https://github.com/stm32-rs/stm32-rs/blob/master/svd/vendor/stm32h7_svd.zip) that contains the files we need to create said PAC does unfortunately only have the following SVD files (SVD files are the files we need for the PAC) :
   creating: STM32H7_svd/STM32H7_svd_V1.4/
  inflating: STM32H7_svd/STM32H7_svd_V1.4/STM32H742x.svd
  inflating: STM32H7_svd/STM32H7_svd_V1.4/STM32H743x.svd
  inflating: STM32H7_svd/STM32H7_svd_V1.4/STM32H750x.svd
  inflating: STM32H7_svd/STM32H7_svd_V1.4/STM32H753x.svd
  inflating: STM32H7_svd/STM32H7_svd_V1.4/STM32H7A3x.svd
  inflating: STM32H7_svd/STM32H7_svd_V1.4/STM32H7B3x.svd
  inflating: STM32H7_svd/STM32H7_svd_V1.4/STM32H7x3.svd
  inflating: STM32H7_svd/STM32H7_svd_V1.4/STM32H7x5_CM4.svd
  inflating: STM32H7_svd/STM32H7_svd_V1.4/STM32H7x5_CM7.svd
  inflating: STM32H7_svd/STM32H7_svd_V1.4/STM32H7x7_CM4.svd
  inflating: STM32H7_svd/STM32H7_svd_V1.4/STM32H7x7_CM7.svd

Nothing for stm32h725/735 in sight (STM32H7x5) refers to another series.
This means that Mission 1 would be to find the SVD files for stm32h725/stm32h735. After that you want to integrate those into https://github.com/stm32-rs/stm32-rs. And at that point we can start looking into getting something going with this HAL.

We could definitely support these parts in future! But @hargoniX is completely right, these parts need to be supported in the Peripheral Access Crate (PAC) first.

Alright, thanks for the insight, I'll see what I can find.

The svd files are provided with ST's firmware packs for a given processor, you can download that easily enough.
The real issue is that ST isn't consistent with their naming/organization, so the stm32-rs project applies patches to the svd files to make them consistent with each other before using svd2rs to generate the PAC.
So the real task here is that patching process. If it lines up closely with some of the other stm32h7 SVD files, it may be as easy as identifying which one and pulling in the right existing patches.
After the PAC supports the processor, I wouldn't expect it to be too hard to get the HAL to support it too (unless ST made some bigger changes or some important app notes about specific things that need to be handled differently). We recently got some PR's to add support for the STM32H7A3, STM32H7B0, and STM32H7B3; you can take a look and see what was involved in adding support for those chips.

Ok so everywhere I look on the ST resources the same svd zip is linked (no new files, no changes which would be related to this μController). Even on the resources for the stm32h735. So I'm inclined to think the STM32H7x5_CM7.svd file does fit?
However I hadn't realized, but this chip seems to be pretty recent (announced September 29th, 2020, and manual's first revision also dating from September 2020). Is ST known to release chips before the SVDs? Could be that the files aren't ready yet.

I think STM32H7x5_CM7.svd is actually meant for the H745/H755 parts.  Hopefully the correct SVD for the stm32h725 would describe some of the new peripherals added in RM0468 (PSSI, OCTOSPI, CORDIC, FMAC..)
I've not heard of it before, but it's certainly possible that ST haven't finished the files yet. It's probably true that the SVDs are lower priority for ST, even though SVD is a standard for ARM microcontrollers.

There's still no SVDs available from ST's website, but there are some SVDs available in the CMSIS pack here: https://keilpack.azureedge.net/pack/Keil.STM32H7xx_DFP.2.7.0.pack (can also be accessed from https://developer.arm.com/tools-and-software/embedded/cmsis/cmsis-search)
Unzip -> CMSIS/SVD/STM32H723.svd/CMSIS/SVD/STM32H73x.svd
There's certainly not perfect, but probably no worse than we're used to either. They could be a good starting point for someone wanting to create a PAC for these parts.

There's still no SVDs available from ST's website, but there are some SVDs available in the CMSIS pack here: https://keilpack.azureedge.net/pack/Keil.STM32H7xx_DFP.2.7.0.pack (can also be accessed from https://developer.arm.com/tools-and-software/embedded/cmsis/cmsis-search)
Unzip -> CMSIS/SVD/STM32H723.svd/CMSIS/SVD/STM32H73x.svd
There's certainly not perfect, but probably no worse than we're used to either. They could be a good starting point for someone wanting to create a PAC for these parts.

Integrating new SVDs into the stm32-rs eco system isn't too hard per se. Basically we have a repository https://github.com/stm32-rs/stm32-rs that holds all of the PAC crates. PAC stands for Peripheral Access Crate and provides a type abstraction for what C developers would usually directly do with volatile writes to certain addresses.
Basically what this repo does is:

Obtain the SVD files, more specifically https://github.com/stm32-rs/stm32-rs/tree/master/svd in here you will find a directory full of ZIP files by STM
Next up it applies the YAML based patch format from here https://github.com/stm32-rs/stm32-rs/tree/master/devices and here https://github.com/stm32-rs/stm32-rs/tree/master/peripherals to the extracted SVD files
Then it uses https://github.com/rust-embedded/svd2rust/ this tool in order to turn the SVD file into lots of Rust types and methods
And finally some formatting tools etc. to "beautify" the code a little bit

It is indeed possible that just adding your new SVD files to the vendor/ directory is already enough, it is most likely the build process will blow up at some point if something else is wrong so you should be able to just go through the README of the stm32-rs repo and try to add the SVD files yourself, then check if it builds.

I've done some work to update to the new SVDs. The good news is that they seem significantly improved over the previous terrible ones. The bad news is that we'll need to remove a lot of the manual patching that has already been done.

Thanks for the work on the PAC @mattico, it looks pretty good already! I'd definitely like to collaborate on this, and hopefully it's not too much work to finish off your PR (https://github.com/stm32-rs/stm32-rs/pull/554)
@TomDeRybel Great that you're interested in helping! As well as the work to update the PAC, I'll make a branch on this repo for updates to this HAL. Once the PAC work is done, the HAL branch would be a good place to contribute

Thanks for the fast responses to all of this!
When I've got my target board ready (about one month from now), I'll start using the PAC/HAL code for my project and report/contribute what I may encounter.

Hello,
I was glad to find that the support for stm32h735 had been started.
I have a STM32H735-DK development board, and I'm trying to get a Rust program run on it. I took the branch stm32h725_735_730 and mattico's pull request for the PAC . I configured this HAL library to use the new PAC by adding in this projects Cargo.toml the line stm32h7 = { version = "^0.13.0", path = "../tmp/stm32-rs/stm32h7" }.
I also had to modify the examples/blinky.rs, since the LEDs are on Port C (pins 2 and 3) on this development board.
I can compile the examples with a few warnings and no errors by running
cargo build --features=stm32h735,rt --examples, as expected. However, when I'm flashing the binaries to the board with st-util and arm-none-eabi-gdb

Hello,
I am glad to find that the support for stm32h735 has been started. However, I would appreciate some assistance.
I have a STM32H735-DK development board, and I'm trying to get the example blinky.rs run on it. I took the branch stm32h725_735_730 and mattico's pull request for the PAC . I configured this HAL library to use the new PAC by adding in this projects Cargo.toml the line stm32h7 = { version = "^0.13.0", path = "../tmp/stm32-rs/stm32h7" }, where the mattico's PACs Cargo.toml is located.
I also had to modify the examples/blinky.rs, since the LEDs are on Port C (pins 2 and 3) on this development board. Replace GPIOE with GPIOC and change pin number accordingly to 2 or 3.
I can compile the examples with a few warnings and no errors by running
cargo build --features=stm32h735,rt --examples
as expected. However, when I'm flashing the binaries to the board with st-util and  arm-none-eabi-gdb the LEDs just won't light up.
In first terminal I run the st-util server:
$ st-util
And in second terminal I connect to it to flash the executable to the board.
$ arm-none-eabi-gdb target/thumbv7em-none-eabihf/debug/examples/blinky
(gdb) target extended localhost:4242
(gdb) load
(gdb) continue

And I'm getting some cryptic messages from gdb:
(gdb) target extended localhost:4242
Remote debugging using localhost:4242
0x0800b61e in core::ptr::read_volatile<u32> (src=0x58024804)
   at /rustc/a435b49e86d16e98dcc6595dd471f95e823f41aa/library/core/src/ptr/mod.rs:1051
1051    /rustc/a435b49e86d16e98dcc6595dd471f95e823f41aa/library/core/src/ptr/mod.rs: No such file or directory.

If I try to step the code, I get more messages like that.
I really don't have a clue why this doesn't work and have been stuck here for days. Needless to say, I'm really frustrated. I'm new to ARM and from Rust I have the amount of knowledge of Rustlings  .
I hope I get this working, I'd be happy to contribute to this project.
Elias

Hi Elias!
Very nice that you're trying out the stm32h735 support already. Your Cargo.toml setup sounds correct, as you've found out it requires this change until at least the PAC support is merged into stm32 master.
The cryptic messages from gdb are a known issue. The debugonomicon briefly discusses them in this section, that should help solve your problem and get a working gdb.
For the STM32H735-DK, I suspect you're not setting the power bits for the SMPS correctly. The examples in the HAL should support boards that have an LDO (some Nucleos) as well as boards that have a SMPS (other Nucleos, and larger boards like the STM32H735-DK). Unfortunately even the most basic examples require different power bits, and the H7 itself can't detect how the traces on the board are connected (or at least I don't know how, I wonder if it's possible..)
Therefore in the HAL we currently have some feature flags to help build the examples for each board layout. There's some more documentation in examples/README.md, but in short for the STM32H735-DK you need to build the examples with cargo build --features=stm32h735,rt,example-smps --examples. And you need to power cycle the board for the new power bits to take effect.
Contributions are very welcome, and particularly any documentation that would make things easier for beginners!

Thank you Richard, I got this working with these tips. :)

Thanks again for starting the support for the newer H7 processors!
I've done some testing on a ST Nucleo-H743ZI2 board (STM32H743v), because I'm still waiting on parts to complete the H725 prototype. Except for the gotcha of needing svd2rust 0.17.0 to build the crate, this all went well.
The test project uses probe-run with defmt-rtt, and RTIC 0.6.0-alpha4. The test includes GPIO, an external interrupt, and USB. No change in behavior compared to the normal crate, except for one warning that now occurs when building the project:
warning: associated function is never used: `new`
   --> /home/tomr/workspace_RoboCow_Rust/stm32-rs-update-stm32h7/stm32h7/src/generic.rs:162:19
    |
162 |     pub(crate) fn new(bits: U) -> Self {
    |                   ^^^
    |
    = note: `#[warn(dead_code)]` on by default

Another thing that may or may not have to do with the crate is this: I can't get USB interrupts to work on OTG2, which is the USB port that is wired on the Nucleo board I have. Polling USB works for OTG2, but the interrupt never fires. OTG1 does, however, trigger interrupts. (This is the same behavior with the old crate.)
Looking at the crate build results, I notice that in "stm32-rs-update-stm32h7/stm32h7/src/stm32h743v", there is only a directory structure for OTG1, and none for OTG2. This part has both the OTG1 and OTG2 peripherals. Given that everything builds just fine with OTG1 and OTG2 peripherals referenced in the code, this is probably fine?
Anyhow, if this is relevant, I have a bug report on this here: https://github.com/stm32-rs/synopsys-usb-otg/issues/13
The bug report also includes example code that demonstrates the issue.

Should this issue be closed after #235 was merged?

Yes - thanks for the reminder!