Use WebCrypto APIs for AES & SHA operations

Related issues:

https://github.com/blockstack/blockstack-browser/issues/1900
https://github.com/blockstack/blockstack.js/issues/681
https://github.com/blockstack/blockstack.js/issues/564

Current associated PR: https://github.com/blockstack/blockstack.js/pull/737
Initial experimental PoC PR: https://github.com/blockstack/blockstack.js/pull/659
Solve some of the CPU & memory issues with our crypto operations, especially as it relates to:

User account registration and restoration.
File encryption and decryption in the storage APIs.

Tasks:

[x] Replace usage of Node.js crypto lib in the auth and storage modules with modern WebCrypto APIs.

Currently, when packaged for the web, Node.js crypto gets polyfilled with IE-era JS implementations. These are both large in file size, and CPU & memory inefficient. One symptom is significant lag during the related authenticator pages, especially on low-powered mobile devices.

[ ] Strategy for handling breaking changes for some public blockstack.js functions that must be converted from sync to async.

This PR is currently dealing with how to handle breaking changes on most of these same functions: https://github.com/blockstack/blockstack.js/pull/679

[ ] (Optional / not sure) Strategy for making downstream dependencies -- mostly bitcoinjs-lib & family packages -- to use W3C crypto APIs rather than the Node.js crypto polyfills or other redundant libs.

Projects that need updated -- higher priority:

[x] blockstack-keychain (draft PR opened https://github.com/blockstack/blockstack-keychain/pull/10)
[ ] blockstack-browser (draft PR opened https://github.com/blockstack/blockstack-browser/pull/1966)
[ ] gaia (draft PR opened https://github.com/blockstack/gaia/pull/272)
[ ] blockstack-cli (draft PR opened https://github.com/blockstack/cli-blockstack/pull/27)

Lower priority:

[ ] iOS SDK
[ ] Android SDK (uses its own native crypto functions?)
[ ] react native SDK (?)
[ ] Wallet app

I still want that!

I still want that!


Include public urls in slack notifications

The slack messages sent on performance changes don't actually contain links to charts for the metrics that changed. We should include public links to the tests so that users can navigate there.

Dup of #104


Fixed scripts to handle spaces

Updated Readme
Added sample Dockerfile

Thanks to Codacy I was able to fix some annoying issues when scripts were run with custom paths. Also modified readme so #41 may be closed.


Added mill-osgi module to documentation, contrib modules

Module is still in development, but can already produce proper bundles.

Thanks! Merged.


Google Sign In

Hi, first of all thank you so much for your good library. I implemented your library and every thing is fine.
Now I want to use Google sign in with your OAuth2 library and get access token after successful login by Goggle sign in.
I use Google sign in for my purpose and I verify the generated code by Google_Client library.
How can I get access token after successful login by google sign in ? I don't have any idea about it and in your example I haven't found about it.
My code:
<?php
class AuthController extends Controller
{
   public function actions()
   {
        return [
            /**
             * Returns an access token.
             */
            'token' => [
                'class' => TokenAction::classname(),
            ],
        ];
   }

   public function actionGoogle()
   {
        $data['ok'] = false;

        $post  = Yii::$app->request->getBodyParams();
        $token = $post['token'] ?? null;

        $client  = new Google_Client(['client_id' => Yii::$app->params['googleSignInClientId']]);
        $payload = $client->verifyIdToken($token);

        if ($payload) {
            $clientId          = $payload['aud'];
            $email             = $payload['email'];
            $emailVerified = $payload['email_verified'];
            $userId            = $payload['sub'];
            $name              = $payload['name'];

            // ================
            // I need access token here
            // ================
       }
   }
}

Hi! You will have to store google token in the session, when google send oauth callback.
This library uses the session to be able to return to the authorization process. Therefore, the simplest solution is to write a token to the session.

Thank you for your answer Andrey. Can I ask you show me your solution in a sample code ?
I have no idea how can I handle it after save google token in session.

I have the same problem.


Listing blobs with prefix query parameter sent as Prefix works in production but not in Azurite

Only tested the prefix but there could be more occurrences. It seems that the query parameter is not case-sensitive for production but is case-sensitive for Azurite, there will be an inconsistent behavior.
The HTTP protocol does not guarantee the casing for a query parameter, it is service behavior to ignore case:

The query string part is available to the server as it is. You can readily use mixed-case as you like, or discard the case (toLowerCase(...)). This also means that using a base64-encoded keys will work. You can't expect the users to type that correctly, though.

Linking to a PHP SDK issue: https://github.com/Azure/azure-storage-php/issues/235

Azurite targets to align with all public documented behaviors of Azure Storage. prefix parameter in following document is lowercase.
In https://docs.microsoft.com/en-us/rest/api/storageservices/list-blobs#uri-parameters,
It's un documented behaior that Azure Storage handles prefix query parameter case insensitively.
We are still thinking what's the best Azurite strategy to align with Azure Storage production especially for the undocumented behaviors.
It's not easy to update in Azurite side to make it case insensitive, because related code are auto generated from predefined swagger, in swagger prefix parameter is lowercased. We will see how to do that, but it may takes time.
In the same time, storage PHP SDK can have take this one line change to unblock customers and align with other Storage SDKs.

@katmsft I agree that a one-to-one behavior would be nice but I also agree with @XiaoningLiu on supporting the documented casing for the query params even though Azure Storage production allows for case-insensitivity unfortunately.
I've sent a PR that just updated the QPs listed in the doc link above but I can dig further into the documentation and update the PR with any other lowercasing that may be needed to match docs if you're willing to merge it.
Thanks!

Well, a fix like this does not work properly, as the QPs are also used in parsing the list result, which by official document should be mixed cases.
See this: https://docs.microsoft.com/en-us/rest/api/storageservices/list-blobs#response-body

Ah, I see, hmm... Nothing is ever easy, ha!
Thoughts on separating that out so that query params can match the lower case per docs and the XML can match the Response Body case per docs?
Something like XTAG_PREVIEW, XTAG_MAX_RESULTS, ... like others that are defined here.

You can do concurrency control by yourself or by 3rd party modules like async
https://www.npmjs.com/package/async
https://caolan.github.io/async/v3/docs.html#parallelLimit
...
var async = require("async");

// ...or ES2017 async functions
async.mapLimit(urls, 5, async function(url) {
    const response = await fetch(url)
    return response.body
}, (err, results) => {
    if (err) throw err
    // results is now an array of the response bodies
    console.log(results)
})


Issue - rooms do not allign

Hello,
I have encountered a bug where two rooms are misalligned, see picture:

This means that when the player passes rooms, they go up or down. For example,when moving from the left room to the right, the player looks like this:

(which isn't the issue but exemplifies the misallignment)
Notes:

Both rooms use the exact same tileset
other rooms ARE aligned properly

Things I've tried:

Check if the room or the tileset have position not vector zero: In both cases, there was no offset
erase the rooms from the editor and re-make them.
reassign the scene to the rooms

The rooms in question appear on the map like this:

So far, nothing helped. is there anything I could do? It's a really odd bug and it messes up screen trasnitions.
Thank you for your time!

Sounds similar to #18
This can happen when your in-game room size is not divisible by tile size. E.g. if you use 32x32 tiles with 960x540 resolution, your room size in tiles will be 30x16.875, which causes misalignment. You can fix it by either adjusting your in-game room size (it does not need to match screen size) or by offsetting your TileMap position to manually align tiles. The adjacent room previews exist to help with that.

Wow, good catch!!! I Slightly changed room size and now it fits. Thank you for your quick and helpful answer!
(Ps: I really love the addon, it's great!)

I added a troubleshooting page to address this: https://github.com/KoBeWi/Metroidvania-System/wiki/Troubleshooting


Translation of the buttons and texts

Hi bonartm,
I realy like your Quiz Engine and the Hugo Plugin you wrote. I realy like to use it in a projekt I'm currently on.
Unfortunately, all texts and buttons are only available in English and the website is currently only being developed in German. The majority of users are also not good at English and the hurdle would be too big.
Is there an easy way to translate all texts? Search and replace in the final minified JS-file is a bit confusing and I'm not familiar with npm development ;-)
A multilingual version woud be very nice! Especially in combination with the multilingual function in HUGO.

Language is now detected based on browser settings and text is set accordingly. Currently only german and english is supported but new languages can be added very easily. Thanks for the suggestion, I've learned a lot!


Licensing

Hi, I'd like to use this code in another open source project, but it currently doesn't have a license. Can you add one?

Added MIT license


Takes too long (beyond 1 hour) to generate the score in Admin account

Takes too long (beyond 1 hour) to generate the score in Admin account
More info:

Have enabled all the controls under cis-benchmark & aws-foundation
Have more than 30 accounts in the org

Thank @farrdoug , Appreciate the new feature. It has worked for me.


Update sbt-wartremover, wartremover to 2.4.18

Updates

org.wartremover:sbt-wartremover
org.wartremover:wartremover

from 2.4.2 to 2.4.18.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.wartremover" } ]

labels: library-update, early-semver-patch, semver-spec-patch, commit-count:1

Superseded by #195.


How to consume api's from external devportal in WSO2 API-Manager 3.0?

Description:
We are implementing a use case that, configuring an external devportal and users can consume the API's with sandbox keys from external devportal.  How we can achieve this ?

Hi,
Based on what I understood from your use case description, you want to develop a deverloper poral for the APIs. Then you can use the REST APIs exposed in our solution [1]. As the keys are not something issue by WSO2 API Manager key manager, you need to write a custom extension using our Key manager interface [2].
[1] - https://apim.docs.wso2.com/en/latest/develop/product-apis/restful-apis/#restful-apis
[2] - https://apim.docs.wso2.com/en/latest/learn/extensions/extending-the-key-manager-interface/#extending-the-key-manager-interface
Thank you!

Thanks for your prompt reply,
Usecase:
I have complete distributed setup like below
Key Manager (Identity Server 5.9.0)
Traffic Manager (API-M 3.0)
Publisher (API-M 3.0)
Internal Developer portal (API-M 3.0)
External Developer portal (API-M 3.0)
Hybrid Gateway (API-M 3.0)
Sandbox Gateway (API-M 3.0)
I want to publish Sandbox API's on external developer portal and Hybrid API's on internal developer portal.
As per below document "publishing to an external Developer Portal only means that the API is advertised there."
https://apim.docs.wso2.com/en/latest/learn/design-api/publish-api/publish-to-multiple-external-api-stores/#publish-to-multiple-external-developer-portals.
My question: Is it possible to external users to subscribe api's from external developer portal (without login to original publisher's tenant Developer Portal) ? if not, suggest me an alternative solution.


Fix issue displaying too many videos

We weren't filtering based on the input state.
JIRA: https://openedx.atlassian.net/browse/MA-1421

@mikekatz please review

:+1:


WIP: react-redux

my attempt to start with react-redux typing met with error. Please help.
ERROR: react-redux_v4.4.0--flow_>=v0.24.2
 * react-redux_v4.4.0--flow_>=v0.24.2 (flow-v0.24.2): Unexpected Flow errors(2):
   0-test_react-redux-v4.4.0.js:2
     2: import { connect, Provider } from 'react-redux';
                          ^^^^^^^^ class type: Provider. This type is incompatible with
   union: type application of polymorphic type: type `ReactClass` | key set. See lib: react.js:174
     Member 1:
     type application of polymorphic type: type `ReactClass`. See lib: react.js:174
     Error:
      21:     props: Props;
                     ^^^^^ property `children`. Property not found in. See lib: react-redux_v4.4.0.js:21
       4: const A = <Provider />;
                    ^^^^^^^^^^^^ props of React element `Provider`
     Member 2:
     key set. See lib: react.js:174
     Error:
       2: import { connect, Provider } from 'react-redux';
                            ^^^^^^^^ class type: Provider. This type is incompatible with
     key set. See lib: react.js:174

   Found 1 error

   null

Needs more work IMO. The connect function is probably better off written like this:
declare function connect<Config, S: Object, A: Object, SP: Object, DP: Object, Store: Object>(
  mapStateToProps: (store: S, props: $Diff<Config, SP&DP>) => SP,
  mapDispatchToProps: (dispatch: (action: A) => void, props: $Diff<Config, SP&DP>) => DP,
  options?: {pure?: boolean, withRef: boolean}
): (component: ReactClass<Config>) => ReactClass<$Diff<Config, (SP & DP)>>;

declare function connect<Config, A: Object, SP, DP, Injected: Object, Store: Object>(
  mapStateToProps: (store: Store, props: $Diff<Config, Injected>) => SP,
  mapDispatchToProps: (dispatch: (action: A) => void, props: $Diff<Config, Injected>) => DP,
  mergeProps: (stateProps: SP, dispatchProps: DP, ownProps: $Diff<Config, Injected>) => Config,
  options?: {pure?: boolean, withRef: boolean}
): (component: ReactClass<Config>) => ReactClass<$Diff<Config, (SP & DP)>>;

Now, I know this super complicated. But Higher-Order-Functions are almost impossible to type with flow right now. This, as far as I know, is the most correct way to do this, but I also know it doesn't work nearly well enough. $Diff doesn't quite work to subtract Objects. It basically doesn't throw any errors.
I'm basically hoping that as Flow keeps getting better something like this will actually start working.

flow-typed now has libdefs for both react-redux@4 and react-redux@5. Can this be closed?

@ahutchings I'm keeping this open for now because the current libdefs for React-Redux require a LOT of manual typing. This is a different approach that does a lot more inference for you. That said this has bugs and holes.
Perhaps, it makes sense to move this PR over to flowtyped as well though. I'll look into that.

Are there any examples of either approach working? Most interested in checking the incoming props to the connect HOC, I've been looking for a while and have worked out how to get it to work.
Sent from my iPhone

On 2 May 2017, at 4:10 am, Naman Goel<EMAIL_ADDRESS>wrote:
@ahutchings I'm keeping this open for now because the current libdefs for React-Redux require a LOT of manual typing. This is a different approach that does a lot more inference for you. That said this has bugs and holes.
Perhaps, it makes sense to move this PR over to flowtyped as well though. I'll look into that.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

This hasn't been updated for quite a while and I can't add commits to the PR.
If this is still relevant please submit a new PR.


Reemc wrong model

Hi WBS rviz-plugin developers,
First, I want to thank for your plugin. I would like to make an example documentation for Reemc; but unfortunately, when I add the plugin in rviz, it shows a wrong piece of the model instead of the right model of Reemc.

Is there a way how to fix this?
Any answer will be helpful.
Thanks in advance!

You need to subscribe to a topic that publishes the whole-body state message.
I suggest you to start with a Python script that publishes such kind of message

Note that the TF information (for displaying the robot) is internally computed from the CoM and joint positions.

In fact, I have made a python script (as an object from the whole_body_state_msgs python classes) which publishes the following:

But when subscribing to the topic in WholeBodyState-plugin, in this case, /robot_states, rviz dies.
process[reemc_rviz-1]: started with pid [26584]
rviz: /build/ogre-1.9-mqY1wq/ogre-1.9-1.9.0+dfsg1/OgreMain/src/OgreNode.cpp:385: virtual void Ogre::Node::setOrientation(const Ogre::Quaternion&): Assertion `!q.isNaN() && "Invalid orientation supplied as parameter"' failed.
[reemc_rviz-1] process has died [pid 26584, exit code -6, cmd /opt/ros/kinetic/lib/rviz/rviz -d /home/niger/reemc_public_ws/src/reemc_tutorials/reemc_basic_tutorials/config/reemc.rviz __name:=reemc_rviz __log:=/home/niger/.ros/log/87ff79ae-38ee-11eb-a86a-00e18c1323fb/reemc_rviz-1.log].
log file: /home/niger/.ros/log/87ff79ae-38ee-11eb-a86a-00e18c1323fb/reemc_rviz-1*.log

I think this is due to that I supposed at the moment that the states q,v, tau are all zero, specifically, for information about the floating base (taking into account the free-flyer joint). These arguments are necessary for the python script publisher.
What I am thinking to do, is to subscribe from gazebo simulator q, v, and tau since they depend on time, and also get information about the floating base.

I think this is due to that I supposed at the moment that the states q,v, tau are all zero, specifically, for information about the floating base (taking into account the free-flyer joint). These arguments are necessary for the python script publisher.

This means that your quaternion would be 0, 0, 0, 0 - which you can also see in base_orientation in your screenshot. Set w to 1 to make it a valid quaternion and the crash should no longer appear.

After subscribing the true values q, v, in the python script, WBS-plugin displays the right model.
Thanks for your feedback!

Happy to hear this.
I will close this issue
Feel free to create other ones if this is needed


Move signals to dedicated file

This PR moves all signals to a dedicated "signals.py" file for each app. Then, using each app's AppConfig.ready(), imports that file. This guarantees registration of all signals.
The AppConfig for each app was not being registered. The INSTALLED_APPS setting was updated to allow Django to find it. Read about this here.

What were the side effects of not registering AppConfig?

Prior to this PR, there were no side effects of not registering AppConfig. The AppConfig subclassing allows some customization of the app (e.g. the label in the admin panel). We were not using any of this customization before.
This PR uses the AppConfig subclassing to more reliably register signals. I had a case where a signal was not being registered because I did not import the relevant "models.py" file where the signal had existed. By hooking into AppConfig.ready(), we can initialize these signals at a safe part of the application registration process every time the Django environment is created.


feat: added login page with hive logo, karen logo, login button

Hello 🍉
In this PR I added the login page with the logo and login button.
Not much, just give it a look and let me know if you find something to improve :)

very neat!
can you please remove the comments from the login page before merging?
In addition, you could create an SVG file for the logo and save it into the public dir.


Bootstrap Popover when clicking on events

I'm looking to integrate something like Bootstrap Popover when clicking on events; I think it is possible to achieve it by custom components; is there any example where it shows how I can integrate it with Bootstrap Popover and using custom components? And also if there is any other ways to achieve that, I appreciate it if you help.

Also please see: https://github.com/intljusticemission/react-big-calendar/issues/871#issuecomment-538204177

@jquense @TrySound @tobiasandersen No update/direction/help here?!


Info window stuck in top left corner for custom marker

I'm using custom marker and try to show an info window but it gets stuck in the top left corner, I have seen your examples and also found the case #498 but the difference is the custom marker. I have tried to add id attribute on the custom marker markup but then my custom markers get placed on wrong locations on the map.
This example does not work for me, I cannot use the r id="'{{eventLocation.id}}'"

//Kalle

@k-ljung take a look at my issuse, its solved
https://github.com/allenhwkim/angularjs-google-maps/issues/498#issuecomment-177223826

Thank you @MortezaAghili but the problem is that a custom marker does not support to use an Id.

Updated the example for this, testapp/custom-marker-ng-repeat.html
  <ng-map zoom="8" center="[40.74, -74.18]" default-style="true">

    <custom-marker ng-repeat="p in vm.positions track by $index"
      id="custom-marker-{{$index}}"
      on-click="vm.map.showInfoWindow('foo', 'custom-marker-'+$index)"
      position="{{p.pos}}">

        position {{p.name}}: {{p.pos[0]}}, {{p.pos[1]}}

    </custom-marker>

    <info-window id="foo" on-mouseover="vm.mouseover()">

        Lat: {{anchor.getPosition().lat()}}

        Lng: {{anchor.getPosition().lng()}}

    </info-window>

  </ng-map>

Closing this, please open another issue if you have more questions.

Just to comment on this issue.
I had the same problem when using id as int number. The way I solved this was to convert the ID to string:
vm.map.showInfoWindow('infoId', data.id.toString());

You saved my day!


one more .dat file extension fix

Matt, let me know when you can accept this one -- I just need this to close a ticket.
https://atlassian.kbase.us/browse/KBASE-3917


fluentd configcheck terminated before running

Describe the bug:
In one of the bigger cluster we observe a regression that the configcheck pod goes in Terminating state before the pod is started after a config change in Flows/Outputs.
Expected behaviour:
configcheck pod started after config change, running to completed if no errors occured.
Steps to reproduce the bug:
Cluster with 40 nodes and 80 flows/outputs
Additional context:
Add any other context about the problem here.
Environment details:

Kubernetes version (e.g. v1.15.2): 1.23.16
Cloud-provider/provisioner (e.g. AKS, GKE, EKS, PKE etc): Rancher RKE
logging-operator version (e.g. 2.1.1): 4.3.0-rc0
Install method (e.g. helm or static manifests): Helm
Logs from the misbehaving component (and any other relevant logs):
Resource definition (possibly in YAML format) that caused the issue, without sensitive data:

event log:
0s          Normal    Scheduled                pod/logging-bronze-fluentd-configcheck-6e8fd6c1           Successfully assigned caas-logging-system/logging-bronze-fluentd-configcheck-6e8fd6c1 to worker20-az2
0s          Warning   FailedMount              pod/logging-bronze-fluentd-configcheck-6e8fd6c1           MountVolume.SetUp failed for volume "app-config" : secret "logging-bronze-fluentd-configcheck-app-6e8fd6c1" not found
0s          Warning   FailedMount              pod/logging-bronze-fluentd-configcheck-6e8fd6c1           MountVolume.SetUp failed for volume "output-secret" : secret "logging-bronze-fluentd-configcheck-output-6e8fd6c1" not found
0s          Normal    Pulling                  pod/logging-bronze-fluentd-configcheck-6e8fd6c1           Pulling image "mtr.devops.telekom.de/kubelogging/fluentd:v1.15"
0s          Normal    Pulled                   pod/logging-bronze-fluentd-configcheck-6e8fd6c1           Successfully pulled image "mtr.devops.telekom.de/kubelogging/fluentd:v1.15" in 354.910688ms (354.918716ms including waiting)
0s          Normal    Created                  pod/logging-bronze-fluentd-configcheck-6e8fd6c1           Created container fluentd
0s          Normal    Started                  pod/logging-bronze-fluentd-configcheck-6e8fd6c1           Started container fluentd
0s          Normal    Killing                  pod/logging-bronze-fluentd-configcheck-6e8fd6c1           Stopping container fluentd
0s          Normal    Scheduled                pod/logging-bronze-fluentd-configcheck-6e8fd6c1           Successfully assigned caas-logging-system/logging-bronze-fluentd-configcheck-6e8fd6c1 to worker20-az2
0s          Normal    Pulling                  pod/logging-bronze-fluentd-configcheck-6e8fd6c1           Pulling image "mtr.devops.telekom.de/kubelogging/fluentd:v1.15"
0s          Normal    Pulled                   pod/logging-bronze-fluentd-configcheck-6e8fd6c1           Successfully pulled image "mtr.devops.telekom.de/kubelogging/fluentd:v1.15" in 321.345112ms (321.354422ms including waiting)
0s          Normal    Created                  pod/logging-bronze-fluentd-configcheck-6e8fd6c1           Created container fluentd
0s          Normal    Started                  pod/logging-bronze-fluentd-configcheck-6e8fd6c1           Started container fluentd

The missing secrets are there some milliseconds later, a second try the config-check Pod is in state completed. Maybe he can wait until the secrets are there?
/kind bug

what was the workaround here? deleting the configcheck pod solved it?

No, it works at the second time randomly


[NEW ALGORITHM] Merge Overlapping Intervals

Name:
Merge Overlapping  Intervals
About:
One of the problem in sorting category

Labels:
new algorithm, gssoc-ext, hacktoberfest, level1

Assignees:

[yes] Contributor in GSSoC-ext
[yes ] Want to work on it

Please assign me the issue  so that i can contribute

Please assign me the issue @pankaj-bind  sir
Thank you!!!


Label masters with 'node-role.kubernetes.io/master' label

Fixes #649

Codecov Report

Merging #650 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #650   +/-   ##
=======================================
  Coverage   38.26%   38.26%
=======================================
  Files          51       51
  Lines        3316     3316
=======================================
  Hits         1269     1269
  Misses       1845     1845
  Partials      202      202

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dd948b7...8e0f0e9. Read the comment docs.

@redbaron LGTM. Thanks for the improvement! 👍


Snake Game Using Go Language

https://github.com/trishan9/HelloWorld/pull/86 please approve this thank you

please accept bro https://github.com/trishan9/HelloWorld/pull/110


Cannot reach Tado

Hello,
first: thanks for your great plugin!
second: I have an issue. my log says, that Tado cannot be a reached. can you help?
Log:
aniels und Sinas Zuhause: Polling User Info...
5.8.2021, 16:45:45HOOBSTadoDaniels und Sinas Zuhause: Checking access token..
5.8.2021, 16:45:45HOOBSTadoDaniels und Sinas Zuhause: Access token NOT expired
5.8.2021, 16:45:45HOOBSTadoDaniels und Sinas Zuhause: Using https://my.tado.com
5.8.2021, 16:45:45HOOBSTadoDaniels und Sinas Zuhause: API request GET /api/v2/me
5.8.2021, 16:45:48HOOBSTadoDaniels und Sinas Zuhause: API request GET /api/v2/me  connect ECONNREFUSED <IP_ADDRESS>:443
5.8.2021, 16:45:48HOOBSTadoFEHLERDaniels und Sinas Zuhause: {"code":"ECONNREFUSED","message":"Cannot reach Tado. No response received."}

Having the same issues several times a day. Maybe an issue of the Tado API?
[18/08/2021, 03:09:06] [TadoPlatform] Lukas & Alicas Zuhause: {"code":"EAI_AGAIN","message":"Cannot reach Tado. No response received."} [18/08/2021, 06:39:53] [TadoPlatform] Lukas & Alicas Zuhause Wohnzimmer Heater: State: 0 [18/08/2021, 07:21:01] [TadoPlatform] Lukas & Alicas Zuhause Wohnzimmer Heater: State: 3 [18/08/2021, 09:47:16] [TadoPlatform] Lukas & Alicas Zuhause: {"code":"ETIMEDOUT","message":"Cannot reach Tado. No response received."} [18/08/2021, 09:48:56] [TadoPlatform] Lukas & Alicas Zuhause: {"message":"Client request error: getaddrinfo EAI_AGAIN auth.tado.com","statusCode":502,"error":"Bad Gateway"} [18/08/2021, 15:28:08] [TadoPlatform] Lukas & Alicas Zuhause Wohnzimmer Heater: State: 0 [18/08/2021, 17:37:29] [TadoPlatform] Lukas & Alicas Zuhause: {"code":"EAI_AGAIN","message":"Cannot reach Tado. No response received."}

Yes, sometimes the Tado API is not available or down for several seconds/minutes.


Introduce ReplicaSyncUpService to help Bootstrap-To-Standby transition

This PR introduces ReplicaSyncUpService interface and its implementation
for HelixParticipant. The service provides a way to track catchup status
between local replica and peer replicas. It contains a count down latch
that blocks replica in bootstrap or being decommissioned state until it
has caught up with peers or peers have synced up with local one.
Also this PR implements Bootstrap-To-Standby state transition logic in
replication manager and state model.

Codecov Report

Merging #1334 into master will decrease coverage by 0.25%.
The diff coverage is 17.77%.

@@             Coverage Diff              @@
##             master    #1334      +/-   ##
============================================
- Coverage     72.18%   71.93%   -0.26%
- Complexity     6498     6499       +1
============================================
  Files           472      473       +1
  Lines         37204    37325     +121
  Branches       4703     4722      +19
============================================
- Hits          26856    26848       -8
- Misses         9097     9223     +126
- Partials       1251     1254       +3

Impacted Files
Coverage Δ
Complexity Δ

...in/java/com.github.ambry.store/StorageManager.java
88.76% <ø> (ø)
54 <0> (ø)
:arrow_down:

...ry.replication/CloudToStoreReplicationManager.java
57.03% <ø> (ø)
10 <0> (ø)
:arrow_down:

...m.github.ambry.replication/ReplicationManager.java
82.3% <0%> (-5.44%)
17 <0> (ø)

...in/java/com.github.ambry.cloud/CloudBlobStore.java
82.15% <0%> (-0.35%)
69 <0> (ø)

...b.ambry.clustermap/StaticClusterAgentsFactory.java
38.7% <0%> (-1.3%)
4 <0> (ø)

...ithub.ambry.clustermap/AmbryStateModelFactory.java
83.33% <100%> (+1.51%)
3 <1> (ø)
:arrow_down:

...hub.ambry/clustermap/StateTransitionException.java
100% <100%> (ø)
2 <0> (ø)
:arrow_down:

.../com.github.ambry.cloud/VcrReplicationManager.java
72.52% <100%> (ø)
9 <0> (ø)
:arrow_down:

...java/com.github.ambry/config/ClusterMapConfig.java
100% <100%> (ø)
3 <0> (ø)
:arrow_down:

...om.github.ambry.replication/ReplicationEngine.java
78.86% <100%> (+0.1%)
43 <0> (+1)
:arrow_up:

... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 31adf46...9511650. Read the comment docs.

Comments are addressed. @ankagrawal @lightningrob gentle reminder to review. Thanks.


all list/string member functions should return a value

all list/string member functions should return a value

done.

done.


Make sure a empty user is usable in a Context

We're not going to do this. Everything should be done within a context. If stuff needs to be done without a context (which has to be prevented at all cost), it can be done using the sqlmodel library directly (which is not directly, I know).


adoption graph is not indenting as it should

see console log for dependencies - the graph bars are not indented.
the issue is that the "padding" to push the bar out is calculated as "NaN" in the script.
this is a customer where this occurs:
http://pathfinder-ui-pathfinder-dev.int.open.paas.redhat.com/pathfinder-ui/report-quadrant.jsp?customerId=b354c697-5505-4193-bbcb-c23764200723

not ordering as well as I hoped either - the dependant apps are not together

fixed the NaN issue so the indenting is correct


Mina Daemon crashed

Hi there!
I'm a little new to mina and it asked me to submit a crash report. :-)
Here's the last bit of text I saw on my terminal when it crashed:
"backtrace": [
"Raised at file "src/error.ml" (inlined), line 9, characters 14-30",
"Called from file "src/app/cli/src/mina.ml", line 667, characters 12-27",
"Called from file "src/deferred1.ml", line 17, characters 40-45",
"Called from file "src/job_queue.ml" (inlined), line 131, characters 2-5",
"Called from file "src/job_queue.ml", line 171, characters 6-47"
]
}
2021-04-24 05:57:21 UTC [Warn] Shutdown before Coda instance was created, not saving a visualization
☠  Mina Daemon crashed.
The Mina Protocol developers would like to know why!
Please:
  Open an issue:
    <https://github.com/MinaProtocol/mina/issues/new>

  Briefly describe what you were doing and attach the crash report /home/phat/.mina-config/coda_crash_report_2021-04-24_05-57-21.154501.tar.gz

phat@ubuntu:~$
phat@ubuntu:~$
phat@ubuntu:~$
coda_crash_report_2021-04-24_05-57-21.154501.tar.gz
Mina originally ran pretty good when I ran this at the terminal:
mina daemon --generate-genesis-proof true --peer-list-url https://storage.googleapis.com/mina-seed-lists/mainnet_seeds.txt
But the next time I tried to run the daemon, I set generate-genesis-proof to false instead:
mina daemon --generate-genesis-proof false --peer-list-url https://storage.googleapis.com/mina-seed-lists/mainnet_seeds.txt
And that's when it crashed! Silly me.

coda_crash_report_2021-04-24_05-57-21.154501.tar.gz

You really do need to set that flag to true. The crash in this case is intentional.

psteckler,
Thanks so much for the quick update. I'm really new to Mina and eager to learn!
Just curious, is there any scenario where the --generate-genesis-proof flag should be set to false?
For certain debugging situations, maybe?
Thanks!
Patrick

It's possible to provide the genesis proof in a file, which is faster than generating one.

Ah, cool. Thanks!
Does the documentation cover that somewhere? I'm sure it does but I haven't figured out where to look yet...
I will be happy to RTFM and learn as much as I can.

I believe the only current documentation for this feature is the CLI reference: https://docs.minaprotocol.com/en/using-mina/cli-reference
As usual, there can always be better documentation.

Thanks! I bookmarked the site.
So if I understand correctly, would the following scenario allow the daemon to start up smoothly:

At the command prompt, include the flags:  --genesis-ledger-dir DIR
A valid ledger would also need to actually exist in the above directory, of course.
--generate-genesis-proof can be set to false or just not typed included at all, since the file mentioned in steps 1&2 has the the needed genesis proof.

Finally, I'm a little OCD about good documentation-- do you happen to know if there's somebody I could talk to about helping to contribute with that? In my day job I spend a lot of time explaining things, so I feel like I have a talent for it.
Best,
Patrick

Yes, that startup should work (I've not tried it myself).
I've asked the O(1) product team to respond re contributing to docs. Thanks for the offer!

You're welcome. Thanks for your help!
-Patrick

Finally, I'm a little OCD about good documentation-- do you happen to know if there's somebody I could talk to about helping to contribute with that? In my day job I spend a lot of time explaining things, so I feel like I have a talent for it.

Heyo, I'll give a bit of instruction on how to contribute to our docs. Please feel free to reach out to me for further details.
Our docs repo is located here: https://github.com/MinaProtocol/docs.git
Feel free to make pull requests for any contributions you see fit! :)
The README should be detailed enough to get the application running locally. Just make sure to pull the LFS assets and then run:
npm install
npm run dev
# or
yarn
yarn dev

It should be accessible on localhost:8000
Our documentation content is written into .mdx files located in this directory: https://github.com/MinaProtocol/docs/tree/main/pages/en
You should be able to edit the markdown files directly and test them in your browser.
One thing I want to flag is that our development experience for the docs isn't great at the moment. When you click on a nav item the link will lead to a 404 error and look like:
http://localhost:8000/docs/...
You will need to replace the /docs/ with /en to get the links to work:
http://localhost:800/en/...
Please let me know if you have further questions.

Hiya Martin, nice to meet you, and thanks for the info!
Is there a wish list or bug database of the most pressing documentation issues (or development issues for that matter)?
Best,
Patrick


Cross check verification method data model from w3c spec

https://www.w3.org/TR/did-core/#verification-method-properties

Linked Issues:

https://github.com/hypersign-protocol/hid-node/issues/48


Add body with features

YouTube files was merged with css and index.html to design simple webpage for review.

Owners of projects cannot change status to Approve or Request Cha;nges

In work description Owners of projects cannot change status to Approve or Request Changes. So please proceed to approve this project.

@rica213  previous reviewers made wrong reviews concerning the status of my review. But little did they know that owners cannot  Approve or Require Change on their projects. Now you have exhausted my last chance to make it across with optional changes.

Hello @Mike47ip
This is an additional review as a result of the second opinion request.

You send a second opinion for this review You are right about that owners cant approve or request changes on their project. However, the code reviewer is right to ask for other requirements.
It looks like you solved all issues so your project will approve soon.

As this was the only reason your project was not approved it will be approved as soon as possible.
Please note that since we will approve it manually so this could take a few hours

Cheers and Happy coding! 💻 👏

GitHub | LinkedIn | Twitter
Reviewed by Ahmet Bozacı


Incorrect condition in worker loop in documentation?

What happened?
I used the code snippet in the FrankenPHP documentation and forgot to set the MAX_REQUESTS environment variable.
FrankenPHP then started logging INFO    restarting      {"worker": "/app/public/index.php"} every millisecond and did not accept any requests.
The loop looks like this:
for($nbRequests = 0, $running = true; isset($_SERVER['MAX_REQUESTS']) && ($nbRequests < ((int)$_SERVER['MAX_REQUESTS'])) && $running; ++$nbRequests) {

I believe it should look like this:
for ($nbRequests = 0, $running = true; $running && (!isset($_SERVER['MAX_REQUESTS']) || $nbRequests < (int)$_SERVER['MAX_REQUESTS']); ++$nbRequests) {

allowing to handle an unlimited number of requests if the variable is not specified and taking it into account if it is, instead of not entering the loop if the variable is not specified.

In fact perhaps the following is more appropriate, given that $_SERVER is “reset” after handling the first request. While environment variables will be part of $_SERVER for both the worker request and the HTTP requests, it feels more correct to store it in a local variable instead of a superglobal.
$maxRequests = (int)($_SERVER['MAX_REQUESTS'] ?? -1);
for ($nbRequests = 0, $running = true; $running && ($maxRequests < 0 || $nbRequests < (int)$maxRequests); ++$nbRequests) {

Build Type
Docker (Debian Bookworm)
Worker Mode
Yes
Operating System
GNU/Linux
CPU Architecture
x86_64
PHP configuration
Docs issue.

Relevant log output
No response

Good catch! Would you mind opening a PR?


Add new changelog fields

Original Proposed Changed
Adding these as filter parameters for the changelog.

Parameter
Description
Example

postRank
Player post rank.
?postRank=2

preRank
Player pre rank.
?preRank=3

dateStart
Date range start.
?dateStart=2013-01-01

dateEnd
Date range end.
?dateEnd=2018-01-01

hasDate
Entry has a known date.
?hasDate=1

id
Changelog id.
?id=1337

wos
Player is banned.
?wos=1

top
Top x post rank.
?top=10

Additional Changes
Adding these changes to the database.

Parameter
Description
Example

modReviewed
Status on if a mod has checked the run
?modReviewed=1

modNote
Note from a moderator
?modNote="Ban Reason: timescale (Daniel)"

The value could be an integer rather than a boolean, that way we could have "not checked", "checked" and "NA". Either that, or we could just use NULL.
Link to iVerb Pull Request

After some review, here are goals.

[ ] Migrate to Postgres (for DISTINCT ON)
[ ] Remove the scores table. (Previously used to filter for only top times, replace with DISTINCT ON)
[ ] Trim coopbundled to 2 profile_numbers, 2 map_ids, 2 changelog_ids and is_blue.
[ ] Either swap out the ORM, or compile custom Diesel
[ ] Add mod_reviewed and mod_note to the changelog table.


GPS on Navio2 "NOT OK"

Bug Report
Hi everyone,
Just downloaded the most recent PX4/Firmware (master-Branch) build it on the Raspberry Pi 3 Model B.
After building an running PX4 the GPS status shows:
pxh> gps status
INFO  [gps] Main GPS
INFO  [gps] protocol: UBX
INFO  [gps] port: /dev/spidev0.0, baudrate: 0, status: NOT OK
INFO  [gps] sat info: disabled

It seems that it was an issue before
https://github.com/PX4/Firmware/pull/5440
Already tested out with Ardupilot if my hardware had a problem. But with Ardupilot I could find satellites.
Thank you for your help!

Hi
#5440 got merged in https://github.com/PX4/Firmware/pull/5561. Did you check that the port /dev/spidev0.0 is correct?

Hi
Thank you very much for your quick reply!
Port /dev/spidev0.0 seems to be correct
This is an GPS Example stating (in line 7) that the GPS receiver is connected as an SPI device on 0.0
https://github.com/emlid/Navio2/blob/master/C%2B%2B/Examples/GPS/gps.cpp

@bkueng Any additional insights?

Hey everyone,
Emlid most recently changed their webpage instructing the Raspberry Pi configuration.
( https://docs.emlid.com/navio2/common/ardupilot/configuring-raspberry-pi/ )
This is a Stretch (Debian) image witch doesn't support the GPS somehow.
Beat indicated to try it out with the older Debian Version Jessie (which was on the Emlid homepage before the last change).
With Jessie the GPS works.
I'm still running in some trouble with the ekf2 but I'm looking in to it.
Here is a log file. Maybe someone is able to find what I'm missing.
https://review.px4.io/plot_app?log=3000a063-b582-438f-ad62-6ed467fd3739
Great Thanks for your help so far Beat!
Cheers
Louis

I am running into this same issue.  I have a RPI 3B+ and the Navio2.  I am using the current image from Emlid (which is Stretch).  GPS works fine with Ardupilot.  When I run PX4, I get Status:NOT OK.
Is the Jessie image still available?  I did not see a way to get to it from the Emlid website.
More importantly, has anyone been looking into the root cause of the issue?  I was going to start looking into it, but if someone has started or has ideas as to what is going on I would love to hear about it.
Thanks!

I think I have found the issue and resolved it for my purposes.  I would consider this a bug in PX4.  I will say what I did to resolve the issue and if there is someone that is more familiar with the development process for PX4 maybe they can incorporate a proper fix in the main code.
I think the main issue was that the SPI clock was going too fast.  I found this link on the RPI forum.
https://www.raspberrypi.org/forums/viewtopic.php?t=177965
Which indicates there is a SPI clock difference between Jesse and Stretch.  It seems that if the clk speed is not explicitly set then it defaults, really fast.  When I looked at the SPI clock line on the board this was confirmed.  The clk rate was too fast for the RPI to properly drive it.
I added IOCTL code to GPS::run()
void
GPS::run()
{
if (!_fake_gps) {
/* open the serial port */
_serial_fd = ::open(_port, O_RDWR | O_NOCTTY);
if (_serial_fd < 0) {
PX4_ERR("GPS: failed to open serial port: %s err: %d", _port, errno);
return;
}
if(_interface == GPSHelper::Interface::SPI)
{
int spi_speed = 1000000;
int status_value = ioctl(_serial_fd, SPI_IOC_WR_MAX_SPEED_HZ, &spi_speed);
if (status_value < 0)
{
PX4_ERR("GPS: failed to set serial port write speed: %s err: %d", _port, errno);
return;
}
status_value = ioctl(_serial_fd, SPI_IOC_RD_MAX_SPEED_HZ, &spi_speed);
if (status_value < 0)
{
PX4_ERR("GPS: failed to set serial port read speed: %s err: %d", _port, errno);
return;
}
}
}
Hopefully someone who knows what they are doing can look at this and incorporate a proper patch into the main code.
Thanks!

Thanks for looking into this @mccandlt. PR is here: https://github.com/PX4/Firmware/pull/10749


Provide examples throughout for Substrate v3

The https://github.com/substrate-developer-hub/substrate-node-template lists branches with solutions for the tutorials, we should clean these and link to the correct branches in all tutorials for those looking (at the end of the tutorial?)
Why? so we have a know build-able source to confirm that those looking can compare to and troubleshoot against.

I agree, but lets do review on a case-by case basis. Many of those branches are reference to videos so they could still teach something and may only need a small update to the readme. Many others could be removed.

Closing in favor of a newer model to use patch files and CI to try and merge into node template releases for all templates. :soon:


Reorganize: Move test_g* unit tests into files/directories that mimic source code being tested

Housekeeping

[X] I am a maintainer of dbt-core

Short description
As part of our Robust Testing Initiative we want to re-organize our unit tests to mimic the source code directory and files. Doing one file is generally smaller than what we'd rank as one unit of work, so this is a grouping of multiple. Here we want take unit tests in unit test files starting with test_g* and reorganize them into appropriate files within appropriate directories. This includes: test_graph_runnable_task.py, test_graph_runnable_task.py, test_graph_selector_methods.py, test_graph_selector_parsing.py, test_graph_selector_spec.py, and test_graph.py. These files contain 72 unit tests.
Acceptance criteria
The unit tests in test_graph_runnable_task.py, test_graph_runnable_task.py, test_graph_selector_methods.py, test_graph_selector_parsing.py, test_graph_selector_spec.py, and test_graph.py have been moved to files within a directory structure under tests/unit/ that mirror the organization of the source code being tested.
Suggested Tests
The work is the tests
Impact to Other Teams
None
Will backports be required?
Narp
Context
https://github.com/dbt-labs/dbt-core/issues/9649

Duplicate of https://github.com/dbt-labs/dbt-core/issues/9961


Some questions about new Cross Contract Call

default with_unused_gas_weight is 1, it means every cross contract method get more gas than static_gas for them, if we call the method with enough gas, that is OK, because static_gas + unused_gas is enough, but if we call the method without enought gas, static_gas + unused_gas may not enough either, it will cause panic in cross contract call. So whether panic is uncertain, this is very bad. If this cross contract method is a callback, we hope it won't panic at any time.

default with_unused_gas_weight is 1, it means every cross contract method get more gas than static_gas for them, if we call the outside method with enough gas, that is OK, because static_gas + unused_gas is enough, but if we call the outside method without enought gas, static_gas + unused_gas may not enough either, it will cause panic in cross contract call. So whether panic is uncertain, this is very bad. If this cross contract method is a callback, we hope it won't panic at any time.

yes, the amount of gas that will be distributed by weight will be relative to the amount attached, but this is purely additive from specifying static amounts which previously existed. You have the right intuition that specifying a sufficient static gas for callbacks is important, but this new API allows you to utilize gas more efficiently without sacrificing control.
Did I understand what you are asking for more context about?


module error

Error: Cannot find module 'dotenv/config'
Require stack:

/node_modules/threads-api/build/threads-api.js
/node_modules/threads-api/build/index.js

npm install dotenv
import 'dotenv/config';
import './index.js';

Hey @nastarrr! Just deployed https://github.com/junhoyeo/threads-api/releases/tag/v1.4.5, which has this issue fixed. Can you update to the latest version?  🚀
Installing dotenv to your project manually, like in @MsLolita's solution, will work for previous versions. 👍


Almost imperceptable buzzing noise when playing / updating editor

If I weren't using headphones I wouldn't have noticed, it's very subtle.
Probably a driver issue / signal interference on my end, I haven't noticed it on anything else though, just Godot.
Just curious if anyone else has noticed this?

Fill out the issue template please (what OS? Godot version? what sound card?)

How to confirm this is related to Godot as compared to an OS driver issue, or a hardware one?

I had this issue with an old laptop and Blender 2.7x, do you have it installed? if you do you could try opening it and moving rotating the scene view (or maybe any other opengl program?)

I can confirm I have this static as well while rotating a 3D scene in the editor using the middle mouse button, can only really hear it when using monitoring headphones/speakers though. I think I have the same problem in blender but I'm not sure.
OS: Linux x86_64, Godot 3.x

Without more information on your hardware, OS, opengl version & all, it is unlikely we fix the issue.
Very likely like a driver bug though.

No one else reported it since then and it lacks information, so should be fine to close this issue.
(it can be reopened if someone reproduces this in 3.2.2)


Release Azure.ResourceManager.ContainerInstance

Link: https://github.com/Azure/azure-rest-api-specs/pull/21530
Readme Tag: package-preview-2022-10
Target release date: No later than 2023-01-27

Here is a pr https://github.com/Azure/azure-sdk-for-net/pull/33426 for Track 1.
Track 2 need extra work.

Track 2 released with https://github.com/Azure/azure-sdk-for-net/pull/33586.


Fix to not apply pagination filtering when there is no pagination template configured

Found based on @azatoth's comment here.
Also bumping the version number

I don't want to merge this PR with a bump in the version numbers. I prefer to do that in a separate commit and I'm leaning towards calling the next release 1.3 anyway since I might merge PR-53 before the release.

@mattiash alright, makes sense.  I've added a commit to put the version numbers back to where they were before


Programming exercises: Change Button Labels and add support for Local Testing and Feedback Mode

Checklist
General

[x] I tested all changes and their related features with all corresponding user types on a test server.
[ ] This is a small issue that I tested locally and was confirmed by another developer on a test server.
[x] Language: I followed the guidelines for inclusive, diversity-sensitive, and appreciative language.
[x] I chose a title conforming to the naming conventions for pull requests.

Server
-> no server changes made
Client

[x] Important: I implemented the changes with a very good performance, prevented too many (unnecessary) REST calls and made sure the UI is responsive, even with large data.
[x] I followed the coding and design guidelines.
[x] Following the theming guidelines, I specified colors only in the theming variable files and checked that the changes look consistent in both the light and the dark theme.
[x] I added multiple integration tests (Jest) related to the features (with a high test coverage), while following the test guidelines.
[x] I added authorities to all new routes and checked the course groups for displaying navigation elements (links, buttons).
[x] I documented the TypeScript code using JSDoc style.
[x] I added multiple screenshots/screencasts of my UI changes.
[x] I translated all newly inserted strings into English and German.

Changes affecting Programming Exercises

[x] I tested all changes and their related features with all corresponding user types on Test Server 1 (Atlassian Suite).
[x] I tested all changes and their related features with all corresponding user types on Test Server 2 (Jenkins and Gitlab).

Motivation and Context

https://github.com/ls1intum/Orion/issues/93/
Description
Orion has received a patch that allows the usage of more IDEs next to IntelliJ. Therefore some of the Button labels weren't up to date anymore.
Furthermore Orion gets an update where students can view inline feedback directly in their IDE. For that a new Reload Button is introduced
Steps for Testing
In Artemis:

Open a programming exercise! no extra button should be there

In Orion

Open a programming exercise
Test if the buttons are all appearing
2.1. Start assessment button
2.2. Open in your IDE button
2.3. Submit

No changes for Tutor buttons but they still should be visible
Without orion, all orion buttons should not be visible. (can be tested with a normal webbrowser)
Review Progress

Performance Review

[ ] I (as a reviewer) confirm that the client changes (in particular related to REST calls and UI responsiveness) are implemented with a very good performance
[ ] I (as a reviewer) confirm that the server changes (in particular related to database calls) are implemented with a very good performance

Code Review

[ ] Code Review 1
[ ] Code Review 2

Manual Tests

[ ] Test 1
[ ] Test 2

Exam Mode Test

[ ] Test 1
[ ] Test 2

Test Coverage

Screenshots

Fixed https://github.com/ls1intum/Orion/issues/93


bugtool: Provide a wrapper script for executing from k8s master node

It would be nice if there is a wrapper script for bugtool that:

calls cilium bugtool on all the cilium pods.
captures logs from cilium pods. It might also be useful to get kubectl logs --previous in case cilium crashed.
bundles all the output files into a single tgz file.

@scanf : Please feel free re-assign if you don't have enough cycles.

Duplicate of #2256


Dead link for Linear Regression in machine-learning.md

The link to the resources for Linear Regression in the machine-learning.md file appears to be broken as it redirects to a 404 not found page.

Thank you for noticing this! Indeed, that repo is sadly no longer in existence. Perhaps it will resurface.


Page not Found for Authorization Endpoint

Problem Description:

when requesting the /login/oauth/authorize API endpoint, "404 page not found" is returned.
The casdoor backend is deployed in a docker container serving http
an example command I used to access the self hosted backend would be
curl -X POST "http://$MY_HOSTNAME/login/oauth/authorize?client_id=$MY_CLIENTID&nonce=$SOME_NONCE&redirect_uri=$MY_APPLICATION_CALLBACK&response_type=code&scope=openid%20profile%20email&state=$SOME_STATE"

Expected Behaviour:

/login/oauth/authorize is a valid API endpoint, understands and process my request and then successfully redirect to $MY_APPLICATION_CALLBACK uri.
Incidentally, this can be the case when I access https://door.casdoor.com
This can be tested with the request below
https://door.casdoor.com/login/oauth/authorize?client_id=b800a86702dd4d29ec4d&nonce=nonce&redirect_uri=http%3A%2F%2F<IP_ADDRESS>%2F%2Fcallback&response_type=code&scope=openid profile email&state=heyiamastate
which return Login Error "Redirect URI: "http://<IP_ADDRESS>//callback" doesn't exist in the allowed Redirect URI list"
Thus showing that the request has been understood and processed

Further Details:
Upon checking [http://$MY_HOSTNAME/.well-known/openid-configuration], the authorization endpoint is returned as follows (N.B. I'm serving the backend with http)
{
"issuer": http://$MY_HOSTNAME,
"authorization_endpoint": "http://$MY_HOSTNAME/login/oauth/authorize",
"token_endpoint": "http://$MY_HOSTNAME/api/login/oauth/access_token",
"userinfo_endpoint": "http://$MY_HOSTNAME/api/userinfo",
"jwks_uri": "http://$MY_HOSTNAME/.well-known/jwks",
"introspection_endpoint": "http://$MY_HOSTNAME/login/oauth/introspect",
...
}
Upon checking the source code at object/oidc_discovery.go line 74
AuthorizationEndpoint:                  fmt.Sprintf("%s/login/oauth/authorize", originFrontend),
It seems that the authorization endpoint is fixed to be /login/oauth/authorize
However in routers/router.go, the path /login/oauth/authorize cannot be found.
Use Case:
I'm trying to setup SSO with an existing application, as autodiscovery is difficult to be done without https, I'm configuring the authorization endpoints, token_endpoints etc. explicitly in the application config file. When trying connect to casdoor through the said application, the authorization endpoint "http://$MY_HOSTNAME/login/oauth/authorize" is requested, but a "page not found" type message is returned.

I believe this is not exactly a question , but could be a bug where the /login/oauth/authorize API does not perform with the expected behaviour shown on the demo website

@ComradeProgrammer

@Steve0x2a

Problem Description:

when requesting the /login/oauth/authorize API endpoint, "404 page not found" is returned.
The casdoor backend is deployed in a docker container serving http
an example command I used to access the self hosted backend would be
curl -X POST "http://$MY_HOSTNAME/login/oauth/authorize?client_id=$MY_CLIENTID&nonce=$SOME_NONCE&redirect_uri=$MY_APPLICATION_CALLBACK&response_type=code&scope=openid%20profile%20email&state=$SOME_STATE"

Expected Behaviour:

/login/oauth/authorize is a valid API endpoint, understands and process my request and then successfully redirect to $MY_APPLICATION_CALLBACK uri.
Incidentally, this can be the case when I access https://door.casdoor.com
This can be tested with the request below
https://door.casdoor.com/login/oauth/authorize?client_id=b800a86702dd4d29ec4d&nonce=nonce&redirect_uri=http%3A%2F%2F<IP_ADDRESS>%2F%2Fcallback&response_type=code&scope=openid profile email&state=heyiamastate
which return Login Error "Redirect URI: "http://<IP_ADDRESS>//callback" doesn't exist in the allowed Redirect URI list"
Thus showing that the request has been understood and processed

Further Details:
Upon checking [http://$MY_HOSTNAME/.well-known/openid-configuration], the authorization endpoint is returned as follows (N.B. I'm serving the backend with http)
{ "issuer": http://$MY_HOSTNAME, "authorization_endpoint": "http://$MY_HOSTNAME/login/oauth/authorize", "token_endpoint": "http://$MY_HOSTNAME/api/login/oauth/access_token", "userinfo_endpoint": "http://$MY_HOSTNAME/api/userinfo", "jwks_uri": "http://$MY_HOSTNAME/.well-known/jwks", "introspection_endpoint": "http://$MY_HOSTNAME/login/oauth/introspect", ... }
Upon checking the source code at object/oidc_discovery.go line 74 AuthorizationEndpoint: fmt.Sprintf("%s/login/oauth/authorize", originFrontend), It seems that the authorization endpoint is fixed to be /login/oauth/authorize
However in routers/router.go, the path /login/oauth/authorize cannot be found.
Use Case:
I'm trying to setup SSO with an existing application, as autodiscovery is difficult to be done without https, I'm configuring the authorization endpoints, token_endpoints etc. explicitly in the application config file. When trying connect to casdoor through the said application, the authorization endpoint "http://$MY_HOSTNAME/login/oauth/authorize" is requested, but a "page not found" type message is returned.

http://$MY_HOSTNAME/login/oauth/authorize locates in the frontend. As you said, I quote "The casdoor backend is deployed in a docker container serving http", if the frontend is not running, it will be normal to get a 404. Besides, it's normal to find that this route doesn't appear in the router.go because it only contains routes available on backend.
Please set up frontend and have a try

Thank your reply, that answers a lot questions, I'll have a go at querying the frontend.
However, if I would like to use a backend only setup, is it possible to achieve a similar effect querying the backend API? Or is such arrangement not a recommended way to use casdoor.

I see, thank you very much. I have originally assumed that the backend alone is enough for oauth functionalities. However, when I come to think about it, I believe you are indeed correct, a front end is required in the authorization grant stage for username and password.
However, as casdoor does have a frontend and backend separation. Is it an expected behaviour for the backend at  "$MY_HOSTNAME:$BACKEND_PORT/.well-known/openid-configuration" to return
{
"issuer": http://$MY_HOSTNAME:BACKEND_PORT,
"authorization_endpoint": "http://$MY_HOSTNAME:BACKEND_PORT/login/oauth/authorize",
...
}
instead of some form of error stating that this api method is actually non-existent on this endpoint? (perhaps there are some other use cases for which I'm not familiar with that requires this type of backend behaviour)
I can see that if I query the frontend, it does indeed show the behaviour as you described
{
"issuer": http://$MY_HOSTNAME:$BACKEND_PORT,
"authorization_endpoint": "http://$MY_HOSTNAME:$FRONTEND_PORT/login/oauth/authorize",
"token_endpoint": "http://$MY_HOSTNAME:$BACKEND_PORT/api/login/oauth/access_token",
...
}
Anyway, thank you for your time and your clarification, I believe that my original question has been well-answered.

@hz324 plz follow the Offical way to setup Casdoor, and then ask questions if you encounter any.
https://casdoor.org/docs/basic/server-installation


/shifts/ page with details of all upcoming shifts

Same style as the your shifts section from the homepage.

Maybe with a calendar subscription! Refs:

#11

Page is here: https://www.pillarpointstewards.com/shifts/
I need to create a private/secret URL for the shifts calendar - for the moment it's available here:
https://www.pillarpointstewards.com/shifts.ics
Preview it in plain text here:
https://www.pillarpointstewards.com/shifts.ics?_plain=1
Or this link should work to subscribe in Google Calendar: https://www.google.com/calendar/render?cid=webcal://www.pillarpointstewards.com/shifts.ics

Final design for that page:

https://www.pillarpointstewards.com/shifts/


rewrite the farmerbot in go

there's a lot of instability in the v version combined with the instability in components + the requirement of having rmb-peer and http server exposing the sdk-ts functionality.
Rawda did some initial work on the a farmerbot version before in go, and it's time to revive, and revise it. Please also introduce simplification in the configurations format, toml or yaml should be very intuitive options ✌🏼

v repo: https://github.com/threefoldtech/farmerbot
go repo: https://github.com/rawdaGastan/farmerbot
farmerbot:
we have 3 managers:

node manager
farm manager
power manager

Jobs can be sent to the farmerbot using RMB
farm manager jobs:

define: to define a new farm

node manager jobs:

define: to define a new node
fine_node: to find a node (returns a node ID)

- _has_gpus_ => if you require one or more gpus you can filter on that with this parameter (should be a positive value)
- _gpu_vendors_ => a list of strings that will be used to filter the nodes on gpu vendor (for example AMD)
- _gpu_devices_ => a list of strings that will be used to filter the nodes on gpu device (for example GTX 1080)
- _certified_ => whether or not you want a certified node (not adding this argument means you don't care whether you get a certified or non certified node)
- _public_config_ => whether or not you want a node with a public config (not adding this argument means you don't care whether or not the node has a public config)
- _public_ips_ => how much public ips you need
- _dedicated_ => whether you want a dedicated node (rent the full node)
- _node_exclude_ => the node ids you want to exclude in your search
- _required_hru_ => the amount of hru required in kilobytes (add suffix mb or gb to define the required resources in megabyte or gygabite)
- _required_sru_ => the amount of sru required in kilobytes (add suffix mb or gb to define the required resources in megabyte or gygabite)
- _required_mru_ => the amount of mru required in kilobytes (add suffix mb or gb to define the required resources in megabyte or gygabite)
- _required_cru_ => the amount of cru required

power manager jobs:

power on: powers on a node
power off: powers off a node
configure: configures power

- wake_up_threshold => a value between 50 and 80 defining the threshold at which nodes will be powered on or off. If the usage percentage (total used resources devided by the total amount of resources) is greater then the threshold a new node will be powered on. In the other case the farmerbot will try to power off nodes if possible.
- periodic_wakeup => the time at which the periodic wakeups (powering on a node that is off) should happen. The offline nodes will be powered on sequentially with an interval of 5 minutes starting at the time defined in periodic_wakeup.
- periodic_wakeup_limit => by default only one node will be woken up every 5 minutes during a periodic wakeup. You can change that behavior by setting the periodic_wakeup_limit setting.

periodic_wakeup: turns the nodes on according to a configured wakeup time
powermanagement: turns nodes on/off according to the resources usage and the configured threshold

It's not clear what those managers do in the comment above. Can we add link to docs.
each module need to define:

Role of operation, what does it's needed in general
Automated tasks the manager does (for example, a power manager does random wake ups)
Configuration
Requests that are handled by this manager
How it work with other managers.

For example above, it seems there is a farm manager job to define a new farm. I have no idea what that mean

docs: https://github.com/threefoldtech/farmerbot/blob/development/farmerbot/farmerbot.md

Added:

testing the farmerbot using Brandon farm
added examples
sync config files updates

Tested here: https://github.com/threefoldtech/tfgrid-sdk-go/issues/787


Why do EndpointReceiver::recv* functions take ownership of BytesMut?

Wouldn't it make more sense to take a reference? My understanding is that, as of right now, every packet received over a Custom endpoint results in an allocation. For example, your upc crate is performing an allocation every time a packet is received.

usb-gadget places the passed BytesMut into the end of the receive queue which is managed by the kernel. When a packet is received the BytesMut in front of the queue is filled with the received data and returned. Thus, the passed-in and returned BytesMut are not the same object.

Also, I'm wondering if you could shed some light on why the design is the way it is right now. Is it because of how FunctionFS works internally or something?
More pointedly, would it be possible to implement the EndpointReceiver in such a way that it can recv directly into a &mut [u8]?
For my particular use case this would be optimal - since then I could just read from the endpoint FIFO directly into a framebuffer (allocated with gbm or something, such that it resides in VRAM or is otherwise ready to be DMA'd).

The main reason for this is performance.
To ensure good USB transfer performance it is important that the send and receive queues of the UDC are always filled. If we would wait for a receive buffer to get filled before posting the next buffer, this could result in a bottleneck. It is also the reason why I use AIO instead of synchronous IO (read, write calls).
Have you considered using BytesMut::split and BytesMut::unsplit for improving performance? This should at least allow you to receive multiple USB transfer in one continuous buffer. However, I am not sure if Linux guarantees that a posted buffer will be filled completely.
Completely avoiding copies might be difficult with the current architecture since [BytesMut] cannot currently be used to manage a pre-allocated buffer. I don't know if this is easily fixable, but if so, the best solution would probably be to extend the bytes crate appropriately.
Another solution might be for usb-gadget to expose the file descriptor of the endpoint file, so that buffer management can be done manually using unsafe code, for such high-performance cases as yours.

Yeah after further work on my crate I realized how the behaviour is supposed to work a bit better.
I understand now why the first few times you call recv with a Buffer, you get a None back. It wasn't so bad to implement something that is reasonably performant: I just keep a Vec<BytesMut> handy and pop one off the end to provide to usb-gadget. Then, when I'm done reading one that I get back, I push it on the end of that Vec.
The issue is that these buffers aren't contiguous, as you've already pointed out. I did look at split_to and I considered that an option of last resort since it would be kinda fussy to work with such an API to achieve what I want.

However, I am not sure if Linux guarantees that a posted buffer will be filled completely.

Yeah I guess that would indeed be an issue. I imagine that if the USB link is under pressure, that particular Bulk endpoint may not receive the full 512 bytes it would normally get (on a 3.0 link at least).

Another solution might be for usb-gadget to expose the file descriptor of the endpoint file

Yeah, I considered this because I noticed there's a few AIO crate options on crates.io. I haven't evaluated any of them to see if they might have the kind of API I'm looking for. But I figured if I found something, it would be a fairly simple patch in usb-gadget to allow the underlying FD to be retrieved. Glad to hear you think that's feasible as well :+1:

However, I am not sure if Linux guarantees that a posted buffer will be filled completely.

Hmm, I guess that would indeed be an issue. I imagine that if the USB link is under pressure, that particular Bulk endpoint may not receive the full 512 bytes it would normally get (on a 3.0 link at least).

I guess you would have to figure that out first. If this is the case, then copies are probably unavoidable.

Hmm, based on my understanding of how f_fs.c handles reads, it seems that anything other than receiving exactly as many bytes as you requested is an error.
I also don't see anything in the io_event struct or the signalling in aio_get_events that would indicate a buffer underrun.

Okay, what do you think about extending BytesMut so that it can be constructed like slice::from_raw_parts?

Alternatively, what do you think about such an api:
pub fn recv_exact(&mut self, buf: &mut [u8], len: usize) -> Result<()>;

The implementation will be responsible for submitting as many AIO operations as is necessary to satisfy the request, ensuring that each submitted request is the max_packet_size (or less for the final submission).
So for example, if I call:
let buf [u8; 51201] = [0; 51201];
ep.recv_exact(&mut buf, 51201).unwrap();

Here's what I expect to happen in the background:

10 (or whatever the queue_len of the AIO Driver has been configured with) AIO tasks are immediately submitted.
(Ideally, all 10 are submitted in one request, currently it seems the driver is submitting them one at at time, but the number of syscalls can be reduced by batching them up)
As the tasks come back, subsequent tasks are submitted, until the requested amount of bytes has been read.
The final task submitted would be for 1 byte (assuming the EP max size is the typical 512).
If at any point AIO returns an error, then all pending tasks are cancelled, and the caller is notified.

My understanding of the bulk endpoint is that you'll typically know exactly how much you're going to read from it, because you've received some kind of signal via a control message on EP0 to inform you that data is on the way. So in most cases, the current API is actually maybe a little too low level - you don't want to be juggling N buffers and worrying about keeping the underlying AIO task queue saturated, you just want to read N bytes from the endpoint (ideally as fast as possible).
I'm pretty sure the API as proposed could work async, as well. In fact it might be better to only implement it as async, and the sync API runs the async func in a single thread executor.

The advantages of this proposed API I think are such:

it's simpler for the cases where you know up front exactly how many bytes you need to read from the bulk endpoint (which as I noted before, I think is basically always the case? I'm curious what you think about that).
it ensures that the logic to do high performance reads with AIO is thought through and implemented once, in the library, rather than being something that every user needs to understand and deal with
it allows the buffer to be any arbitrary writable segment of memory somewhere (which was what I originally started this issue to ask about! xD)

If you think this sounds sane, I'm happy to take a stab at implementing such a thing. Let me know!

Alternatively, what do you think about such an api:
pub fn recv_exact(&mut self, buf: &mut [u8], len: usize) -> Result<()>;

The len parameter is unnecessary, since a slice in Rust already knows its length.

I'm pretty sure the API as proposed could work async, as well. In fact it might be better to only implement it as async, and the sync API runs the async func in a single thread executor.

How do you plan on handling cancellation? In Rust a Future may be dropped at any time, thus buf might be dropped as well; but you have already sent a pointer into buf to the AIO subsystem of the kernel which might now perform writes to dangling pointers.

If you think this sounds sane, I'm happy to take a stab at implementing such a thing. Let me know!

Generally I like the idea, provided that the existing API stays available.
I think that your approach will work out fine for the non-async case. However, providing an async function that takes a reference to a buffer and interacts with the AIO subsystem might be challenging in current-day Rust.

You might want to have a look at the io_uring implementation approaches for Rust. I have not followed them deeply, but I think they are facing similar problems.

However, providing an async function that takes a reference to a buffer and interacts with the AIO subsystem might be challenging in current-day Rust.

Yeah I've been thinking through how this would work but I'm pretty sure it'll be fine. This is what pinning is for. When you call an async func with a slice, the lifetime of that slice is guaranteed to last as long as the future does.
Also, I feel like my idea is validated somewhat, because tokio AsyncRead exists.

but you have already sent a pointer into buf to the AIO subsystem of the kernel which might now perform writes to dangling pointers.

The io_destroy syscall blocks until all pending IO request is definitely cleaned up. In the case of a single pending iocb, io_cancel seems to also do the same but it might need extra care since it can potentially require retries on EAGAIN. Not sure if blocking the Drop on a Future until the underlying io_cancel succeeds will be problematic. I'll be sure to do some tests.

You might want to have a look at the io_uring implementation approaches for Rust. I have not followed them deeply, but I think they are facing similar problems.

I did exactly this earlier today. Sadly, it seems that io_uring doesn't support FunctionFS yet (or the other way around?). Specifically, I tried the example shown here: https://crates.io/crates/io-uring, using the file descriptor for the endpoint. It seemed to not do anything. Maybe I just didn't test it properly. I was really hoping io_uring might work once I thought of it, because there's already much better ecosystem support for uring than there is for Linux AIO.
As a side rant, while thinking about this I realized that actually the perfect API for my need already exists. Are you ready for it? It's called std::io::Read, haha! I already know exactly how much data I wish to read, and I just want to read it into a buffer. read_exact does exactly that, submitting the request to the kernel as a single syscall.
... Problem is, it seems that somewhere deep in the kernel, I'm running into this error with the particular UDC hardware I'm testing with. My understanding is that udc/core is employing scatter-gather to satisfy my read syscall, but somehow it ends up providing the low-level UDC driver with a chunk of memory that isn't page aligned.
Oh well.

Generally I like the idea, provided that the existing API stays available.

The way I'm currently planning to go ahead with this idea is to implement a separate crate that wraps Linux AIO in an async API. It turns out the API to get an FD for an Endpoint already exists, so no changes to usb-gadget should be necessary.


Fail loudly

A better way of handling errors

See #10

tl;dr the current PR breaks the run and register commands, I want to merge your fork onto a branch in this repo so that I can fix the register/ run command and then merge into master.  Which would also set a president for how work is done on this project going forward.
The run and register commands relied on the alreadyRegisteredCheck to return a true or false value to function properly, so as it is neither work and I can't merge the pull request without the functionality to handle new jobs or jobs that don't have tjm data.
But, I'm also thinking now that the run command is basically redundant because the start command starts jobs that have tjm data and a flag could easily be added to the register command to start after they are registered with the cluster.
tjm register -r jobFile.json -c localhost:5678 where the -r flag would indicate that the job would be run immediately.  With this change the run command could just be removed.
I took the liberty of adding the flag option to the register command, but because of the weirdness of the forking situation I can't just commit it to your branch.
So maybe the best thing to do would be to merge your fork to a branch in this repo and I could change the register command on that branch and also add tests for all the commands before it is merged into master.  Kimbro would need to grant me permission to do this as it is I can't add branches.

It felt odd to return a boolean without a reason, especially since there were multiple reasons why it return false. I’d be okay with returning more context that a boolean.
I also agree with you that some of these commands might be conflated. Let’s have a discussion about that after we fix get this merged.

Going to close this pull request because it is covered in pull request #23


stop, clean; improve output; stub menu options

Codecov Report

Merging #362 into experimental will decrease coverage by 2.6%.
The diff coverage is 33.33%.

@@               Coverage Diff                @@
##           experimental     #362      +/-   ##
================================================
- Coverage         80.88%   78.28%   -2.61%
================================================
  Files                23       23
  Lines              1057     1096      +39
================================================
+ Hits                855      858       +3
- Misses              202      238      +36

Impacted Files
Coverage Δ

vent/core/rq-worker/test_rq_worker.py
100% <ø> (ø)
:arrow_up:

vent/menu.py
28.37% <23.07%> (-0.66%)
:arrow_down:

vent/api/actions.py
61.74% <7.14%> (-10.58%)
:arrow_down:

vent/api/plugins.py
91.94% <78.94%> (-0.24%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 04c175a...4ad3d31. Read the comment docs.


[docs] Unpleasant initial UI update for dark mode

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Current behavior 😯
The site initially displays light mode and then updates to dark mode, bringing pain to my eyes.
https://user-images.githubusercontent.com/48589760/176994691-5b955d74-eb8e-4969-b36a-ef01030b9edf.mp4
Expected behavior 🤔
Initial UI render should be in the correct mode
Steps to reproduce 🕹
Steps:

set the site to the dark mode
close the site
reopen the site

Context 🔦
The issue always happens when entering the site, often when refreshing.
The issue will not appear on doc running with yarn doc:dev, it is probably related to SSR (not quite sure).

  Output from `npx @mui/envinfo` goes here.

duplicate of #15588


When the text is too long, there may be a problem of being out of center?

Like

Talk
I am a rookie developer.
I used the custom function for rendering because the built-in template information icon does not have a corresponding color or close control, so I don't know how to make it correctly center the display of the control itself when the text is long. It would be perfect if the built-in template had corresponding icon colors and close buttons. This is my humble opinion. Thank you for your development. It should be the missing required component for Vuetify.

I'm also having trouble here. Tried overriding the size limits in the css with !important and tried adding custom classed but still was unable to.
Component: https://github.com/SelfhostedPro/yacht-nuxt/blob/d259cf7cefa41ca9a8c7f723b5b77259263d9392/components/notifications.vue
Composable (where toast() is called from): https://github.com/SelfhostedPro/yacht-nuxt/blob/d259cf7cefa41ca9a8c7f723b5b77259263d9392/composables/toast.ts


Compile Time TypeError Fix

Looks like during a refactor the return type for http.get in getHero was messed up - this fixes it.

I signed it!


Cache busting - Change build process to uniquely name css and js files

Ideally we want to make the css and js files created as part of a build have unique names so that we reduce the risk of caching issues when we make presentation layer changes.

I have a really nice technique for this that I use for Anvil. Shouldn't be hard to reuse.

Well, I say "really nice". Perhaps I meant "horrid".
## Add SHAs to files in /ide
RUN find /ide /runtime -type f -name "*.html" -not -path "*node_modules*" -not -path "*bower_components*" -print0 | xargs -0 sed -i -r -e '/\"\/ide([^\"]*)\?buildTime=0/s/\"/\\\"/g' -e 's/(.*\"\/ide([^\"]*))(\?buildTime=0)(.*)$/set -- `sha256sum \/ide\/app\2`;echo "\1?sha=$1\4"/eg'

Also, this issue is probably more applicable to isaac-app.

This commit is relevant too: https://github.com/ucam-cl-dtg/isaac-app/commit/b2d8dda8454ca9a1b0992d052676d06b09ec7f49

We've done pretty well without these!


Presets size classes support

Add iPhone support
Update layouts for compact width environment for the Presets view

Roadmap:

[x] Finalize landscape handset layouts once design is ready
[x] Handle trait collection changes
[ ] Edit pagination behavior to auto-select category in compact width environment

Before:

After:

Compact height updates:


Ability to add and duplicate states between states

Separate the next and last buttons from adding and duplicating states. Should be close to no code changes

Added in 602b6d732c387447d6115a92902bcca1a56c9315


METRON-1127: Add ability to escalate alerts for external consumption

Contributor Comments
This PR exposes a way to put alerts (or any message) into a configurable escalation Kafka topic.  In summary:

Adds a "metron_escalation_topic" parameter to MPack and Spring property file
Provides a default value and sets up the topic upon MPack installation
Adds a new AlertController for this and future alert-related functions
Adds a producer to the Kafka service and controller
Adds and updates the appropriate unit and integration tests, including making the KafkaControllerIntegrationTest simpler
Adds documentation to the README

This can be tested in full dev with Swagger by first ensuring the "escalation" topic is automatically created and showing up in the list of topics (use the "/api/v1/kafka/topic" endpoint).  Next use "/api/v1/alert/escalate" endpoint in the alert-controller to escalate an alert, you should receive a 200 response code.  Finally get the latest message from the Kafka "escalation" topic with "/api/v1/kafka/topic/{name}/sample" endpoint.  You should get the same message that you escalated in the previous step.
Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our Development Guidelines for the complete guide to follow for contributions.
Please refer also to our Build Verification Guidelines for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
For all changes:

[x] Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
[x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

[x] Have you included steps to reproduce the behavior or problem that is being changed or addressed?

[x] Have you included steps or a guide to how the change may be verified and tested manually?

[x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
mvn -q clean integration-test install && build_utils/verify_licenses.sh

[x] Have you written or updated unit tests and or integration tests to verify your changes?

[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

[x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

For documentation related changes:

[x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via site-book/target/site/index.html:
cd site-book
mvn site

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.

This is an important piece of functionality.  I think it requires more than the rest endpoint documentation.   I'm not sure where to put that though.  I almost imagine like recipes for rest.
creating_a_parser_and_starting_it.md
querying_and_escalating_alerts.md

I'd say the docs belong with the UI docs, since this is pretty much an endpoint to drive the UI buttons, no?

Platform wise, we want both.   The metron configuration UI is the 'default' ui for the project, but the rest api is the main supported interface.  As such they should be treated as separate entities, each documented on their own.  There could be many other rest clients.

I don't see the need for additional docs.  Escalating an alert here means to push the alert data to a Kafka topic.  That is all documented in the README, as far as I can see.  Additional detailed docs about querying and escalating alerts would be specific to the Alerts UI or whatever other UI uses this REST API.  Additional docs should go there.

@ottobackwards agreed, this is very separate from the management ui (won't touch, or be used by anything in the management ui). Also agreed this is a separate entity, but one that will be used by the alerts ui. Honestly, I'm not sure what else this needs to do. If we want to expand the scope, maybe we can do that as a follow on.
@nickwallen thinking about it, I'm inclined to agree that the path of least surprise for the soc user is the one to go for here, so you're right. The representation passed should be that shown in the UI. The challenge here is things like meta-alerts, where the actual alert to pass may not be fully present on the ui due to paging on its member alerts. (I can certainly see use cases where a meta-alert might contain O(1000s) of alerts).

part of the issue is there is no use case for how this is going to be used beyond the general.  How often do analysts that have possibly 1000s of alerts find the one magic alert?
I can imagine a flow in the ui like:
Do some queries and dig around, select some set of alerts from there to escalate, get an escalate dialog where you add some info, fill out some fields required for target routing etc ( so you need per target config unless you are going to automate it with nifi in-between? ) and then send the alerts as a set.
Aside from that, to restate where I am coming from:

it is my exp. that escalateTheseAlerts is preferable and more practical than escalateThisOneAlert
it is my exp. that most query api with possibly huge returns, paging etc for dense data objects ( remember the original string etc ) try to return contextually appropriate subsets of the data, across different network zones etc for performance etc, and the assumption that what the UI has is what you need to push to the other system is not a good one.  There is def. a tradeoff between optimizing for server-client ( only send what you need to, LDAP comes to mind for example, you get the props you need ) and not wanting to make other ES/? calls to get the real data.

These things are worth talking about though.
At this point in our REST api, we only have the main readme.  So just the call descriptions.  We don't have examples, more in depth docs per call or anything.  That is just were we are at.  But in the context of the PR I think we can talk about the call patterns, and maybe we should have a usage example.  Since we did not have a DISCUSS on this before hand, the PR is it.

@ottobackwards Those are all important points to consider.  I just don't know that we need to think through all of that right now.  I am thinking of this PR as a very basic, first pass.  Let's get some really basic functionality in, work with it, learn from it, then iterate a few times.  Let's solve those problems as we go, piece-by-piece.
@merrimanr @simonellistonball @all Do we have a JIRA for the basic UI changes that would leverage the functionality in this PR?  Maybe if we get that JIRA defined and documented we can better assess whether the functionality that we have here will work for a first pass.

I create a Jira for this:  https://issues.apache.org/jira/browse/METRON-1147.  It's could probably use some more description but this is a start.

The alert or list of alerts should be annotated with the user who escalated the alert(s) and time the alert(s) was escalated.

That JIRA works for me @merrimanr .  So the assumption right now is that the caller (the Alerts UI) would do the annotation that is required?

I don't think it matters that much but I'm leaning towards keeping the REST layer simple.  The Alerts UI will be responsible for adding other annotations so it's not a stretch to add these too.  Are there any advantages to pushing this responsibility to REST?  I don't have a strong opinion either way.

I think your current approach works best right now.  Keep it simple.  +1

@nickwallen
The question of the assumption of how complete the passed data is can wait.
My point about escalateOne vs. escalateMany stands.

Ok, then I will leave that for @merrimanr to respond to.  In my opinion we can tackle escalate many in a future PR.

The latest commit addresses the escalateMany request.

+1


SchedulingDefault_spec.js fails with Node v6.8.0

I suddenly have a broken build https://travis-ci.org/TryGhost/Ghost/jobs/167170509.
grunt test:core/test/unit/scheduling/SchedulingDefault_spec.js fails on Node  v6.8.0

https://github.com/nodejs/node/blob/master/doc/changelogs/CHANGELOG_V6.md#6.8.0
https://github.com/nodejs/node/pull/8655

weird weird.
When i decrease the timeout diff from diff - 200 to for example diff - 70, then it works.
diff is the difference between the next job and now. And 200 tells the timer to wake up 200ms before, because setTimeout is not accurate.
So if the timeout get's invoked earlier to now, then it works?
https://github.com/TryGhost/Ghost/blob/master/core/server/scheduling/SchedulingDefault.js#L173

We’ve noticed the same kind of issue with post scheduling: if there are at least 3 schedules, the scheduler won’t publish (at least) the last one.
Used Setup

Node.js: 4.5.0
Ghost: 0.11.1
Database: MySQL
Steps to reproduce: local Ghost setup with post scheduling available, requires Ghost 0.9.0 or later. Then create 3 posts, schedule them with 11 minutes time difference and let the scheduler publish them. In our setup, the scheduler didn’t recognize the third post.

@marcuspoehls I can't reproduce what you are describing.

In our setup, the scheduler didn’t recognize the third post.

What do you mean by saying "didn’t recognize the third post"?
So you schedule the third post and then it remains in status "scheduled"?

Node issue at https://github.com/nodejs/node/issues/9084

@marcuspoehls Can you please contact me in our slack channel?
I am curious why you have problems on node 4.x.x 👍
This issue here describes a problem with node 6.x.x!

@kirrg001 Hi Katharina, sorry for the confusion. I found the actual issue: Ghost doesn’t pick up scheduled posts correctly after restarting the process.
Looks like while testing and reproducing the behaviour I didn’t let the process run continuously.
The init method of post scheduling loads already scheduled posts with the following filters:

status:scheduled
created_at:>=moment().subtract(7, 'days').startOf('day').toDate()
created_at:<=moment().endOf('day').toDate()

That results in a options object like this:
{ context: { internal: true },
  from: Fri Oct 07 2016 02:00:00 GMT+0200 (CEST),
  to: Sat Oct 15 2016 01:59:59 GMT+0200 (CEST),
  filter: 'status:scheduled',
  columns: [ 'id', 'published_at', 'created_at' ] }

If there are already scheduled posts, they need to be created in between the from and to date range! Every earlier created post than the from date won’t get selected and never published through the scheduler.
Looks like the filter on from and to isn’t required at all?

@marcuspoehls yeah we can kill from/to filters and just fetch all scheduled posts you have left in the database 👍

It's fixed in 6.8.1!


Client disconnects after a minute and a half

I've just started using this library with Unity. My server is sending updates about 30 times a second. After about 1.5 minutes of receiving the events perfectly the client just disconnects. It does not attempt to reconnect.
I will also need to send updates about 30 times/second. I'm not sure if that is possible given another open issue that I saw which says the events are buffered before emit?
My most urgent issue is the disconnecting. Is there a way to find out why it's suddenly disconnecting?

http://stackoverflow.com/questions/ask


Removing vendors and upgrade operator sdk

Removing vendors directory
Upgrading operatorsdk, client-go and few other pkg to the latest version
Code change to pkg/controller/resource{deployment.go, statefulset.go, deploymentconfig.go, daemonset.go} due to version upgrade

Signed-off-by: Raj Babu Das<EMAIL_ADDRESS>

Build process also require to update @rajdas98

Build process also require to update @rajdas98

Yes, @chandankumar4 . I am giving this PR some break. I will do it after rhe release.


Hash mismatch errors on 8.30 in Windows 10 and Windows 11

Hi DigitalSleuth,
I have been using win-for 8.30 release with success, but today when trying to install it on Win10 or Win11 VirtualBox VM it is having a hash mismatch with the win-for-salt-v2023.26.2.zip - pic attached.
Is there anything I can do to fix this?
Thanks
Robert

Update, after checking the output logs, this might be due to not locating https://repo.saltproject.io/salt/py3/windows/3005/salt-3005.2-1-windows-amd64.exe

It looks like salt-3005.2-1 does not exist but has been replaced with salt-3005.3-1-windows-amd64.exe

I installed Salt 3005.2.1 manually but the SHA256 mismatch appears to be the issue between the calculated hash and the stored hash.

Hi @4NSIK1 , the issue is actually with the release file and not with Saltstack. I'll be issuing a new release of the Salt States this evening which should fix that issue, and encompass many changes. I'll also release a new version of the Win-FOR Customizer in the next day or two which will add some new features.
I'll let you know as soon as the Salt State issue is fixed!

Brilliant, @digitalsleuth thanks so much. I am loving this tool.

Hi @4NSIK1 , I've released a new version, v2023.27.0, which should address this issue, Let me know how it goes.
Cheers!

Hi @4NSIK1 , well the new version had a few bugs, so I've fixed those and released a new one, v2023.27.1. This should fix all the issues.
Cheers!

Hi @digitalsleuth, I know you mentioned this has been resolved but I'm getting this same issue with the newest version:

Is there anything that I could do to solve this?

Hi @afp235 , this issue should be resolved now. It appears to be an issue with some changes happening on Github's end. I've re-uploaded the current version (with a new version number), and everything should work now.