Fix kernel_size != 3 on winograd arm/mali.

This PR fix/enable cases for kernel_size other than 3x3.

It is a continuation on #3553 work.
Fix testcases for arm / mali and also #3641 .

@tmoreau89 , @merrymercy , @kevinthesun , @FrozenGene
Please help with the review.

Main benefit:
We have flexible tile_size and any kernel_size for winograd.

Examples (yolov3-tiny) of having tuned tile_size:
original (only direct)
[Task 13/20 (1, 384, 26, 26)|(256, 384, 3, 3)] (conv2d) {53.46 GFLOPS /direct}
[Task 14/20 (1, 512, 13, 13)|(1024, 512, 3, 3)] (conv2d) {31.17 GFLOPS /direct}
[Task 15/20 (1, 256, 13, 13)|(512, 256, 3, 3)] (conv2d) {28.77 GFLOPS /direct}
original (no tile_size autotune)
[Task 13/20 (1, 384, 26, 26, 'float16')] (conv2d) {93.78 GFLOPS /winograd} tile_size=2
[Task 14/20 (1, 512, 13, 13, 'float16')] (conv2d) {93.31 GFLOPS /winograd} tile_size=2
[Task 15/20 (1, 256, 13, 13, 'float16')] (conv2d) {81.34 GFLOPS /winograd} tile_size=2
proposed (with tile_size autotune)
[Task 13/20 (1, 384, 26, 26, 'float16')] (conv2d) {98.48 GFLOPS /winograd} tile_size=4
[Task 14/20 (1, 512, 13, 13, 'float16')] (conv2d) {106.15 GFLOPS /winograd} tile_size=4
[Task 15/20 (1, 256, 13, 13, 'float16')] (conv2d) {84.54 GFLOPS /winograd} tile_size=4
It is quite tedious to expose more benefits (WiP on my side) for tuned tile_size, these 3 was found running >1 week on mali. That's all what I personally found up to this day, i am sure there are more, will try soon 5x5 & 7x7 kernels (Unet, SuperRes), and post on tophub exceptional ones.

Is there a reason why we are disabling cuda tests for now?

Is there a reason why we are disabling cuda tests for now?

(kernel_size > 3) was running on cuda only.
now with this runs on all targeted arch.

(kernel_size > 3) is new for our winograd, since its a larger kernel the accuracy may also drop, interesting that wasn't catched before (i think memoization data was changed).

Furthermore,

Also, in future would like to add float16 too for this testcase, and for that rtol has to be even higher ~1e-4, so i think cannot avoid to handle different rtol values in our winograd conv2d tests.

@merrymercy , @kevinthesun , @FrozenGene
Will update on tophub .
Allow a little time for tophub update (need to fill tile_size), until then this PR is WiP.

@merrymercy , @kevinthesun , @FrozenGene
Will update on tophub .
Allow a little time for tophub update (need to fill tile_size), until then this PR is WiP.

It's done, review ready at https://github.com/uwsampl/tvm-distro/pull/8 .

Aha... I remember why I don't include tile_size as a tunable knob. This is not the xgb vs xgb_knob problem. It is because the search spaces for different tile sizes are different. But AutoTVM does not support this kind of space definition.
There are some knobs for tiling in the search space. However, they are dependent on tile size. For example, in the arm cpu's schedule, https://github.com/dmlc/tvm/blob/36702a7678573fa4ff9ea04c3920e60d590496c9/topi/python/topi/arm_cpu/conv2d.py#L390-L392. P is the number of winograd tiles. In one case, P is 196 if tile size is 4 while is 784 if tile size is 2. We construct the split knob according the the lengths of P, so different tile sizes will have different spaces.
Currently, AutoTVM does not support dependent relationship between tuning knobs. What it does is picking the first candidate of tile size (which is 2 according your construction) and initializing other knobs. In the above case it will enumerate all split schemes for P = 784. But some split schemes for P = 784 are invalid for P = 196, which will cause the AutoTVM hangs forever during feature extraction.
Ideal solution
Support hierarchical space definition in AutoTVM
Workaround

Fix the feature extraction. We hang on this line https://github.com/dmlc/tvm/blob/11da1ca31b8121e9a40c70f33ade27670d22e2d0/python/tvm/autotvm/tuner/xgboost_cost_model.py#L311 because we get exception when run this function https://github.com/dmlc/tvm/blob/11da1ca31b8121e9a40c70f33ade27670d22e2d0/python/tvm/autotvm/tuner/xgboost_cost_model.py#L328
Use feature_type = knob in the XGBTuner. I won't merge this solution to master.

You don't meet the problem when tuning for CUDA and mali because I added some restrictions to their schedule tempaltes and luckily they work around this problem.

@merrymercy ,

Aha... I remember why I don't include tile_size as a tunable knob. This is not the xgb vs xgb_knob problem. It is because the search spaces for different tile sizes are different. But AutoTVM does not support this kind of space definition.

I was driven by the idea that too simple formula like (tile_size = 4 if H % 4 == 0 else 2) can be rather a simple knob with a small list. Now we afford it with flexible winograd matrix computation.
As result fact, I posted in the header here, knob found better solution overriding mentioned static formula (2 vs 4 tile_size).

There are some knobs for tiling in the search space. However, they are dependent on tile size. For example, in the arm cpu's schedule,
https://github.com/dmlc/tvm/blob/36702a7678573fa4ff9ea04c3920e60d590496c9/topi/python/topi/arm_cpu/conv2d.py#L390-L392
. P is the number of winograd tiles. In one case, P is 196 if tile size is 4 while is 784 if tile size is 2. We construct the split knob according to the lengths of P, so different tile sizes will have different spaces.

I also found this correlation and dependency to P in case of our arm/mali.

Yes i agree, that is also my finding during tests regarding P. Would be great help me elaborate an exact relationship tile_size -> P . Otherwise, if we don't narrow down the possible lists,  we just can enumerate bulk range(0,256) for some of knob's there and let xgb find it out but it wold have to search in 10e+7 cfg.space than.

Again, as fact, even ignoring this dependency for now we are still better than existing formula.

In fact, for now, solutions with  this knob will be 2 or 4 (very rare 3 hits) in most cases.

Currently, AutoTVM does not support dependent relationships between tuning knobs. What it does is picking the first candidate of tile size (which is 2 according to your construction) and initializing other knobs. In the above case, it will enumerate all split schemes for P = 784. But some split schemes for P = 784 are invalid for P = 196, which will cause the AutoTVM hangs forever during feature extraction.

So, could help me find the relation of P given tile_size ?

Ideal solution
Support hierarchical space definition in AutoTVM
Workaround

Fix the feature extraction. We hang on this line https://github.com/dmlc/tvm/blob/11da1ca31b8121e9a40c70f33ade27670d22e2d0/python/tvm/autotvm/tuner/xgboost_cost_model.py#L311
because we get exceptions when running this function https://github.com/dmlc/tvm/blob/11da1ca31b8121e9a40c70f33ade27670d22e2d0/python/tvm/autotvm/tuner/xgboost_cost_model.py#L328
Use feature_type = knob in the XGBTuner. I won't merge this solution to master.

OK, i put xgb issue on higher pripority, it is trigerred 100% on such scenarios like #2111

You don't meet the problem when tuning for CUDA and mali because I added some restrictions to their schedule templates and luckily they work around this problem. But for arm cpu we will meet this problem.

Where are those restrictions exactly ?

Let me know if you want to try to fix it or want more information from me. Otherwise, I can fix it later.

Yes i give it a try to fix it, I'll update you on progress.

To sum up lets settle for now the order :

For now, as mentioned & tested,  our PR is still better than existing two cased conditional.
Try find xgb fea issue and repair it.
Lets see a smarter P given tile_size (please help me elaborate a generating formula).
If get tile_size->P we can try introduce hierarchical knob mode (i have an simple idea for that).

Arguing by point 0. , the PR would be mergeable, otherwise we transform this PR in a long multi PR.
Agree 1 +  2 + 3 are mandatory, i would help tackle each in upcoming another 3 PR (there will be too much debate if we keep in a single PR like this one).
What you think ?

Opened a disscuss topic and started first with xgb & hierarchical knob issue.

#3687 #3689 are necessary to merge this PR, otherwise this PR will make tuning on arm cpu fails.
I can merge this after the above two are merged.

#3687 #3689 are necessary to merge this PR, otherwise this PR will make tuning on arm cpu fails.
I can merge this after the above two are merged.

#3687 #3689 are necessary to merge this PR. Otherwise this PR will make tuning on arm cpu fails.
I can merge this after the above two are merged.

@merrymercy ,
May for  maliexpand other four vars in this PR , namelyyt, t1, t2 and unroll see here ?
Or lets do in other PR where I can show up with retuned/enhanced TOPI entries with comparing in numbers ?

Currently, the spaces of P and other knobs that are dependent on tile_size are created with tile_size=2
Should we modify these knobs to be the union of all valid value or wait until you implement the conditional knob?

Currently, the spaces of P and other knobs that are dependent on tile_size are created with tile_size=2
Should we modify the spaces of these knobs to be the union of all valid values or wait until you implement the conditional knob?

I would implement conditional/hierarchical knob,  partially done, need to validate/test. Allow a little more time, will post and reference it here.

Note that  #3717 is also proposing to update CUDA TOPHUB version

ping @cbalint13 can you update the PR?

ping @cbalint13

I close this for now, in order to complete the following is needed

Conditinal-Knob , a knob that hierarchicaly depend on other knob

implies that list's combinatoric space will be not fixed (not an easy way, triky to implement)

Once such a knob is implement revise winograd schedule schema for <>3, in this very PR.


Needs OnSave prop

Create a onSave() prop/event that is called when a user clicks the save button.  When onSave() is passed in props, the image should not be downloaded, rather it should return the image file.

Thank you for your advice, i will add it as soon as possible.

@richardjob has updated on version 0.3.3


Clean up state variable bindings

AdaptiveStateVariableBinding is used by producers and state accessor functions. There is a bit of confusion what a binding exactly means and what is it supposed to exactly do.
I've removed the fromBinding parameter from AdaptivePropertyProvider.setValue during adat class development, so now
setValue calls the binding that called setValue as well.
    override fun setValue(path: Array<String>, value: Any?, fromBinding: AdaptiveStateVariableBinding<*>) {
        when {
            path[0] == "i" -> values[0] = value
        }
        bindings.forEach { if (it != fromBinding && it.path.contentEquals(path)) it.setValue(path, false) }
    }

I have to decide the pattern/convention for state accessors:

I really want to keep the clean syntax: input { a.b.c }
The use cases might be overwhelmingly copyStore based

That means I don't really need AdaptiveStateVariableBinding.setValue as the change goes through the store anyways. More precisely, I don't have to set the value directly.
The only question is if I want to keep the possibility to directly write back to the state variable. I'm not sure about that.

Also, don't forget: adaptive-kotlin-plugin/testData/box/foundation/call/propertyAccessBinding.kt

Fixed some binding related bugs and added some documentation: Bindings


chore(main): release 5.2.2

:robot: I have created a release beep boop
5.2.2 (2024-07-03)
Features

bump twilio from 5.2.1 to 5.2.2 (2120e69)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/somleng/somleng-node/releases/tag/v5.2.2 :sunflower:


Bar Chart Bar Clicked Event

How can I listen to bar clicked event in a bar chart?

Even I am looking the solution for same

duplicate of #5


Room message download feature

#372

Submitted an additional commit for the CSS file, that was set to tabs not spaces.

Also, merge with the latest.


Unable to log in after upgrading

Hi,
After upgrading to the latest version today I'm no longer able to authenticate with the admin panel, I simply get the below error which makes me think a breaking change was made to the API layer.
Below is the error in question.
[2018-10-18 16:05:39] ERROR "GET /ghost/api/v2/admin/users/me/?include=roles" 401 1ms

NAME: UnauthorizedError
MESSAGE: Access denied.

level: normal

empty
empty
ERROR DETAILS:
    empty

UnauthorizedError: Access denied.
    at new UnauthorizedError (/var/www/blog/versions/2.2.3/node_modules/ghost-ignition/lib/errors/index.js:101:23)
    at ensureUser (/var/www/blog/versions/2.2.3/core/server/services/auth/session/middleware.js:95:10)
    at Layer.handle [as handle_request] (/var/www/blog/versions/2.2.3/node_modules/express/lib/router/layer.js:95:5)
    at next (/var/www/blog/versions/2.2.3/node_modules/express/lib/router/route.js:137:13)
    at getUser (/var/www/blog/versions/2.2.3/core/server/services/auth/session/middleware.js:79:16)
    at Layer.handle [as handle_request] (/var/www/blog/versions/2.2.3/node_modules/express/lib/router/layer.js:95:5)
    at next (/var/www/blog/versions/2.2.3/node_modules/express/lib/router/route.js:137:13)
    at cookieCsrfProtection (/var/www/blog/versions/2.2.3/core/server/services/auth/session/middleware.js:104:16)
    at Layer.handle [as handle_request] (/var/www/blog/versions/2.2.3/node_modules/express/lib/router/layer.js:95:5)
    at next (/var/www/blog/versions/2.2.3/node_modules/express/lib/router/route.js:137:13)
    at session (/var/www/blog/versions/2.2.3/node_modules/express-session/index.js:454:7)
    at getSession (/var/www/blog/versions/2.2.3/core/server/services/auth/session/middleware.js:48:12)
    at Layer.handle [as handle_request] (/var/www/blog/versions/2.2.3/node_modules/express/lib/router/layer.js:95:5)
    at next (/var/www/blog/versions/2.2.3/node_modules/express/lib/router/route.js:137:13)
    at Route.dispatch (/var/www/blog/versions/2.2.3/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/var/www/blog/versions/2.2.3/node_modules/express/lib/router/layer.js:95:5)

Rolling back to my previous 2.1.3 install fixes the issue.
System Details

Ubuntu 16.04.5 LTS
nginx 1.14.0
Ghost-CLI version: 1.9.6

Hey :wave: Is that error after you enter your username and password into the admin panel?

Yes, immediately upon hitting the Sign in button it redirects back to the login screen with that 401 message in the console.

What does the request to the /session endpoint look like when signing in? Is that successful? Can you see the session cookie in the developer tools?

/ghost/api/v2/admin/session responds with a 201 code but the body in chrome is:

Failed to load response data

That's strange - the response data should be the string 'Created'..
Any more data you could give would be appreciated!
Are you seeing the Set-Cookie header in the response?
Are you able to see the session cookie in the developer tools?
Are you able to see the cookie sent in the Cookie header on the request to /ghost/api/v2/admin/users/me/?include=roles?

 As for cookies, they are on and work since rolling back yields a successful result.

As for cookies, they are on and work since rolling back yields a successful result.

2.1.3 doesn't use cookies so that's not an indication that cookies are working.
"Failed to load response data" in Chrome suggests you may be using the preserve log function? It won't show you the response data for requests after you navigate away so it may be a red herring. Could you try again? Do you see any redirects happening?
Do you get any errors in the DevTools console when trying to log in? If for some reason CORS is tripping up then there would be an error there.

That's strange - the response data should be the string 'Created'..
Any more data you could give would be appreciated!
Are you seeing the Set-Cookie header in the response?
Are you able to see the session cookie in the developer tools?
Are you able to see the cookie sent in the Cookie header on the request to /ghost/api/v2/admin/users/me/?include=roles?

I don't see any cookies being sent back from the server, upon just loading the sign in page it fails with a 401 error code.

As for cookies, they are on and work since rolling back yields a successful result.

2.1.3 doesn't use cookies so that's not an indication that cookies are working.
"Failed to load response data" in Chrome suggests you may be using the preserve log function? It won't show you the response data for requests after you navigate away so it may be a red herring. Could you try again? Do you see any redirects happening?
Do you get any errors in the DevTools console when trying to log in? If for some reason CORS is tripping up then there would be an error there.

I'm using my MacBook at home whereas I was on my iMac at work earlier and the issue is persistent so it's not a browser issue on my end.
For quicker debugging from your end, below is the URL for my sign in page.
https://cshaw.tech/ghost/#/signin

@cshawaus the 401 error is expected on initial load because you won't be logged in after upgrading to 2.2.3, the 401 is what tells the admin area to show the signin screen.
Can you clarify that you're unable to log in with your username/password?

Yep, same username and password that works if I rollback fails with 2.2.3.

For quicker debugging from your end, below is the URL for my sign in page. https://cshaw.tech/ghost/#/signin

Your sign in page is behaving as expected but we're not able to test the login because we don't have the username/password (please don't share that!)
Can you share the exact reproduction steps you are trying? This is what is expected after upgrading even if you were logged in on 2.1:

Load you admin page https://cshaw.tech/ghost/

you'll see a 401 in the network requests for /ghost/api/v2/admin/users/me/?include=roles this is expected

You're redirected to https://cshaw.tech/ghost/#/signin
You need to put your username/password in and click on "Sign In"
A POST request will be made to https://cshaw.tech/ghost/api/v2/admin/session

response should be a 201
if you inspect the response it should have a set-cookie: ghost-admin-api-session=.... header

If you don't see the set-cookie header please let us know what you see in the request's origin and referer headers.

Here are the request headers.
origin: https://cshaw.tech
referer: https://cshaw.tech/ghost/

Looking at the response for /session, there is no set-cookie header attached to it. Below is the complete headers I get back from nginx.
cache-control: no-cache, private, no-store, must-revalidate, max-stale=0, post-check=0, pre-check=0
content-length: 7
content-type: text/plain; charset=utf-8
date: Thu, 18 Oct 2018 12:00:10 GMT
etag: W/"7-rM9AyJuqT6iOan/xHh+AW+7K/T8"
server: nginx
status: 201
vary: Accept-Encoding

Looking at the response for /session, there is no set-cookie header attached to it.

This is the problem, maybe your nginx is not configured to forward Set-Cookie headers? Can you think of anything else between the node process and your browser that might remove that header?
Maybe you could share your nginx config here?

Now I feel dumb, turns out I had proxy_hide_header Set-Cookie; set for the ghost and signout paths. Not sure if I put it there a year or so ago or I copied the vhost from something else but all is good now.
🤦♂

:dancer: Glad you can sign in now!


New ind_11 Inconsistent contract objects across the crisis

text mining brainstorm
there are a number of techniques that may do the job, but each of them one of more of the following:

computationally expensive
bring large dependencies
not really designed for Italian lang 🇮🇹

As a consequence we (@giuliogcantone and me) tried to figure out what we can do, these are some of the proposals:

sort of topic mining on contract objects compared to a manually annotated dictionary from domain experts ❌
0-shot classification (italian using roberta) on very specific critical terms that may let us think there's something suspicious with it ❌
from cpv exact description and compare against the objects over a number of similarity measures (Levenshtein, Jaccard etc) ✅

We are currently investigating the third solution, but we are really open to discuss the other two.

I am working to build a pipeline that build a dictionary on a traning set.
Preliminary work is here:
https://github.com/CORE-forge/coresoi/commit/18d71117c7cb6a9f69cfe9f97f3109ea80d4fb7b
Some main issues must be solved in a certain order:

Explicit categories of contracts are not a good classification method; they are too much and too thin. To solve this, probably we need a different level of aggregation through cpv.
Tokens must be weighted (tf/idf).
I am working on a mock training set. A proper training set should be identified.

On .3, consider this:

If all data-at-disposal are employed to build the dictionary, any use of ind_11 would just reflect 'divergence from standard patterns' of text, which is still a-ok propriety for a semantic indicator, but strictly speaking a textual divergence is not sufficient to be characterised as a semantic difference. Example: a contracting agency bought some bathrobes, but the contract was characterised as bath products. This is a weird bath product, but a reasonable semantic, e.g. not suspicious.
I do not think CORE needs it but a dedicated approach would process the semantic of the tokens with a pre-trained model.
If all data-at-disposal are employed to build the dictionary, any bias (i.e. corruption attempts) that is clustered (i.e. it happens more than 1, it's a scheme) will be processed as a standard pattern. However, by how I intend to build the dictionary, standard patterns must be really very frequent, hence the risk of false negative is minimal at cost of high risk of false positive in of the test ('low power'). It means that this ind_11 will be frequently set on 1 (ON). This is fine, given that this is a system of multiple 12 indicators. It could mess with weighting schemes of a composite indicator, tho.
If only safe-data (e.g. when other indicators are set on 0) are employed as training set, then ind_11 will be stochastically collinear by-design with the previous 10 indicators.

The last 2 points IMO are solved adopted a FA/PCA approach for the composite indicator, which polishes the multivariate structure from multicollinearity.

I also want to know more about roberta and what exactly it (she?) could do; is it pre-trained right? Because if we need to train it, it would incur the same issue as above.

macroclass
n of contracts
gini
atkinson

3
430
0.267
0.1

9
9188
0.757
0.517

14
455
0.297
0.113

15
2923
0.539
0.261

16
226
0.347
0.14

18
2091
0.491
0.232

19
575
0.385
0.162

22
2622
0.535
0.264

24
2427
0.503
0.253

30
10799
0.69
0.438

31
5429
0.551
0.295

32
4780
0.598
0.328

33
187380
0.829
0.628

34
13941
0.652
0.384

35
2301
0.471
0.225

37
658
0.467
0.206

38
4494
0.513
0.264

39
5018
0.568
0.311

41
470
0.293
0.1

42
5902
0.626
0.383

43
516
0.345
0.126

44
10078
0.644
0.386

45
557834
0.916
0.776

48
4359
0.585
0.325

50
228193
0.959
0.886

51
980
0.366
0.137

55
9027
0.727
0.479

60
6170
0.65
0.393

63
3170
0.531
0.258

64
3744
0.621
0.342

65
6453
0.684
0.438

66
6627
0.732
0.486

70
317
0.279
0.083

71
61628
0.823
0.615

72
18364
0.735
0.491

73
1331
0.369
0.142

75
1520
0.446
0.192

76
300
0.245
0.078

77
7378
0.708
0.46

79
23322
0.741
0.494

80
4414
0.612
.0343

85
26699
0.762
0.521

90
32129
0.773
0.538

92
6215
0.597
0.323

98
32351
0.885
0.718

code and business logic in 18d71117c7cb6a9f69cfe9f97f3109ea80d4fb7b and 9c80688b719950eb5b9db320ca7543d0e6988af1. Then some other chunks here and there!


Resolves #235: xcode 9 beta 5 deprecation fixes: truncatingIfNeeded + string slicing

What's in this pull request?
Some small updates to adjust to deprecations between Xcode 9 Beta 4 and Beta 5.
Namely:

init(extendingOrTruncating:) has been renamed init(truncatingIfNeeded:)
myString.subscript(with:myRange) is now String(myString[myRange])

Why merge this pull request?
Resolves #235. Necessary for Xcode 9 Beta 5 support on the swift-develop branch.
What's worth discussing about this pull request?
This is a pretty straightforward change.
What downsides are there to merging this pull request?
None that I am aware of.

This looks good, thanks! Would you mind trying to fix the CI errors?

That should do it! It looks like the issue is simply that Apple changed the directory for Swift 4 snapshots. I updated it to the newest snapshot, which is swift-DEVELOPMENT-SNAPSHOT-2017-08-03-a-ubuntu14.04.tar.gz.

I am seeing the following error, but only for the Linux build. On MacOS, all checks are passing.
/home/travis/build/typelift/SwiftCheck/Tests/SwiftCheckTests/SimpleSpec.swift:13:9:
error: cannot convert return expression of type '(A, B) -> C' to return type '((A, B)) -> C'
        return f
               ^
/home/travis/build/typelift/SwiftCheck/Tests/SwiftCheckTests/SimpleSpec.swift:17:9:
error: cannot convert return expression of type '(A, B, C, D, E, F, G, H, I, J, K, L, M, N) -> O' to return type '((A, B, C, D, E, F, G, H, I, J, K, L, M, N)) -> O'
        return f
               ^

LGTM

⛵


isomorphic-fetch

This is in the wrong directory definitions/iso... vs definitions/npm/iso...

Again?... I'm sure I moved it... I sure fail at git quite often :/


Roberts/Mock data version 2

Description
Matt and I worked on making the mock data more relevant for the task at hand. We discovered at the stakeholder meeting that the matching process is relatively simple: the exp_level and skills(tech_stack) are used primarily, with other features like time zone, gender, general interests are used as a secondary matching feature.
Loom Video:
https://www.loom.com/share/98a98d86ac544ed5b201a6351800943b
Co-Author:
Matt Clibanoff
Type of change

[x] New feature

Checklist:

[x] My code follows PEP8 style guide
[x] I have removed unnecessary print statements from my code
[x] I have made corresponding changes to the documentation if necessary
[x] My changes generate no errors
[x] No commented-out code
[x] Size of pull request kept to a minimum
[x] Pull request description clearly describes changes made & motivations for said changes

Looks great! I think the additional fields will help the matching process.


'openssl rsa -in rsapss.key -check' not working for RSA-PSS keys with OpenSSL 3.0

The OpenSSL rsa command line tool does not work anymore with RSA-PSS type keys in OpenSSL 3.0:
# openssl genpkey -algorithm RSA-PSS -pkeyopt rsa_keygen_bits:1024 --out rsapss.key
# openssl rsa -in rsapss.key -check
shows 'Not an RSA key'

This is caused by the following code in apps/rsa.c:
    if (!EVP_PKEY_is_a(pkey, "RSA")) {
        BIO_printf(bio_err, "Not an RSA key\n");
        goto end;
    }

This came in with commit https://github.com/openssl/openssl/commit/d7e498ac55f12bc2f4e7f948cbb8de2e3eeafc74
I guess it should check if the key is either "RSA" or "RSA-PSS" and accept both.

grepping the codebase reveals a few other locations which at least look suspicious:

I am going to review these places in next sprint.


TI CC2650 SensorTag Data Reading Problem

Hi,
I want to create a sensor network using your library. I installed also noble and node-gyp accordingly. I use a bluetooth 4.0 device with Ubuntu. I have encountered some problems as I tried to read data from sensors. Actually no data from SensorTag can be read.
To get a better understand of the process I used your "test.js"
I was able to discover my device, its id and type. However after ConnectAndSetUp is done, no further functions are called. And I have no idea why it is happening, (I am also new to JS and Node.js)
Here is the screenshot of the process:

Here is the hcidump -t -x output:
Seems a bit problematic, since no events here is published, but I don't know why.

I'll be glad if someone can help me with this problem.
Regards.

I solved the problem.
It was a problem about bluetooth. My device was connecting sensor automatically, before I run the program. I added then deleted my device on bluetooth device list, after then it started to work.


New documents don't automatically get added to the graph

Loving your work on this extension!
It would be cool if a new document could get added to the graph automatically.
Consider a use case:

I am writing a document while the graph is open
I want to create a new document, called LInk
I type [Link](link.md)inthe current document and create the file in the directory (possibly via an automation)
I give the new file a # Heading

Expected outcome: New document "Link: shows up in the graph.
This is definitely a nice to have, just logging it here.

Sorry for late response.
I believe that I implemented a mechanism for watching files in the workspace and what you've described exactly matches the intended behavior. I will have another look what can cause problems with it.


commit a change to see if we get codecov to send happily

Description
Include a summary of the changes this PR is bringing. Bring in any relevant context or motivation for the change that you deem useful to have in this PR description.
Example:
The aim of the PR is achieved by:

doing this amazing thing
this other one
deprecating this thing
changing this thing

How has this change been tested?
Supporting doc, tickets, issues
Merge Instructions (Optional)
Note to reviewer (Optional)
I'd like the reviewer to focus especially on ...

closing this because the testing is satisfactory

closing this because the testing is satisfactory


Slash command needs description, but addSlash not provide argument

Describe the bug
As reading source, this line
in addCommand needs description, but addSlash (link) doesn't provide argument for description.
Steps to reproduce
From example of dimscmd provides the code: cmd.addSlash("add") do (a: int, b: int):
Expected behavior
we should be able to add argument for description or make description is not required.
Screenshots

Additional context
Maybe am I missing something or I have to provide description via parameters? Sorry if I didn't understand it correctly.

Best bug report I've every gotten! thank you for taking the time to get all the info (Might turn this into an issue template)
But basically the description for chat/slash commands is added with doc comments which are like comments except you use two pound signs instead of one (## vs #)
cmd.addSlash("test") do ():
    ## This is the description
    doStuff()

Ohhhhh is that description? I didn't know that, thanks!

The description requirement is a bit annoying but is required by discord

The description requirement is a bit annoying but is required by discord

Maybe your library could either make an error message saying that you either need to provide a comment?

it provides the assertion defect but I guess that isn't clear that you need to provide a doc comment
I think I can attach an error directly to the addSlash line with some ast manipulation 🤔


fix to handle method signatures with types like Map<String,dynamic>

I went through the Dart_sip_ua example and added types.
EventEmitter isn't able to handle type information like Map<String,dynamic>.
It incorrectly determined the number of parameters from Func(Map<String,dynamic, String) as three in stead of two.
I've added some code to remove the information between < and > which fixes this problem.

LGTM


Can I set newlines in commentstrings?

I want my block comments for lua to be like this:
--[[
some commented-out text or code
--]]

So I thought it would be easy to just do something like this:
require("nvim-treesitter.config").setup{
    ...
    context_commentstring = {
	...
        config = {
            lua = {
                __default = "-- %s",
                __multiline = "--[[\n%s\n]]"
            },
        },
    },
}

but when commenting a block of code with Comment.nvim the newlines are replaced by spaces.
Am I doing something wrong, or is it just not possible to have newlines in a commentstring.
This is a link to the specific code in my dotfiles repo.

Hey!
I tried this out myself too and couldn't get it to work with the Lua multiline strings either:
require('Comment').setup {
  pre_hook = function()
    return [=[--[[
%s
]]]=]
  end,
}

I think that this plugin can't do much about this though since the commenting plugin needs to handle the commentstring with new lines.
I found this issue in Comment.nvim that might be similar to your use case: https://github.com/numToStr/Comment.nvim/issues/38 Maybe this helps?
But you could open an issue/feature request in the Comment.nvim repo since I don't think we could do anything about this in this plugin.
Let me know how it goes!


The first candlestick in the figure is cut in half

I copied the example from http://bl.ocks.org/andredumas/edf630690c10b89be390
It seems the first candlestick is always cut in half. It only happens when zoom is used. Without zoom functionality, it works fine. The example is cut in half as well, but it is too small to see clearly. In my example, it is much clearer:

You see the first red bar is only half the size.
Can that be fixed?
Thanks.

Hi @sf-wind
What you're observing is currently by design. If you print out the zoomable state on that plot you will see the right extent is not a whole number, where the left extent is.
console.log(x.zoomable.domain());

All techanjs plots consider the midpoint of the plot object (such as candlestick) as the mapped x point on the range. So setting zoomable extent to an integer as is done in that example will cut the candle in half. This was a key decision in the early days of the library and has simplified other calculations and render logic. In order to get a full candle you could zoom an additional half index point:
// Cuts candle in half
zoomable.domain([indicatorPreRoll, data.length]);

// Starts the plot between candles, getting a first full candle
zoomable.domain([indicatorPreRoll-0.5, data.length]);

Note that on first initialise of the domain, the viewable domain is widened via the financescale.outerPadding variable. When setting a zoom extent, this is no longer applicable as the zoom takes over.
andre

Closing due to no activity.


Modify role icons

Summary
With the addition of role icons in 3.1.2 there were no ways to modify or remove role icons. This should be added into the next release

API documentation can be found here


add support for files that dont have command lines

Tools like https://github.com/rizsotto/Bear produce compile_commands.json files which do not have 'command' strings they instead have an 'arguments' array.
This change allows json2cmake to accept these files without issues. An example of a compile_commands.json with this change can be found here https://gist.github.com/jfmherokiller/c4f7b288bbf44a464098ef7978815869

Sorry for taking so long to get around to this, I only just noticed it!

This is now fixed in the latest released version, available on PyPI.


Fix slot names that start with call

When a component is registered that starts with call the compiler
detects it as a template method which is not correct. This changes the
compiler to look for call(_|$) instead of call to avoid this
issue.
This means that slots can't (and couldn't) start with call_ since we
still rely on the call_ naming convention to generate the
template methods. To make this more dev friendly, this adds a check
that raises an error if a slot name starts with call_.
A more long-term fix would be to use some kind of template method
container instead of relying on the call_ naming convention.
Fixes https://github.com/ViewComponent/view_component/issues/1825

@BlakeWilliams maybe adding the fact that slot names cant start with call should be added to known issues in the docs so users can better troubkle shoot this issue going forward

Minor nit, could add a test that the #1825 case is fixed:
def test_slot_names_can_start_with_call
  assert_nothing_raised do
    Class.new(ViewComponent::Base) do
      renders_one :callhome_et
    end
  end
end

Otherwise LGTM 👍

@BlakeWilliams any progress on this ?

@reeganviljoen I can take this one over and get it across the finish line 😄

I haven't had a ton of time to dedicate to this recently and documentation is always the hardest part. 🙂
Thanks for the ping @reeganviljoen and @camertron for getting this shipped! 🚀


Update test-action.yml

Summary

Make sure that all actions are using commit hashes.
Ticket Link

Fixes
Make sure that all actions are using commit hashes.
Release Note

Signed-off-by: Jauder Ho<EMAIL_ADDRESS>

@jauderho please sign the DCO :)

Done

@jauderho thanks for updating the ci and I see now you updated the action itself, can you please update both PR title and summary to reflect that?

just remember now the cosign was already updated in this pr https://github.com/sigstore/cosign-installer/pull/53

@cpanato I think it is because I had not yet pulled in your changes into my branch. I have now done so and if you look again, it's once again just the test-action.yaml change again.


API review tool feedback part 1

Rename property sentimentScores to confidenceScores
For LinkedEntity, rename id to dataSourceEntityId and getId() to getDataSourceLinkId()
Rename the class SentimentScorePerLabel to SentimentConfidenceScorePerLabel to reflect swagger naming.
Delete all intermediate WithResponse suffixed and Update APIs
Unroll defaultCountryHint and defaultLanguage to builder API and remove TextAnalyticsClientOptions.
Remove InnerError classs
Change all input List to Iterable

Are there any tests to make sure that defaultCountryHint and defaultLanguage are properly use after

Unroll defaultCountryHint and defaultLanguage to builder API and remove TextAnalyticsClientOptions.

Are there any samples or code snippets that can show the user how to set, use the default country hint and default language?

@maririos Created two issues: #8352 and #8353 to address two of your feedbacks


Add option to filter passed message

I only care errors. There are too many passed messages, make it difficult to find errors.

Hey, in upcoming 2.0 release our cli tool will show only errors. Thanks for the issue.

Cool. Thanks


Mrc 5604 single status endpoint

This adds the endpoint for the single status. Main changes include:

get endpoint status/{taskId}
adding new field queue position to run info table
status in sql file has an extra NOT NULL after discussion in #110
TaskStatus data class, this is the orderly.runner response to status request

How to test:

./scripts/dev-start --super-user (with basic login enabled)
POST super user email and password {"email": ..., "password": ...} to http://localhost:8080/auth/login/basic to get the token (need this to make any requests to runner controller because it requires the packet.run permission), send the remaining requests with this bearer token
GET http://localhost:8080/runner/git/branches with token to get the commitHash (you'll need this later)
POST

{
  "name": "parameters",
  "branch": "master",
  "hash": <YOUR-COMMIT-HASH>,
  "parameters": {
    "a": 6,
    "c": 28
  }
}

to http://localhost:8080/runner/run you will receive a taskId
5. GET http://localhost:8080/runner/status/<YOUR-TASK-ID> this should say completed status by this point and give the appropriate logs and details
OR you can wait for the FE so it will be easier to interact with it

Note have run lint and tests on local, we have reached pull rate limit RIP

What does the runner give back as queue position for a task which has already started, or even finished? Does it revert to null?

it reverts to null yep


CVE-2013-0867 and CVE-2013-2277

Adding researched information about vulnerabilities

Make sure to add upvotes: 0 to your CVEs if they didn't receive any upvotes


Jenkins login issues  account not populating

Problem with this page
Your Profile | Jenkins page
Expected behavior
The expected behavior is the accounts page was supposed to populate  where I can build check builds added webhooks for my MS Teams account and add the connector for Ms teams It is not giving me the account page
Actual behavior
Hello I installed jenkins on a new server with azure Iadded my azure crditials at log in for the jenkins password  I have a differnet pass word for the login page for jenkins it would not let me added it when I was finally able to get logged in but it will only populate the the profile on my page
My Account
Profile
Logout
Account
You can create/manage your user account that you use for accessing Jira, Artifactory, VPN and other services within the Jenkins Infrastructure.
Profile - View and edit profile, change password, etc.
Logout
Possible solution
I had another server that i terminated with azure stated another which is the one im working from now the first was not acting as expected i could only see that builds were successful but no information populated It was done with my fresh works account
although I did have the account page

Hi @carolynrenahuggins , this repository hosts the code for the application served under https://accounts.jenkins.io whihc which allos managing account for the jenkins.io websites (mainly Jenkins's source code contributors and plugin maintainers).
It's not aimed for individual Jenkins instances at all: it eplains why the informations you've put in accounts.jenkins.io are not visible in your own Jenkins instance.
As such, I'll close this issue as it is not a problem related to the account-app application code, neither with the Jenkins infrasructure (*jenkins.io websites).
You should open a thread in the community forums at community.jenkins.io to get started and ask fro help in setting up your Jenkins instance.
Please note, that if you follow the  Jenkins configuration wizard, a default admin user is created (see https://www.jenkins.io/doc/pipeline/tour/getting-started/).
Jenkins has its own authentication realm for storing users by default, but can be extended to delegate authentication to an external system, using plugins from https://plugins.jenkins.io, such as Azure AD, SAML, LDAP and many more.
Also, you might be interested by https://www.jenkins.io/doc/book/security/. Good luck in your journey and do not hesitate to ask in the community forums!
aks-syspool-23443997-vmss000004-privatek8s

Thanks for your response
On Fri, May 19, 2023, 3:04 AM Damien Duportal @.***>
wrote:

Closed #269 https://github.com/jenkins-infra/account-app/issues/269 as
not planned.
—
Reply to this email directly, view it on GitHub
https://github.com/jenkins-infra/account-app/issues/269#event-9286853086,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APWOMAEX7DSVZHE4LU2KLGLXG4SQ3ANCNFSM6AAAAAAYG7XGTU
.
You are receiving this because you were mentioned.Message ID:
@.***>


Match broke after changing Apple developer account password

New Issue Checklist

[x] Updated fastlane to the latest version
[X] I read the Contribution Guidelines
[X] I read docs.fastlane.tools
[X] I searched for existing GitHub issues

Issue Description
I changed my apple developer account password. And now I get,
[!] Invalid password passed via 'MATCH_PASSWORD'

Do I need to regenerate the MATCH_PASSWORD ???


Add option to ignore CI runs on commit

Right now there's no need for us to waste quite a lot of CI resources after diagram.svg is commited.
Ideally I would want to have [ci skip] tag in a diagram commit message.
But, I understand that this might not work for every project, so this looks like good candidate for an option, something like with: skip_ci: true
What do you think?
Btw, our diagram: https://wemake-python-stylegui.de/en/latest/pages/api/index.html#overview
🙂

Yeah for repos with CI, this is a must have! 👍

I just ran into this.
The current example setup, when combined with user token branch protection bypassing, seems to trigger an endless loop. Which can definitely have some bad effects on subscribed CI/CD pipelines.

Great call! I added the ability to set the commit message, which will allow users to add [ci skip]. Let me know if this misses a use case!


After deploy I am facing Network Error

When I deploye this app at Galaxy.meteor I am getting the following error logs :

And output screen showing as following

Environment variable in my settings.json file
"galaxy.meteor.com": {
     "env": {
       "ROOT_URL": "http://www.streetail.co.uk",
       "MONGO_URL": "mongodb path"
     }
   },

How to fixe?

I found my mistake in ROOT_URL.
I have fixed this issue using https instead of http in root_url


Added test for other supported data type test in ActionTestController

Added unit and integration test for other supported data type test in ActionTestController

This change is

Coverage decreased (-0.03%) to 59.687% when pulling 605644b66c8e0783fcfaaa8ac5f7bb69c89948a1 on AddTestsToActionTestController into 2ac767876e28b759fdffb867e8a855ccfcde7497 on master.


Fix inconsistent null/undefined usage in OneOf block

alternative to https://github.com/vivid-planet/comet/pull/1060

In existing blocks, activeType can be null

yes, that changes with with PR. But that didn't match the types anyway.

I've tested this locally with existing blocks it works perfectly fine.
But I've noticed something very interesting: The activeType is still existing in the database and has the value null. I did some further digging and came to the conclusion that this caused by a) our transformToSave returning { "key": undefined } for undefined values and MikroORM (or Postgres) converting { "key": undefined } to { "key": null } in order to be valid JSON. We could fix this by omitting undefined values in transformToSave (I've already successfully tested that) – but should we?

We could fix this by omitting undefined values in transformToSave (I've already successfully tested that) – but should we?

yes, I think we should. Either we use null (and type accordingly) or undefined. There should be no mixture where types are not 100% correct.
(I also changed that - didn't test though - I hope my implementation is similar enough to yours)

(I also changed that - didn't test though - I hope my implementation is similar enough to yours)

This should work for the OneOf block. I was rather thinking of a general solution in transformToSave

good idea, I force pushed away mine, please add yours.

good idea, I force pushed away mine, please add yours.

Done


Option to ignore merge keys?

Right now, when you try to sort a YAML file that contains a merge key using, e.g., a YAML alias, you get the following error message:
test.yml:8: '<<:' references are not sortable:
  <<: *foo
  ^~~

An option to ignore such merge keys would be nice to have. Now YAML files containing merge keys cannot be sorted at all.

I guess we can do some sorting, e.g. when given:
data:
  d: d
  a: a
  <<: *foo
  c: c
  b: b

we can generate:
data:
  a: a
  d: d
  <<: *foo
  b: b
  c: c

That is sort each group of items between the beginning of a dictionary, the end of a dictionary, or a reference. << would act as a stop token when sorting.
Would that work for you? Do you see any pitfall?

@smortex I think I would prefer to sort the merge keys to the top of the hash, then sort the rest. So something like this:
data:
 <<: *foo
 a: a
 b: b
 c: c
 d: d

Reason being that we do not know exactly what is being merged, plus I think it's the idiom to keep merge keys to the top.
@OtherCroissant what do you think?

In that case, that would break things like:
foo:
  a: 42
data:
  d: d
  a: a
  <<: *foo
  c: c
  b: b

here, data.a = 42, but with your sorting, data.a = "a":
foo:
  a: 42
data:
  <<: *foo
  a: a
  b: b
  c: c
  d: d

Also, the "correct" sorting should output data before foo, and this reordering is not possible with merges… Maybe there is no reliable sorting mechanism that can exist for this use case…

@smortex ah, you are right, I did not consider the overriding of existing keys. Then yeah, your proposed solution is the best one.


Add Font Weight option to config

Along with setting the font family and size, there's currently no way to set the font weight without adding custom css. I hope you can support this in the future.

In the meantime, here's a workaround:
termCSS: 'x-row {font-weight: 100}',

Any idea on how to apply it on directory font weight?

I'm also still looking for a way to only apply this on directory font weight..

@matthewconstantine I tried adding font-weight property . But it seems like the hterm package doesn't support font-weight property. Should I overwrite hterm's package set method? Any other way around it? Any suggestion would be appreciated.

termCSS: 'x-row {font-weight: 100}', is deprecated in Hyper 2.1.1 (canary)

Working in 2.0.4 (stable)
termCSS: '.terminal {font-weight: lighter} .terminal .xterm-bold {font-weight: normal}'

Added in #2669.

@brandon93s Thanks!


Warrior specs implementation

Also includes some spell scaling coefficient helpers in utils.go as well as a new Multiplier field on core.Cooldown and a corresponding SpellMod.
Warrior has effects that modify CDs by flat amounts as well as percentages (on the same ability, see Shield Wall's glyph and the T11 tank set). A previous implementation that multiplied CD.Duration directly was not safe to use alongside Cooldown_Flat as the final CD would be dependent on the order the mods were applied in ((5min*0.5) - 2min = 30sec,  (5min - 2min)*0.5 = 1.5min). The multiplier is currently accumulated additively to allow reaching 0 CD, though I don't believe it's possible to have multiple CD reduction multipliers at once in Cata.
The spell scaling helpers are really just an effort to get them somewhere they can be found so we don't need to keep referencing random DB tables and discord posts for an already obscure mechanic. Their actual uses are waiting on the beta build actually getting the missing variance field filled in.

The autoformat in VS code seems to have poked some files I didn't actually edit


CSV Imports don't work in certain scenarios due to specific content type requirements

Which version of Skybrud.Umbraco.Redirects.Import are you using? (Please write the exact version, example: 4.0.1)
4.0.1
Which version of Skybrud.Umbraco.Redirects are you using? (Please write the exact version, example: 4.0.15)
4.0.17
Which Umbraco version are you using? (Please write the exact version, example: 10.1.0)
10.7.0
Bug description
When uploading a csv file there is a conditional check for the content type which checks it is "text/csv" in the CsvImporter file.
However, I noticed that in most cases the validation was failing due to the content type differing from "text/csv".
Further digging further it seems common types that are used for CSVs are:

text/csv
application/vnd.ms-excel

But this could also extend to:

text/plain
application/vnd.ms-excel
text/x-csv
application/csv
application/x-csv
text/comma-separated-values
text/x-comma-separated-values
text/tab-separated-values

I'm happy to raise a PR that includes validation for the below content types as these seem to be the most common or I can extend to also include the other types but I haven't tested these:

text/csv
application/vnd.ms-excel

Hi @jawood1
Can you elaborate about in which cases you've experienced that the content type isn't text/csv? I haven't come across this, but I'd imagine different browsers and OSs could provide different content types.

Hi @jawood1
I cant really say for Apple based systems, but I can't reproduce any issues on Windows 10 and 11. In both cases, I have .csv associated with Excel, but still the mime type comes up as text/csv.

Generally application/vnd.ms-excel should be used for .xls files, not .csv.
Can you share how .csv files are associated in your Windows setup?

Hi @abjerner
My CSV files were associated with Excel like yourself.
Oddly I don't seem to be able to re-create the issue now since I've had my work laptop rebuilt at the end of last year.
Though it was definitely was the case for myself before hand that application/vnd.ms-excel was the content type being seen when:

Creating a CSV within Excel
Creating a CSV outside of Excel
Converting a Excel Spreadsheet to CSV

Perhaps it was some issue with my machine or some weird edge case?

@jawood1 thanks for the reply 👍
My guess is that your file association has been wrong on your old system then. I'll try doing some more research as I'm also not sure what Apple based systems report as the content type for CSV files.  Ideally they should also report text/css, but they widely report text/plain instead, it could possibly make sense to include this content type as well. Or make the content types configurable.

Great stuff, I think it could be a good solution perhaps to have configurable content-types for edge cases like this.
I found an additional source here that states some of primary content-types used, however it doesn't mention text/plain or application/vnd.ms-excel.
https://www.digipres.org/formats/sources/ffw/formats/#csv
In the case of Apple devices it seems text/plain may occur due to how the files are formatted from what I've seen, but I can't verify if that's a theme in Apple devices or if it's another edge case scenario.

Closing this issue as the content types should now have been addressed by https://github.com/skybrud/Skybrud.Umbraco.Redirects.Import/commit/dcf7e98213ffd3dc12e0e96b4af327d426e71048


Feature: S3 Object Storage

Hey all. Few days came across Saltcorn for simple no-code platform, and I find it really promising yet simple for minimal small-business use. I am planning to use Saltcorn for my use case. However, I was wondering if AWS S3 (compatible) file storage can be implemented as well.
Since no AWS S3 plugin available yet, I tried to look at the code directly, wondering if I can make a plugin for S3. But I found that the file upload mechanism is tightly coupled to the middleware and currently no access from the plugin realm.
So I decided to experiment and develop a quick-and-dirty implementation for S3 object storage. I use AWS-SDK combined with Multer to handle the multipart-form request and forward it to S3 endpoint. I also modified the route for download, serve, and delete to use S3 handler if it is configured. Additionally, I created a new settings page to enter S3 config and to enable/disable S3 file upload.
I overlayed the express-fileupload middleware to use Multer instead, because there is a middleware to forward multipart-form request to S3 directly. If we stick to use fileupload, the file will be stored first locally before being forwarded. It is too inefficient compared to passing through the multipart request to S3.
I also created some "hack" function to let the existing file upload flow intact, particularly the part where Saltcorn moves the file from temp folder to the designated local folder. Also, I modified the delete function in the File model to detach the unlink function and able to use S3 deleteObject function.
In addition, I added additional column in files table to indicate if a file is stored in S3 or locally. Therefore, enabling/disabling S3 store on the fly does not break existing files.
You can check the code here: https://github.com/gilanghamidy/saltcorn/commit/1baf6730702a3609cb7951cd903c07c2b0dbb25f
This is a totally quick and dirty implementation, but it is working correctly. I haven't made any test fixture for this, since I don't understand yet. I am also not sure if this work in multitenant scenario as I haven't try it yet.
Basically, I made this in just half-day, including learning to code NodeJS (I am not a JS dev, more to C++ and C#), configuring my dev machine with WSL, and reading all docs and StackOverflow to craft this. So pardon me if the implementation is not that good :)

@gilanghamidy thank you for doing all this work! I will try and get this merged in the next few days with an option for storage on S3.

Sure! I have cleaned the code, and removed an unnecessary commit (I think) to npm-shrinkwrap.json. I'll make the official pull request so you can take a look :)

Wow! I don't use S3 but thank you @gilanghamidy!
Bonjour à la Belgique :)

this is merged in #1001


chore: add e2e tests and demos

Summary
Add demos and e2e tests to the default site for a bunch of next features.
Namely:

Environment tokens
Font handling
Routing
CSS handling
Layouts
TypeScript
Scripts
Images
Middleware

Test plan

Visit the Default Deploy Preview (/ should work even though it's typescript)
Go to /css
Go to /script
Go to /layouts
Go to /env
Go to /font
Go to /broken-image (expected broken page)
Go to /middle
See the default demo cypress test run

Example new demo page:

Relevant links (GitHub issues, Notion docs, etc.) or a picture of cute animal
Standard checks:

[ ] Check the Deploy Preview's Demo site for your PR's functionality
[ ] Add docs when necessary

🧪 Once merged, make sure to update the version if needed and that it was published correctly.

The errors should be fixable by regenrating the package lock

So close! Needs npm run test:update

Arghh yeah thanks for the reminder

cries

these snapshots are gonna be the end of me

So the order of the redirects in the snapshots is based on the order in the prerender manifest. This seems to imply that the ordering there is indeterminate. That's a bit of a nightmare!


If the project FLASH ORIGIN was changed, the Elf program sections will still be loaded in the old addresses

Create any STM32 project
Open mem.ld linker file
Changes ORIGIN attribute of the FLASH to anything
Run this program and see how the old ORIGIN is still intact

When linking, use the linker argument -Xlinker --nmagic in project settings.


Fix rustfmt problems

This PR fixes two problems with the current code related to rustfmt:

The Travis config still uses the --write-mode rustfmt command line option which has been replaced with the --check flag in the current version of rustfmt.
The rustfmt check fails with the current version of rustfmt, probably because some format rules were updated recently.

Thanks for this :+1:


Randomise Level Layout

Currently the rooms are all layout very evenly with every room interconnected.
Preferably the rooms would randomly sized and in a random layout.

fixed in 9113faec95763b466cac00ea5f23e6f498131c29


Enhance cilent pub sub tests

Fixed tests that will produce duplicate PUBLISH packets in directly push to a QoS 2 client. #281
and state packet type in Broker ack event #257

Replaced PR https://github.com/mcollina/aedes/pull/273


Merge branch 'master' into feat/import-feature

Backmerge master updates

useless, feature has already been merged to master


fix boarder fix #1920

Fix problem with render boarder. Variable side is increment only if condition is true. But if you have only boarder-top and boarder-bottom, them boarder-bottom render it like border-left.

Hi, when will this PR be merged in?


Abstract duplicate Edit methods to separate EditUtil class and add defensive programming for Schedule

Codecov Report

Merging #35 (4fe3484) into master (b117639) will not change coverage.
The diff coverage is 36.29%.

@@            Coverage Diff            @@
##             master      #35   +/-   ##
=========================================
  Coverage     60.16%   60.16%
- Complexity      413      414    +1
=========================================
  Files            80       82    +2
  Lines          1554     1554
  Branches        168      172    +4
=========================================
  Hits            935      935
  Misses          581      581
  Partials         38       38

Impacted Files
Coverage Δ

...java/seedu/address/logic/commands/EditCommand.java
100.00% <ø> (+4.10%)
:arrow_up:

...dress/logic/commands/schedule/AddEventCommand.java
0.00% <0.00%> (ø)

...ress/logic/commands/schedule/EditEventCommand.java
0.00% <ø> (ø)

.../seedu/address/logic/parser/EditCommandParser.java
92.30% <ø> (ø)

...u/address/logic/parser/EditEventCommandParser.java
0.00% <ø> (ø)

...in/java/seedu/address/model/schedule/Schedule.java
51.85% <ø> (+6.69%)
:arrow_up:

.../seedu/address/logic/commands/EditTypeCommand.java
18.91% <18.91%> (ø)

src/main/java/seedu/address/logic/EditUtil.java
45.78% <45.78%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b117639...4fe3484. Read the comment docs.

lgtm


Reduce occurrence of Star and Gas giant prefs

Stellar preference and such are occurring too often. Need to find a way to regulate it, like the normal traits. Hab traits are currently not assigned through custom event but by some hidden game mechanic which probably needs to be killed somehow.
Apparently hab traits now have an 'ai_weight'. Might be usable. Going to plug in conditional modifiers as well and see if the game responds to that.

Can't be regulated as planet selection and placement is done before game start. Hab picks are random with no way to influence or weigh picks. Applying partial fix only; custom made races still provide a solution for people who want their species to be perfect.

Reopened. Going to make a post-gamestart regulation event to enforce some planet diversity.

Wrote completely new system based on chain of events on game start, to re-assign some species planet preference. Not working, for unknown reasons. Game 'cannot find event' even when triggered using console. Encountered that one before, but not sure what causes it; events are based on the template for trait regulation events, which do work.

I was able to execute all 4 of the planet regulation events.

See that's really weird. You can fire the events but my game can't fire them. I've encountered this bug before but I forgot what it was about. It's like the game makes its own 'event cache' that refuses to update sometimes.
Anyway good to see it's working somewhat. I won't be able to fix it entirely for as long as my game acts weirdly. Robots with traits are intentional btw. In the Heavy version they need those too because they don't get a straight up +200% hab bonus just for being mechanical anymore. I do intend to separate their hab traits from the organics though.

Planet regulation events are now at full functionality. Encountered zero bugs in total of 60 randomly spawned empires. Only standard races for now, might add FE later. Great succes either way.


[See PR note] Add ES7 support

Add ES7 support to ILS.
Note: Requires the following PRs:

https://github.com/inveniosoftware/invenio-circulation/pull/119
#503

included in https://github.com/inveniosoftware/invenio-app-ils/pull/511


store icon in Asset directory

store icon in Asset directory according to https://github.com/kanboard/kanboard/blob/master/doc/en_US/plugin-registration.markdown

Can you remove the conflicts. The other PR has been merged.


This code is based to esp now and have max 250 byte?

In need extend Comunication for master and slave but have 355byte and 55byte speed 9600baud do you have any idea?
I have found this info online
https://stackoverflow.com/questions/60879021/how-to-increase-rx-serial-buffer-size-for-esp32-library-hardwareserial-platform
Thank you

Hello the link you sent is related to USB serial and not ESPNOW. Please refer to Espressif systems for detailed questions.

I need use this? Compatible with esp8266?
https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/peripherals/uart.html

No UART is a protocol over cable connection.
Espnow is a wireless protocol.
On the espressif website there is a pdf called:ESP-NOW User Guide
Inside the pdf espressif systems state that the esp-now protocol il linited to 250byte payload size.

Tommy I need help for transmit 350byte and 50
I have this log one wire 350 and other 50

No UART is a protocol over cable connection.
Espnow is a wireless protocol.
On the espressif website there is a pdf called:ESP-NOW User Guide
Inside the pdf espressif systems state that the esp-now protocol is limited to 250byte payload size.

{MCUMAINBOARD} {ESP} WIFI{ESP} {MCUBOARDDISPLAY)


[Bug]: Create a lock-free structure change task, and the log shows data too long for column

ODC version
ODC430
OB version
independent
What happened?
Create a lock-free structure change task, and the log shows data too long for column

What did you expect to happen?
ALTER TABLE `config_system_configuration` CHANGE COLUMN `value` `value_deprecated` varchar(1024);

How can we reproduce it (as minimally and precisely as possible)?
.
Anything else we need to know?
No response

have done


clarified documentation about current_company confusion

related to https://github.com/intercom/intercom-rails/issues/118

config.company.current = Proc.new { current_company } is used to insert the user current company in the the widget settings.
current_company in the above code is not an intercom helper and can be replaced in most cases (devise) by current_user.company
config.user.company_association = Proc.new { |user| user.companies.to_a } is uniquely used to perform bulk api user import using rake intercom:import command but is not injecting any new settings in intercom's widget.

In sum, it doesn't really make sense to track user events related to several companies at the same time that's why we do not inject companies in the widget settings.
But since a user can be related to several companies it is possible to import the users with several companies using config.user.company_association = Proc.new { |user| user.companies.to_a }
I have a question though, and a customer is facing this problem : why isn't it possible to run rake intercom:import in development ? It is really inconvenient when developing/debugging.
Should we instead warn the developer of his action ?
"You are about to import your local user database in your intercom application. Are you sure you want to continue ? (Y/n)"
@bobjflong @RuairiK

In answer to your question, I don't know why you can't, I guess it was just never implemented. You're right we should warn the user though. In my opinion the rake intercom:import feature just shouldn't exist at all. I think @bobjflong feels similarly
:ship:


Extension api for DataScience

For #13892

Codecov Report

Merging #13923 into main will decrease coverage by 59.86%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             main   #13923       +/-   ##
===========================================
- Coverage   59.86%    0.00%   -59.87%
===========================================
  Files         693       14      -679
  Lines       38355       38    -38317
  Branches     5516        3     -5513
===========================================
- Hits        22961        0    -22961
+ Misses      14214       38    -14176
+ Partials     1180        0     -1180

Impacted Files
Coverage Δ

src/client/debugger/constants.ts
0.00% <0.00%> (-100.00%)
:arrow_down:

src/client/common/process/constants.ts
0.00% <0.00%> (-100.00%)
:arrow_down:

src/client/common/platform/constants.ts
0.00% <0.00%> (-100.00%)
:arrow_down:

src/client/datascience/notebook/constants.ts
0.00% <0.00%> (-100.00%)
:arrow_down:

src/client/interpreter/autoSelection/constants.ts
0.00% <0.00%> (-100.00%)
:arrow_down:

...t/datascience/jupyter/jupyterDataRateLimitError.ts
0.00% <0.00%> (-50.00%)
:arrow_down:

...n/interpreterSelector/commands/resetInterpreter.ts

...nt/testing/common/services/discoveredTestParser.ts

...rc/client/application/diagnostics/commands/base.ts

src/client/common/contextKey.ts

... and 675 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e9e42e5...0b5feb1. Read the comment docs.


MM-35063: fixed static path traversal vulnerability

https://mattermost.atlassian.net/browse/MM-35063

Codecov Report

Merging #173 (c8ab605) into master (b340b80) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #173   +/-   ##
=======================================
  Coverage   20.16%   20.16%
=======================================
  Files          29       29
  Lines        1344     1344
=======================================
  Hits          271      271
  Misses       1055     1055
  Partials       18       18

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b340b80...c8ab605. Read the comment docs.

@jupenur: I added the unescaping. Please review.


WIP: Lesion tracker right panel

PR Checklist

[ ] Brief description of changes
[ ] Links to any relevant issues
[ ] Required status checks are passing
[ ] User cases if changes impact the user's experience
[ ] @mention a maintainer to request a review

CC: @swederik
Questions, for PDF, which of the following do we want to support?

Preview in Viewport?
Save (DICOM PDF, STOW)
Download

For other actions, which ones?

Download CSV
Save (DICOM SR) (Current study AND/OR Comparison Study?)

Do we create two DICOM SRs per save?

Do we want "Comparison", which only shows one set of measurements, or should we always show "Key Timepoints" (both sets of values) and nix the tabs?

We do not need the tabs, we will only be displaying Current and Comparison for now, so there won't be any other timepoints.

Does Targets/Non-Targets reflect the language we want?

No. We do not need these sections

Expand/Shrink section does not work but it does nothing in Lesion Tracker as well. Should we remove the plus signs?

There was never any expand / shrink section functionality. The plus signs would enable the tool that is used to make the measurement in that section. They are not necessary, you can remove them.

What actions should each measurement have?

TBD, but I think Relabel, Edit Description for now. Potentially Delete later, but we might let them just delete the tool on the image itself instead.

Should clicking a measurement jump to it in both viewports, if it exists?

Yes

For naming, did we land on "Current" and "Comparison" instead of "Baseline" and "Follow-up", where the naming is dictated by which study is in each viewport and the current HP settings?

The naming is constant. The ones that are marked Current when they open it is the Current. The other ones are the Comparison timepoint. HP settings are only used to specify where these appear in the viewport arrangement.

Questions, for PDF, which of the following do we want to support?

Preview in Viewport?
No

Save (DICOM PDF, STOW)
Not yet

Download
Yes

For other actions, which ones?

Download CSV
Do PDF first. When that's perfect, we can do CSV

Save (DICOM SR) (Current study AND/OR Comparison Study?)
Yes, but save Current only.

Do we create two DICOM SRs per save?
For now, we are only saving changes in the Current study, so it will only be one. Later on we will consider editing multiple timepoints simultaneously.

One more thing: Case Progress circle bar is not necessary for the first iteration, it's just polish which is not critical whatsoever.


[Feature Request] Full backup to also include sync status

Why/User Benefit/User Problem
Currently for people who have large EH favourites and change phone yearly, sync takes ages. with 10k+ gallery it takes me more than 3 days to complete and even with battery restriction turned off, sometimes the app might get closed or crash in the background so you have to basically keep it in foreground most of the time.
What I used to do with rooted phones was to backup with TB and restore everything.
With android features meeting most of my rooting needs I feel less the need to unlock bootloader and rooting.
What/Requirements
Would it be possible to have a full backup option to include login hashes and sync status so you don't need to perform a multi day full sync whenever you upgrade your device.

It would be really helpful to have such a feature, I had a crash while syncing with EH a few days ago (before I knew the EH extension was broken).
And it corrupted my sync status, so I needed to sync all my favorites again.
I have about 14k favorites and the sync would stop everytime my screen turned off, even with the battery setting set as unrestricted.
So what I did was to delete every favorites I had on EH and sync it again from tachiyomisy little by little.
I don't know if it is easy to do though but it would be really cool to be able to backup the sync status.
Thanks, and have a nice day.


chore: fix demo build

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 80.486%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
7391

Relevant Lines:
8365

💛  - Coveralls


Fix broken ATs

Motivation and Context
The Spring "Data REST" auto-generated APIs are great for rapid application development, but the security features we need don't lend themselves readily to the pattern: we will end up exposing sensitive data, and creating a large attack surface, unless we are very careful.

[x] Context Index has been kept up to date

What has changed
Replaced all the Spring auto-generated APIs with handwritten ones, which have full security and only expose explicit data, which is allowed to be viewed to prevent accidental data breaches.
How to test?
Every single feature of Support Tool has been affected, but there should be zero regression. Basically, do a full manual retest. Or, just trust me: I tested everything.
Links
Trello: https://trello.com/c/qDr6CDkJ

 Replace Support Tool API Spring auto CRUD endpoints with proper ones - FIRST ATTEMPT (2 days)


Adds Create View Support - closes #245

Hi.
As proposed in #245, I implemented a Create View support. The idea is that any select query can be turned into a view by calling the createView method.
So, an example of use would be:
var sql = require('sql');

var user = sql.define({
  name: 'user',
  columns: ['id', 'name']
});

var query = user.select(user.star()).where(user.id.equals(1));

var view = query.createView('myView');

var text = view.toQuery();

This returns a query object as folows. Note that since we are creating a view, the parameters are replaced with its values.
   {
      text: 'CREATE VIEW "myView" AS SELECT "user".* FROM "user" WHERE ("user"."id" = 1)',
      values: []
   }

Any doubts, just get in touch...

Really nice!! Sorry I missed this before - had some email issues. Will merge & push new version now! :heart:

:+1: thanks @brianc and thanks @edudutra, really useful!


Video seeks to 0 when reaching the end with midrolls/postrolls with client side ads

Have you read the FAQ and checked for duplicate open issues?
Yes
What version of Shaka Player are you using?
4.2.0
Can you reproduce the issue with our latest release version?
Yes
Can you reproduce the issue with the latest code from main?
Yes
Are you using the demo app or your own custom app?
Custom app
If custom app, can you reproduce the issue using our demo app?
Demo
What browser and OS are you using?
Chrome Version 104.0.5112.101 (Official Build) (x86_64) on MacOS 12.5.1
For embedded devices (smart TVs, etc.), what model and firmware version are you using?
N/A
What are the manifest and license server URIs?
Manifest: https://storage.googleapis.com/shaka-demo-assets/angel-one/dash.mpd
Ad tag URL: https://pubads.g.doubleclick.net/gampad/ads?iu=/21775744923/external/vmap_ad_samples&sz=640x480&cust_params=sample_ar%3Dpremidpost&ciu_szs=300x250&gdfp_req=1&ad_rule=1&output=vmap&unviewed_position_start=1&env=vp&impl=s&cmsid=496&vid=short_onecue&correlator=
What configuration are you using?  What is the output of player.getConfiguration()?
Default
What did you do?
With client side ads.

Start the content with the ad tag URL provided.
Seek close to the end/to the end.
Watch the midrolls and postrolls.
When the video is done playing postrolls, it automatically seeks to the beginning.

This also happens without postrolls. As long as there are midrolls, the player will seek to 0 when reaching the end.
Seeking to the end again when all ads have completed will not seek to 0, the player will stay at the end, as expected.
What did you expect to happen?
The video should stay at the end and not restart automatically.
What actually happened?
Already described extensively above. I'm not sure why this happens, it seems like when the adStopped event happens, the player automatically seeks to 0. Could this be because a video.play() is called when an adStopped event happens? Not sure what the workarounds is here besides releasing/stopping the ad manager when all ads have completed or when the video element reaches the end.

Hi again! :)
This is actually a pretty important issue for us, and I can't seem to find a reliable workaround for this... any ideas?


Sass keywords 'to' and 'through' in a @For loop.

Sass keywords to and through works the same way in a @For loop.
For example:
@for $i from 1 to 3 {
  h#{$i} {
    font-size: #{$i}em;
  }
}

@for $i from 1 through 3 {
  h#{$i} {
    font-size: #{$i}em;
  }
}

Result:
h1 {
    font-size: 1em;
}

h2 {
    font-size: 2em;
}

h3 {
    font-size: 3em;
}

According to Sass docs:
In directive @for $var from <start> through <end> the range includes the values of <start> and <end>
In directive @for $var from <start> to <end> the range should not include <end> value.

Precss is not compiling the following sass statement
@for $i from $minMultiplier to $maxMultiplier
{
#{if(&, "&", "*")}.#{$selectorName}-#{$i} {
#{$propertyName}: ($i * round($propertyMultiplicand))
}
}
I am getting the following error
Fatal error: precss: /Users/utilities/math.scss:70:9: Unknown word  68 |     @for $i from $minMultiplier to $maxMultiplier
69 |     {

70 |       #{if(&, "&", "*")}.#{$selectorName}-#{$i} {
|         ^
71 |         #{$propertyName}: ($i * round($propertyMultiplicand))
72 |       }

Any thoughts...

@sanvyx If it will still make no difference whether you use to or through (using the newest version 4.0.0) this problem belongs to PostCSS-advanced-variables.
Details about @for: https://github.com/jonathantneal/postcss-advanced-variables#for-and-each-rules
@hawckins it appears to be the same problem as #114. I provided a potential solution there. Please provide feedback if this solved your issue.


Cached animations/updaters not working correctly...

Description of bug / unexpected behavior

When you run certain scene again without editing anything, the output is not the same. New partial_movie_files gets created with different names and different output...
How to reproduce the issue
I guess the animations involving updaters get affected... Running the same scene again without editing code at all, will result in weird things. For example: This scene and here is the one in which I found this issue after spending hours figuring out whats wrong...
Doing rm -rf media is a temporary workaround
Additional media files

Example 1
First run:

Run again without editing anything, objects teleport at origin in last second

Example 2
First run:

Run again without editing anything

Edit: #550

Sorry to not get to this back sooner. It seems that it's a duplicate of #550 (but with adding some additonal informations, which is very great!)
Therefore, I'll close it/


[Updating] Refactor autoupdate mechanism to use msgbox instead of toasts

Summary of the Pull Request
What is this about:

toasts notifications complicate update logic, since a user could click them while PT isn't running
they also caused issues for some users where the notification isn't visible for some reason

Therefore, we've decided to move to simple messagebox notifications for now.
What is include in the PR:

do not use toast notifications for autoupdate
unifies the logic for periodic update check and "Check for updates" button
we only download the release from github after the user has confirmed update intent by pressing "Yes" on the confirmation dialog

I haven't removed some of the "update at next launch" functionality, since we haven't decided if we want to keep it. The relevant code is marked with TODO deprecation.
How does someone test / validate:

modify Version.props to some lower version
test "Check for updates" button
remove %localappdata%\Microsoft\PowerToys\update_state.json to reset periodic check cooldown and restart PT

Quality Checklist

[x] Linked issue: #10589
[ ] Communication: I've discussed this with core contributors in the issue.
[ ] Tests: Added/updated and all pass
[ ] Installer: Added/updated and all pass
[ ] Localization: All end user facing strings can be localized
[ ] Docs: Added/ updated
[ ] Binaries: Any new files are added to WXS / YML

[x] No new binaries
[ ] YML for signing for new binaries
[ ] WXS for installer for new binaries

Contributor License Agreement (CLA)
A CLA must be signed. If not, go over here and sign the CLA.

since we need to do out of scope work for this, moving the code to a separate branch


Offer choices for energy strategy

This feature will add options for the user to configure his/her own energy strategy from a set of predefined strategies. For now this is more of a testing tool, but it may prove useful later for users.
In the short term, this patch will allow us to increase the energy limit to test system limitations further.

Closed in https://github.com/aionnetwork/aion/commit/c3d8d792a72c8e384a9e0c3e58fbe00058680d00


Fade-in effect instead of "Loading..." text?

Hi,
Another enhancement request... :-)
It would be nice to have an option to replace the "Loading..." text with a fade-in effect!
If there is no preview image defined, the default starting image should be a black screen... and the panoramic image could fade in from that.
Thanks!
Benke

What happens if the user has a slow connection and it takes a minute to load the panorama? I don't think this is a good idea, so I'm not going to implement it.


Does mxGraph support a sub-process container?

In my case, the sub process is needed to embeded in a main process, and it seemed that we can put these sub process activities elements into a group box, this box can be connected to the nodes from main process. the picture is similary here:
sub process box container exmaple
By the way, is message flow supported when crossing two different swimlanes? the style of message flow usually is dashed line in toolbox.
Thanks your reply, we really like mxGraph library.

@besley can you provide more context please?
It looks like you are working on a BPMN solution as most of the wording you are using are related to this business domain, so most mxGraph users won't know what you are talking about. So please, keep that in mind for a future question.
Here are some hints

for a subprocess, you can create a mxRectangle vertex and make it be the parent of elements (vertexes and edges) that defined the subprocess content
for message flow, this is an edge so you can assign it a dedicated style when creating it to have a dashed line. On insertion, you make the graph default parent as its parent to ensure it is displayed. You may have to experiment on this area if you want to manage lane collapsing and keep consistent display of the message flow

I am also currently working on a BPMN rendering solution based on mxGraph. I will be interesting in talking with you about use cases, implementation choices, targeted application

@tbouffard , thanks your reply. This is very helpful to me. I have looked into the mxGraph examples in the project. By the way, I find swimlane control is also a choice to separate different processes. I am glad to talk about it with you.

@besley great to know you found a solution
Good catch about the swimlane: it can be a way to do it, especially as it already provides expand/collapse capabilities out of the box

@tbouffard , I have a question, the customer suggested we use bpmn.io rather than mxGraph, How can I respond to their requirement? what's the benefit of these two projects each other?

@besley I guess you are talking about bpmn-js, mainly developed by Camunda.
If you need a BPMN modeler/editor, this is the way to go except if you want to develop a competitor lib which is going to be a large amount of work.
For BPMN viewing/visualization, this is a big amount of work as well. You can also have a look at bpmn-visualization, a Typescript lib using mxGraph that I am working on and sponsored by Bonitasoft. Notice that this lib is in early development and is not BPMN features complete.
Be aware of the bpmn.io license which forces you to display the bpmn.io logo on the bottom right position of the diagrams (the license forbids you to remove it)


Reflow is taking long time to load the graph

What part of Carbon does this bug correlate to?
General graph logic:

[X]  Graph attributes (axes, legends, regions...etc.)
[ ]  Other (please specify):

Specific graph logic:

[ ]  Bar graph
[ ]  Bubble graph
[ ]  Gantt chart
[ ]  Line graph
[ ]  Paired result
[ ]  Pie chart
[ ]  Scatter plot
[ ]  Timeline

Describe the bug
When we try to load a graph during reflow it takes a considerable amount of time than the time taken to create the graph.
Expected behavior
the time to load the graph for that particular timeframe during reflow should be less than the time taken to create the graph since we are not destroying or the existing graph object and we are just updating the graph.
It took for hardly 4 seconds to load the graph

Whereas it took around 7 seconds to update graph during panning

Can you include a code sandbox that demonstrates this behavior?

@sdadn The reflow legend takes more time when there are multiple legends in the graph canvas. I am have made a sample code sandbox that shows the same behaviour.

@NarasimhaShenoi in the sandbox, why are you calling the reflow method separately for each dataset?
const createGraph = () => {
  graph.config.axis.x.lowerLimit = new Date(2016, 0, 1, 23, 59).toISOString();
  graph.config.axis.x.upperLimit = new Date(2016, 0, 0, 11, 59).toISOString();
  graph.reflow({ ...CONTENT_DATA });
  graph.reflow({ ...CONTENT_DATA2 });
  graph.reflow({ ...CONTENT_DATA3 });
  graph.reflow({ ...CONTENT_DATA4 });
  graph.reflow({ ...CONTENT_DATA4 });
  graph.reflow({ ...CONTENT_DATA6 });
  graph.reflow({ ...graphData1 });
  graph.reflow({ ...graphData2 });
  graph.reflow({ ...graphData3 });
  graph.reflow({ ...graphData4 });
  graph.reflow({ ...graphData5 });
  graph.reflow({ ...graphData6 });
};

According to the documentation, the reflow method should be called once and additional datasets should be loaded usin the loadContent method:
https://github.com/cerner/carbon-graphs/blob/main/docs/controls/Panning.md
export const renderLineWithPanning = (id) => {
    const axisData = graphConfiguration;
    axisData.pan = {
        enabled: true
    };
    const graphData = dataSet;
    graphData.regions = [regions[0]];

    const createGraph = () => {
        graph.reflow();
    };

    const graph = Carbon.api.graph(axisData);
    graph.loadContent(Carbon.api.line(graphData));
    // Additional data-sets to be loaded here only, like:
    graph.loadContent(Carbon.api.line(/* Data array */));
    axisData.axis = graph.config.axis;

    createPanningControls(id, {
        axisData,
        creationHandler: createGraph
    });
    return graph;
};

@sdadn loadContent() is used to load the data series(or legend) into the graph object. For example, in the dev site, we will load the content for each line graph here. We need to call the reflow method separately for each dataset.

@NarasimhaShenoi I tried a few examples from the dev site and your sandbox and I'm not seeing any difference between calling the reflow method once or separately per dataset, besides performance. Unless if you have an example that specifically shows why calling the reflow method separately is required, this does not seem to be a bug for general use cases. The examples and documentation both indicate to call it once.

@sdadn
We need to call the reflow function for each dataset that needs to be updated (in the sense that some new data is being added to the dataset with the key already present in the graph) for that particular timeframe.
For eg., if there are 2 datasets for blood pressure in a graph, and the user requires to dynamically update both the datasets once the panning button is clicked. Then they must call reflow twice once for each dataset.
I have created a sandbox to view what happens when you use loadContent instead of reflow for multiple datasets. View line 16 in graph.js.
And regarding the performance issue, I think we can improve it by changing the graphData sent by the user into an object where users can send in multiple key-value pairs, and later we can filter out which graph type to call over here. This will save us the time required to check whether the graph axes has been updated or not. And reduce the reflow calls from users' end.

@sdadn / @mraplumb / @AshishMotanamGurunadham  If this can be picked up, it will very helpful for us. it will increase the performance to a considerable amount.

Hi @NarasimhaShenoi , sorry we've had our hands full with some recent issues that came in. But this is still on the table and we are exploring potential solutions.

Hello @NarasimhaShenoi , we are looking at a solution to improve performance and consumer experience but it is a long term solution that won't be implemented immediately. In the meantime, here is one potential short-term solution:
There was a recent update that improved the performance of loading and unloading datasets by passing them as an array:
graph.loadContent([dataset1, dataset2]);

Similarly, you can unload content this way:
graph.unloadContent([dataset1, dataset2]);

My suggestion for now would be to try to unload and load datasets within the panning button method. You can also add logic to make sure only appropriate datasets are unloaded and loaded with new data.
const createGraph = () => {
  graph.config.axis.x.lowerLimit = new Date(2016, 0, 1, 23, 59).toISOString();
  graph.config.axis.x.upperLimit = new Date(2016, 0, 0, 11, 59).toISOString();
  graph.unloadContent(old_dataset_array);
  graph.loadContent(new_dataset_array);
  graph.reflow();
};

Let us know if you have any questions or if this solution doesn't work, then we can look at other short-term options.

Hey @sdadn, as you suggested I tried to unload and load the datasets within the panning button, but I was getting this error: Error: Invalid input format, unique key must be provided. I think it might be result of same input key we are trying to pass via panning. Not sure, I am doing it wrongly. Can you please provide me a sandbox example. Also, I am assuming, if we are passing the dataset as a batch, then we will not be using the changes we did in the reflow.

@NarasimhaShenoi  Yes, the error, Error: Invalid input format, unique key must be provided occurs when you’re loading two datasets with the same key.  So you’ll need additional logic on your end to make sure that you are not loading a dataset with an existing key. Two possible solutions are:

Make sure the keys are unique, such as DatasetA_1, DatasetA_2 etc.
Or maintain an array/object that keeps tracks of what datasets are loaded/unloaded.

Hope this helps. Let me know if you're still having trouble.

@NarasimhaShenoi  Yes, the error, Error: Invalid input format, unique key must be provided occurs when you’re loading two datasets with the same key.  So you’ll need additional logic on your end to make sure that you are not loading a dataset with an existing key. Two possible solutions are:

Make sure the keys are unique, such as DatasetA_1, DatasetA_2 etc.
Or maintain an array/object that keeps tracks of what datasets are loaded/unloaded.

Hope this helps. Let me know if you're still having trouble.

@sdadn  Can you please provide a sandbox example for the same? I am not sure how we can ensure the reflow concept will be working if we load and unload content with uniques keys as you suggested. During reflow, the legend state will be retained as it is. But with the approach of loading and unloading the dataset, I don't think it can be retained.
I believe the approach mentioned by Pranav above is a good one where we pass multiple key-value pairs as an array to reflow and do the logic in carbon reflow.  Let me know your thoughts

@sdadn  Can you please provide a sandbox example for the same? I am not sure how we can ensure the reflow concept will be working if we load and unload content with uniques keys as you suggested. During reflow, the legend state will be retained as it is. But with the approach of loading and unloading the dataset, I don't think it can be retained.
I believe the approach mentioned by Pranav above is a good one where we pass multiple key-value pairs as an array to reflow and do the logic in carbon reflow.  Let me know your thoughts

I have created a branch based on the solution that I had proposed earlier over here.
I have tested this on @NarasimhaShenoi repo and the result is as follows:
With the current method: Approximately 8-10s

With the new changes: Approximately 2-2.5s

Currently, I have only made changes for the graph construct as a POC, based on which I can update Timeline and Gantt as well.
cc/ @mjhenkes @mraplumb @sdadn

I have created a branch based on the solution that I had proposed earlier over here.
I have tested this on @NarasimhaShenoi repo and the result is as follows:
With the current method: Approximately 8-10s

With the new changes: Approximately 2-2.5s

Currently, I have only made changes for the graph construct as a POC, based on which I can update Timeline and Gantt as well.
cc/ @mjhenkes @mraplumb @sdadn


Cleaned up the documentation for v0.8

Removed outdated documentation that isn't needed for v0.8

Thanks for the cleanup!
LGTM


cython yield-equivalent

IIRC cdef and yield don't play together nicely.  I'm wondering if this can be made viable by some kind of reverse-inlining in some subset of situations.
The motivation is from pandas timezone-conversion functions, which have a lot of repetition that in python-land we could avoid with yield.  (One cast of several: https://github.com/pandas-dev/pandas/blob/master/pandas/_libs/tslib.pyx#L128)
The relevant functions all look something like:
def myfunc(...):
    if condition1:
        for i in range(N):
            val = func1(values[n])
            callback(val, ...)
    elif condition2:
        for i in range(N):
            val = func2(values[n])
            callback(val, ...)

where the only difference between the repeated code is in callback  (we can't actually pass a callback because they have different signatures).
It seems like it should be possible to write:
cdef myfunc(...):
    for val in cyielder(...):
         callback(val)

cdef cyielder(...):
    if condition1:
        for i in range(N):
            val = func1(values[n])
            cyield val
    elif condition2:
        for i in range(N):
            val = func2(values[n])
            cyield val

and have cyielder inlined to make these two versions of myfunc equivalent.  Is this at all plausible?

In theory, this should be possible. In practice, someone has to implement it and deal with all the little details. ;)
I could also imagine allowing @cython.inline on generators to trigger this behaviour.

In practice, someone has to implement it and deal with all the little details. ;)

OK.  I'll get around to this eventually if no one else does.  I still haven't managed to get a good enough handle on the cython codebase to guess when that will happen.


TimerInfo object shows incorrect Next execution time

The TimerInfo object that comes as the default timer lists the "Next" execution as the current execution. Shouldn't this show the next execution that is coming based on the schedule, not the current execution?
Otherwise, should it be named Current rather than Next. The Last shows the previous execution, so I would expect Next to show the upcoming execution.

Closing as duplicate of #954
Please leave a comment on that issue if you have further suggestions to improve our docs/templates. Thank you!


🌱  Contextual logging fixes

Summary
Move the following to contextual logging

virtual apireconciler
pkg/serve

Also disables the newly-introduced key name check for the time being.
Related issue(s)
Fixes #

/retest


Testing, please ignore. (cyzwrl)

Bug bounty test - please ignore.... Please DO NOT APPROVE THIS! (cyzwrl)

Spambot, closing PR.


Data Cleanup

I went through old compilation reports linked here: http://water.usgs.gov/watuse/50years.html and added in data that was missing. A couple were 0 where they should have been numbers, others were NA where the reports had data. I recorded my changes in #75 and we can review from there.

wowsas 👍