Introduce validate-inputs, which can be used to check for variable alignment

Refer to the docs for info on this new command.

Thanks for review! Will merge and release!


Broker load shall fail immediately with error message on wrong PROPERTIES setting

Enhancement
When there is misconfig in the broker load PROPERTIES(e.g. wrong AWS AK/SK or endpoint), broker load job state becomes QEUEUEING  with out and error message to help debug.
Suggest fixing this issue by setting the state to INVALID or FAILED and have details in ErrorMsg fields.
This is essential for the upstream service to check the status and take appropriate measure for failure resilience.

cc @chaoyli

Thanks for reporting this issue. It's highly OK to add a check whether the ak/sk is set.
I'll resolve it.

I've checked the latest version of branch-2.5/branch-3.0 and found that the result has been shown like this. It seems to be what you want.

It is PENDING state in a built from branch-3.0 commit hash https://github.com/StarRocks/starrocks/commits/a5dd8c36
on 5/23/2023
In this test, I issued the load sql with a wrong secret key in the properties.

LOAD LABEL abb_10m.KNA1_4
(
DATA INFILE("s3a://celostar/test_data/abb_10m/fd1c713c-d76f-4113-99d2-a22c38afc7a6/*.parquet")
INTO TABLE KNA1
FORMAT AS "parquet"
******
WITH BROKER
(
"aws.s3.access_key" = "xxx",
"aws.s3.secret_key" = "WRONG SECRET", <===================
"aws.s3.region"="us-west-1"
)
PROPERTIES
(
"timeout" = "36000", --  the timeout time in seconds for the import operation. e.g 3600 means 1 hr time out. Jobs not completed in timeout window will be CANCELLED state.
"max_filter_ratio" = "0.2", -- Data ratio of maximum tolerance filterable (data irregularity, etc.).
"timezone" = "America/Los_Angeles"
);

This is enhanced in https://github.com/StarRocks/starrocks/pull/21982.
Could you check it with a newer version again?


Reformat en-US style guide to incorporate style guides from SUMO and other areas of Mozilla

The en-US style guide needs to be restructured to ensure that we can add content from various other areas of Mozilla. Off the top of my head, that would mean establishing everything we have currently under a h2 "Products and web pages" and allow other areas of Mozilla to follow suit (e.g., h2 "Translating articles for SUMO knowledge base" with content following it).

#53 is merged for some time. What is the next move for SUMO style guides?


adding more images and a listener for 'success'

:+1: :rocket: :cake: :+1: Nice!
Be sure to bump the version so you can publish this too.

good catch. I'll fix it now.


[refactor] #4323: telemetry features, tokio console, logs, and configuration

Description
This is an improved iteration of https://github.com/hyperledger/iroha/pull/4369

Remove logger.tokio_console_address configuration parameter entirely: this could be configured in a more refined way using default ENVs (see console_subscriber docs)
Rename [telemetry.dev] config section to [dev_telemetry]
Update telemetry and dev-telemetry features of iroha
Update tokio-console feature of iroha_logger: now console seamlessly starts whenever this feature is enabled
Improve logs & warnings
Fix dev-telemetry file logs: separate each item with \n, and recursively create parent directories
Improve config validation: handle file paths

Linked issue
Closes #4323
Benefits

Less config parameters
More flexible way to configure tokio console
Less feature mess (?)
Better warnings
Refactoring

Checklist

[x] Document how to use tokio console
[x] Figure out what happened to Cargo.tomls
[x] Check manually several cases, how Iroha behaves with telemetry/dev-telemetry features enabled depending on telemetry configuration (warnings, logs, file output)
[x] Check manually Tokio Console (works, but requires TRACE log level)

Does telemetry used at all? I'm aware that QA and DevOps team using prometheus metrics.
But i'm not aware of using WS for telemetry, if it's not required we can make code simpler.

@Erigara, created a separate discussion for it: https://github.com/hyperledger/iroha/issues/4385

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

133 of 194 (68.56%)  changed or added relevant lines in 7 files are covered.
4990 unchanged lines in 91 files lost coverage.
Overall coverage increased (+0.8%) to 57.623%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

cli/src/main.rs
0
2
0.0%

torii/src/lib.rs
0
2
0.0%

logger/src/lib.rs
15
18
83.33%

config/src/parameters/user.rs
33
40
82.5%

cli/src/lib.rs
80
92
86.96%

telemetry/src/dev.rs
0
35
0.0%

Files with Coverage Reduction
New Missed Lines
%

primitives/src/conststr.rs
1
91.14%

ffi/derive/src/convert.rs
1
84.45%

primitives/src/lib.rs
1
0.0%

core/src/sumeragi/network_topology.rs
1
98.78%

telemetry/src/dev.rs
2
0.0%

crypto/src/signature/bls/mod.rs
2
0.0%

config/src/snapshot.rs
3
78.57%

data_model/derive/src/has_origin.rs
3
95.16%

config/src/kura.rs
3
80.0%

logger/src/lib.rs
3
92.5%

Totals

Change from base Build<PHONE_NUMBER>:
0.8%

Covered Lines:
23107

Relevant Lines:
40100

💛  - Coveralls


Current function name as a launch variable

Out of all the variables I see in the launch configuration, I'm surprised there isn't one that puts in either the name of the function/method that the cursor is at, or the contents of the editor's selected text. Use case being, I use launch.json in my PHP-based project to run PHPUnit in debug mode. I can currently launch specific test files, but I can't narrow it down to a single test function. I'm aware of the PHPUnit extension, and that's great for running specific tests, but I'd rather not mess with my environment variables to do debugging and rely on other team members to be able to do it as well.
This may be a niche case, but maybe this could be useful for other cases?

This functionality can be easily implemented as an extension.

@weinand thanks for pointing that out. Due to that closing this, we try not to keep extension candidates open


Protozoa demo broken due to incorrect smoothstep usage

Thanks for the info. I'm not quite sure why it did work in the past. It should be fixed. Does it work for you now?
The npm package is not updated as the npmjs.com is doing some maintenance now and package publishing is temporarily unavailable.


Bump to latest ZMQ

Signed-off-by: Ryan Roberts<EMAIL_ADDRESS>

Painless update to current jeromoq 5.2, only a minor interface change. I am assuming this smoke tests against the correct version of the validator


How can I get the content lines?

if I want to get the content lines ,How shoud I do?

There is a method getFlexLines in the FlexboxLayout.
Is that what you are referring to?

Btw, technically it's not an issue. Let me close this.


Call to PostTimedTask fails on Windows (x86_64) in HostSample and NodeSample

See the following issue: https://github.com/tondrej/chakracore-delphi/issues/20
The bug did not affect the PostTimedTask method in the WasmSample project, but I still modified it for consistency.
The fix was tested on:

Windows (i386) through Delphi 10.1 Berlin and Free Pascal 3.2.2 compilers
Windows (x86_64) through Delphi 10.1 Berlin and Free Pascal 3.2.2 compilers
Linux through Free Pascal 3.2.2 compiler
Darwin through Free Pascal 3.2.2 compiler

Thank you for the fixes!


Move deprecations along.

std\process.d(3457): Error: static assert  "Unsupported platform"

Current coverage is 88.87% (diff: 100%)

Merging #4769 into master will increase coverage by 0.03%

@@             master      #4769   diff @@
==========================================
  Files           121        121
  Lines         74284      74216    -68
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits          65990      65957    -33
+ Misses         8294       8259    -35
  Partials          0          0

Powered by Codecov. Last update caba440...c8d942b

In theory, that fixes the std.process problem on Windows, but we'll have to see what the auto-tester has to say.

Okay. That did fix the windows build problem.

Auto-merge toggled on


Fix linter issue.

Description
Fix an issue reported by the golang linter in the test file.
Test
docker run --rm -v $(pwd):/app -w /app golangci/golangci-lint:v1.52.1 golangci-lint run -v

As a very simple change and all CI green, I'm merging it with no reviews


Add Prefix & Suffix Widget in TextField

Hey @Aditya-Mi  everything is good. Awesome work.
But currently there is no way to change the icon color as we are only taking IconData so I guess it would be much better if we can take Icon itself which will allow users to change the color of Prefix and Suffix Icons.
Else everything LGTM 👍🏼

@Aditya-Mi  also we can still see the space for icons even when no icon is provided which should not happen at all.


installtion with ZPM fails

Running Angular build script...
Building in 2 stage(s).
Running in background: npm ci --prefix C:\InterSystems\IRIS\mgr.modules\USER\isc.perf.ui\1.0.1\ng\monlbl-viewer\ --no-progress --node-flags --max-old-space-size=8000
Der Befehl ""node"" ist entweder falsch geschrieben oder
konnte nicht gefunden werden.
Angular build process completed. Elapsed time: 1.048151 seconds.
Running in background: npm run --prefix C:\InterSystems\IRIS\mgr.modules\USER\isc.perf.ui\1.0.1\ng\monlbl-viewer\ build --node-flags --max-old-space-size=8000 -- --baseHref=/csp/user/monlbl-viewer/ --progress=false
Der Befehl ""node"" ist entweder falsch geschrieben oder
konnte nicht gefunden werden.
Angular build process completed. Elapsed time: 2.080767 seconds.

hint: nodejs is not mentioned as prerequisite

@r-cemper thanks for reporting. Ultimately what I'd like to be able to do is publish the built artifact to the package registry (without having artifacts in the repo - this is something all of the underlying machinery in IPM / isc.ipm.js supports except Open Exchange AFAICT).
Until then it's a documentation thing, same as reported in #2. I'll update the documentation and follow up re: the upstream issues.

I installed node.js on my ages old test box
now install works OK:
SO MY issue is closed -  Just a matter of docu.


Added functionality to upload files to PasswordState

I have updated the New-PasswordStatePassword function to support uploading files to Password state, plus added a new function called Set-PasswordStateDocument, to upload a file to and existing document.
PS
I am relatively new to this Git stuff, so I hope I have done everything right, otherwise please let me know, what I need to change.

Thanks for the contribution!


where is with() ?

where is with() ?

Use setPrefix() - set the prefix to whatever you need, add all the routes, then set it to an empty string.


Always add the document boxes to correct field set

The override of the method getFormItems in the ExtJS class Shopware.apps.Config.view.form.DocumentPaypalUnified is used to add the document elements of paypal to the document element configuration in the backend.

Unfortunately the override always adds this to the second field set found in the right panel of the window. In case another plugin adds a field set before, then the override will add the document elements to the wrong fieldset. Therefore they will not be editable for the user anymore.

The PR improves the determination of the correct field set so that the document element are always added to the correct fieldset.

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/PT-9754
Please use this issue to track the state of your pull request.

thanks @windaishi for your contribution :+1: :blue_heart:
your fix will be part of the next release


Produce more useful logs and fix unhandled crash when an upstream message contains invalid JSON characters

Codecov Report
Attention: Patch coverage is 0% with 16 lines in your changes missing coverage. Please review.

Project coverage is 70.51%. Comparing base (5c98c00) to head (ec4497f).

Files with missing lines
Patch %
Lines

src/Application/Handlers/HttpErrorHandler.php
0.00%
16 Missing :warning:

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #167      +/-   ##
===========================================
- Coverage    71.52%   70.51%   -1.01%
===========================================
  Files           40       40
  Lines          913      926      +13
===========================================
  Hits           653      653
- Misses         260      273      +13

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Not sure why it would create a 2nd json exception, but this now LGTM.

Ah, might have lost the thread of what handles the ActionError vs. payloads


PMR ami3 PDF reader clips last line of text

Initial inspection of text from ami3 PDF reader suggests that the list line of text on a page has been clipped. This may be an off-by-one error or it might be the wrong media-box for the reader. In practice it mainly clips the footer and does not affect the running text.
Will need to assemble all ami3 errors and debug so as to create a better release.

Can you check if the last line is actually in an earlier position?
When I was looking at PDF.js, the footer tended to be the second entry for each page, and would appear in position number 2.
I had to sort lines by their Y coordinate to be able to detect paragraphs.

I will sort by Y.
The problem with spaces is that they have several meanings:
This         sentence       has   a   lot   of     whitespace     to   pad
out
These are headings:
Name             Place           Date
There is no deterministic algorithm to decide. Has to use content and
context
On Sat, Apr 16, 2022 at 11:22 PM Dimitar Simeonov @.***>
wrote:

Can you check if the last line is actually in an earlier position?
When I was looking at PDF.js, the footer tended to be the second entry for
each page, and would appear in position number 2.
I had to sort lines by their Y coordinate to be able to detect paragraphs.
—
Reply to this email directly, view it on GitHub
https://github.com/mitko/readable_climate_reports/issues/10#issuecomment-1100763567,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAFTCS7GUW4MFTBHNQLYY43VFM4QPANCNFSM5TR3W6KQ
.
You are receiving this because you were assigned.Message ID:
@.***>

--
Peter Murray-Rust
Founder ContentMine.org
and
Reader Emeritus in Molecular Informatics
Dept. Of Chemistry, University of Cambridge, CB2 1EW, UK

My original; issue may indeed be a sorting artefact (and not a bug). Have written an x-then-y sorter. Will investigate further.
There is no guarantee of reading order. I think these pages may be in the order

write header
write footer
write main text


Add new, edit old entries in /docs/design for milestone 0.4

Includes write-up for Auto-Search

2 docs updated.  Pushed to master. Closing


New package: hPFMD v0.1.0

Registering package: hPFMD
Repository: https://github.com/Chenghao-Wu/hPFMD.jl
Created by: @Chenghao-Wu
Version: v0.1.0
Commit: ba387b81d81ad01afbe29545f52714acd93fee9e
Git reference: HEAD
Description: Massively parallel hybrid particle-field molecular dynamics (hPF-MD) simulation method in Julia

@JuliaRegistrator register

Error while trying to register: Register Failed
@Chenghao-Wu, it looks like you are not a publicly listed member/owner in the parent organization (JuliaRegistries).
If you are a member/owner, you will need to change your membership to public. See GitHub Help

Error while trying to register: Register Failed
@Chenghao-Wu, it looks like you are not a publicly listed member/owner in the parent organization (JuliaRegistries).
If you are a member/owner, you will need to change your membership to public. See GitHub Help

I changed the package name and fixed the issues. Should I re-register the package since the URL and name of the package are changed?
Thank you for taking care of this.


[Bug]: No search parameter for HealthcareService in $find operation

Contact Details
<EMAIL_ADDRESS>Section Number
OperationDefinition: Find_Appointments_Operation
What is wrong
In the list of search parameters of the $find operation it would make sense to also be able to search for free appointments with a specific HealthcareService. The search parameter specialty might not be sufficient to narrow down the search in cases where a treatment exists in multiple variations.
E.g. a massage could be offered with duration of 30 or 60 minutes, which can be represented as two separate HealtcareServices. In that case the patient would likely want to select only one of the two and see free appointments just for the selected type of massage when booking an appointment online.
Describe the solution you'd like
Add search parameter healthcare-service in the same way as practitioner and organization to $find operation.
Relevant log output
No response
Priority
{"Low"=>"Typo or other minor classification that an editor can manage. Requires no group discussion."}
Code of Conduct

[X] I agree to follow the IHE Code of Conduct

It is not clear how HealthcareService can provide additional details beyond specialty and location combinations. In the example given about 30 minute vs 60 minute massage, unless you have different codes for each duration, it doesn't seem that using the HealthcareService resource can help.


Controller memory usage continues to increases

Pre-requisites

[X] I have double-checked my configuration
[X] I can confirm the issues exists when I tested with :latest
[ ] I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?
Our argo-workflows-server-workflow-controller pod gradually increases its memory usage due to what seems to be old workflows not being deleted.  The usage is currently:
% kubectl top pods -n argo --containers|{head -1; grep controller}
POD                                                            NAME            CPU(cores)   MEMORY(bytes)
argo-workflows-server-workflow-controller-5557fd8494-8wff8     controller      9m           8550Mi

and argo list will never return:
% argo -n argo-workflows list
FATA[2023-05-25T10:49:16.956Z] rpc error: code = Internal desc = stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 1; INTERNAL_ERROR; received from peer

Looking at the helm chart, it appears the retention defaults are
  # -- Workflow retention by number of workflows
  retentionPolicy: {}
  #  completed: 10
  #  failed: 3
  #  errored: 3

but this may not be relevant.  It's my understanding based on another thread that I can no longer find that controller memory should remain below 1GB.
Any ideas what is going on here?
Version
3.4.7
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
N/A

Logs from the workflow controller
N/A

Logs from in your workflow's wait container
N/A

due to what seems to be old workflows not being deleted

Can you elaborate?

The behavior we have seen is argo list was slower and slower to return as more workflows were created and now it will not return at all.  I'm just trying to figure out why the controller memory continues to grow and argo list now is timing out.
Am I on the right track with the retention policy settings I referenced?  As mentioned, I recall a thread where someone stated controller memory should always be below 1GB.

See this: https://github.com/argoproj/argo-workflows/issues/10360
@terrytangyuan I also think there is a memory leak in the controller.

We just had our controller OOMKilled with a memory limit of 12GB:
    Last State:     Terminated
      Reason:       OOMKilled
      Exit Code:    137
      Started:      Thu, 01 Jun 2023 08:06:03 -0500
      Finished:     Thu, 01 Jun 2023 08:07:37 -0500

and the pod entered CrashLoopBackOff:
argo-workflows-server-workflow-controller-5557fd8494-mdzkj     0/1     CrashLoopBackOff   667 (2m33s ago)   2d16h

After deleting the above pod a new one was created, but argo list still timeouts:
% argo -n argo-workflows list
FATA[2023-06-01T08:16:19.877Z] rpc error: code = DeadlineExceeded desc = the server was unable to return a response in the time allotted, but may still be processing the request (get workflows.argoproj.io)

There seems to be two separate issues.  One, there is something causing the controller memory to grow unbounded.  Two, the retention settings mentioned above do not seem to have any effect.
We are also seeing the controller pod fail health checks which I'm sure is related.

I believe this has always been an issue, but cannot say for sure.  I suspecting that this is a retention problem in that the default settings are not working:
argoworkflows=> select count(*) from argo_archived_workflows;
 count
-------
 10810
(1 row)

I'm currently running
argoworkflows=> select * from argo_archived_workflows order by finishedat;

and this has yet to return after a few minutes.  We are using db.r5.large RDS instances and the metrics seem fine.

I've explicitly set
  retentionPolicy:
    completed: 300
    errored: 50
    failed: 100

and no change in the number of rows in the argo_archived_workflows table.  Perhaps this configuration has nothing to do with that, but letting that table grow indefinitely isn't going to scale.

Not sure if this is related, but the controller pod does not seem to be well behaved under conditions where it restarts:
% k get pods
NAME                                                           READY   STATUS                   RESTARTS         AGE
argo-workflows-server-argo-workflows-server-7cb79fcdc6-x8z4h   2/2     Running                  0                10d
argo-workflows-server-workflow-controller-5557fd8494-76n4s     0/1     Error                    6 (4h37m ago)    5h
argo-workflows-server-workflow-controller-5557fd8494-q85gl     0/1     ContainerStatusUnknown   15 (5h2m ago)    6h16m
argo-workflows-server-workflow-controller-5557fd8494-s8hst     0/1     ContainerStatusUnknown   1 (4h30m ago)    4h34m
argo-workflows-server-workflow-controller-5557fd8494-vpvt9     1/1     Running                  48 (6m42s ago)   4h22m
argo-workflows-server-workflow-controller-5557fd8494-xxc9j     0/1     ContainerStatusUnknown   1 (4h23m ago)    4h28m

Why would all of the controller pods that are in a bad state collect?

I believe this has always been an issue, but cannot say for sure. I'm suspecting this is a retention problem in that the default settings are not working:
argoworkflows=> select count(*) from argo_archived_workflows;
 count
-------
 10810
(1 row)

I'm currently running
argoworkflows=> select * from argo_archived_workflows order by finishedat;

and this has yet to return after a few minutes. We are using db.r5.large RDS instances and the metrics seem fine.

I believe this has always been an issue, but cannot say for sure. I'm suspecting this is a retention problem in that the default settings are not working:
argoworkflows=> select count(*) from argo_archived_workflows;
 count
-------
 10810
(1 row)

I'm currently running
argoworkflows=> select * from argo_archived_workflows order by finishedat;

and this has yet to return after a few minutes. We are using db.r5.large RDS instances and the metrics seem fine.

This is probably due to the combined size of the values of the json column and a red herring.

I believe this has always been an issue, but cannot say for sure. I'm suspecting this is a retention problem in that the default settings are not working:
argoworkflows=> select count(*) from argo_archived_workflows;
 count
-------
 10810
(1 row)

I'm currently running
argoworkflows=> select * from argo_archived_workflows order by finishedat;

and this has yet to return after a few minutes. We are using db.r5.large RDS instances and the metrics seem fine.

This is probably due to the combined size of the values of the json column and a red herring.

Here is an update.  We figured out what the retention policy above does.  It controls how many workflows are shown (and presumably kept in controller memory) with argo list.  We are slowly increasing the values for our retention settings to see how memory behaves, but I suspect the json representation for each workflow stored in the backend DB is also kept in memory which causes the controller to consume a large amount of memory.  The json value for each workflow can be very large.

@bwmetcalf thanks for the update, that does make a ton of sense, I suspect the status is also stored in memory, which would likely be in the 100s of kb (if not a few mb) already for some large workflows.

Perhaps the fix is to store basic metadata about the workflows in memory and query the DB as needed for argo commands?

I also found the memory of controller keep growing. if the controller has the memory limit of 12GB, I think the memory leak of controller need to be fixed with high priority.

Might be relevant


Request for Opening Dua/Surah (Customizable) + Potential for muting?

Salaamun Alaikum!
First off, my sincerest gratitude @DBChoco and team for creating a wonderfully intuitive program that is lightweight. InshaAllah Ta'Aala may Allah SWT reward you for your efforts in the thousands ameen.
Please let me know if my requests are not feasible. I've yet to find a program as good as this one MashaAllah so these are not gripes, but more so suggestions to further improve usability.
Primarily, a feature to play an opening dua/surah of the users choice before or after the Adhan would be a valuable addition, in my humble opinion. This option would perhaps mirror the custom adjustment tab, as well as the adhan tab in certain ways. I've taken the liberty to mock up what i believe this page would look like, but my background in UI rather than code keeps me from implementing it unfortunately XD. please let me know if this is something that you or your team could implement, as hearing the Holy Quran or maybe a select dua would be a welcome addition to most users.

Secondly, the Shi'a Madhab from what i understand only recite the Adhan for Fajr, Dhuhr, and Maghrib - as the Asr and Isha prayers are prayed directly after the Dhuhr and Maghrib prayers respectively. A couple Shi'a friends i know stated that whilst there is no problem with the additional beautiful recitation of the Adhan, they have already prayed the Asr and Isha prayers and so the Adhan calls them to pray what has already been offered, Subhan'Allah. Additionally, im sure the option to mute a certain adhan may help students or others who may not have access to a room all the time (My university had a multi faith room which was only accessible at certain times, some of which did not correspond with the time of the Salaat), but the out loud recitation of the Adhan during these times would perhaps disturb others (non Muslims) using the room at certain times. I understand this is a niche use case, but i believe it is one that adds additional versatility to the app, which is never a bad thing. I have also mocked up what i believe those updates would look like, as seen below.

These mockups were done rather quickly, so if you'd like additional work to be done, please do let me know. Additionally, if you would like an additional hand with regards to UX/UI, i'd be more than happy to assist.
Again, Jazak'Allah Khairan for putting this together. May Allah SWT increase you ameen ya Rabb. If you have any questions please reach out.
Salaamun Alaikum

Wa Alaykoum Assalam wa Rahmatou Lahi wa Baraketu brother !
Thank you for the suggestions, let me tell you what I think about them:

I don't know of any Sunnah of the Prophet PBUH that indicated that they'd listen to Qur'an or anything before the Adhan; and I wouldn't like to innovate something which our beloved Prophet PBUH didn't come with. Thus I'm against this.
This is a very nice feature, the mute button. Strangely enough it didn't cross my mind until now; Jazaka Lahu Khairan ! This has nothing to do with the Shi'a, with all due respect; it is just a nice feature to have in all cases.

I'll keep it in mind inshaAllah; I've been extremely busy for the past year or so, so the software didn't improve much in that time frame; I'll do my best inshaAllah.
Baraka Lahu feek for all the help ! Wa Alaykoum Assalam wa Rahmatou Lahi wa Baraketu brother !


Refactor to address GC not ready code duplication

As suggested in issue https://github.com/RJacksonm1/node-dota2/issues/107 I added a helper function and then updated all the handlers to use them.
I've only done one quick test so far which is logging a bot on and it was able to join a channel without a problem. Haven't done a test showing the actual log, but the change is trivial so I don't see why it wouldn't work. EDIT: Ran a test setting _gcReady to false and it logged properly.

Thanks, waiting for ACK from @paralin

I don't think this solves much at all, it just moves where the 'util.log' statement resides. All of our GC interfacing code all boil down to doing 2 things: Building a payload, sending that payload to the GC (with the relevant type); so I think we can improve this by replacing the  this._client.toGC with a custom method, e.g. this.sendToGC(type, payload, [callback]); this function can do the GC ready check, as well as abstract away some of the other "noise" that our methods don't really need to concern themselves with.
For example, instead of having:
Dota2.Dota2Client.prototype.joinChat = function(channel, type) {
  type = type || Dota2.DOTAChatChannelType_t.DOTAChannelType_Custom;

  /* Attempts to join a chat channel.  Expect k_EMsgGCJoinChatChannelResponse from GC */
  if (!this._gcReady) {
    if (this.debug) util.log("GC not ready, please listen for the 'ready' event.");
    return null;
  }

  if (this.debug) util.log("Joining chat channel: " + channel);
  var payload = dota_gcmessages_client.CMsgDOTAJoinChatChannel.serialize({
    "channelName": channel,
    "channelType": type
  });

  this._client.toGC(this._appid, (Dota2.EDOTAGCMsg.k_EMsgGCJoinChatChannel | protoMask), payload);
};

We could refactor things so we only need:
Dota2.Dota2Client.prototype.joinChat = function(channel, type) {
  type = type || Dota2.DOTAChatChannelType_t.DOTAChannelType_Custom;

  if (this.debug) util.log("Joining chat channel: " + channel);
  var payload = dota_gcmessages_client.CMsgDOTAJoinChatChannel.serialize({
    "channelName": channel,
    "channelType": type
  });

  this.sendToGC(Dota2.EDOTAGCMsg.k_EMsgGCJoinChatChannel, payload);
};

So we abstract away the GC ready check, the passing of appid, and the bitwise OR with proto type and protomask.  In this example it doesn't clear up a lot, but it would reduce a lot of duplication across the whole library.
This is something I was going to look into when I next have time to work on the library, but I'm more than happy if anybody else wants to take on this refactoring work.  That's assuming there are no glaring problems with this approach that I haven't identified.

I agree with Rob, there's a lot more simplification possible here.
On Thu, Sep 17, 2015, 5:54 AM Rob Jackson<EMAIL_ADDRESS>wrote:

I don't think this solves much at all, it just moves where the 'util.log'
statement resides. All of our GC interfacing code all boil down to doing 2
things: Building a payload, sending that payload to the GC (with the
relevant type); so I think we can improve this by replacing the
this._client.toGC with a custom method, e.g. this.sendToGC(type, payload,
[callback]); this function can do the GC ready check, as well as abstract
away some of the other "noise" that our methods don't really need to
concern themselves with.
For example, instead of having:
Dota2.Dota2Client.prototype.joinChat = function(channel, type) {
type = type || Dota2.DOTAChatChannelType_t.DOTAChannelType_Custom;
/* Attempts to join a chat channel.  Expect k_EMsgGCJoinChatChannelResponse from GC */
if (!this._gcReady) {
if (this.debug) util.log("GC not ready, please listen for the 'ready' event.");
return null;
}
if (this.debug) util.log("Joining chat channel: " + channel);
var payload = dota_gcmessages_client.CMsgDOTAJoinChatChannel.serialize({
"channelName": channel,
"channelType": type
});
this._client.toGC(this._appid, (Dota2.EDOTAGCMsg.k_EMsgGCJoinChatChannel | protoMask), payload);
};
We could refactor things so we only need:
Dota2.Dota2Client.prototype.joinChat = function(channel, type) {
type = type || Dota2.DOTAChatChannelType_t.DOTAChannelType_Custom;
if (this.debug) util.log("Joining chat channel: " + channel);
var payload = dota_gcmessages_client.CMsgDOTAJoinChatChannel.serialize({
"channelName": channel,
"channelType": type
});
this.sendToGC(Dota2.EDOTAGCMsg.k_EMsgGCJoinChatChannel, payload);
};
So we abstract away the GC ready check, the passing of appid, and the
bitwise OR with proto type and protomask. In this example it doesn't clear
up a lot, but it would reduce a lot of duplication across the whole library.
This is something I was going to look into when I next have time to work
on the library, but I'm more than happy if anybody else wants to take on
this refactoring work. That's assuming there are no glaring problems with
this approach that I haven't identified.
—
Reply to this email directly or view it on GitHub
https://github.com/RJacksonm1/node-dota2/pull/109#issuecomment-141027372
.

Superseded by https://github.com/RJacksonm1/node-dota2/issues/110


Update Swig Defaults

The previous defaults setting is apparently invalid.  Express will not cache templates if swig is told not to.  This setting is safer.

Thanks @emaddoma


find usages/references

It's a feature request:
Use something like haskell-indexer to implement functionality such "find usages/references"

Closing in favor of #868


Licenses

Hello,
The readme says individual audio file licenses are availible in audio_licenses.txt, but I cannot find this file--could you point me to it?
Some audio files in this dataset (ex. https://www.jamendo.com/track/214/intro-chiante) seem to now be under the Jamendo license, not a CC license. Has the licensing changed?
Thank you!

Hi @wagnew3,
The audio_licenses.txt file is located in the root folder of the repository: https://github.com/MTG/mtg-jamendo-dataset/blob/master/audio_licenses.txt.
We are not aware of any changes to the licensing on the Jamendo website, but such changes could occur. We report the licenses that the tracks had at the time of compiling the dataset.


Sorting a book as myself, changes the update time of a page for another user

Describe the Bug
Updated time will update when sorting, but not the user
Steps to Reproduce
go to page, check created and last updated time

then go to book the page is in
select sort
move the order of the page
save
then go back to the page and the updated time will change, but not the user

additionally, the updated time of the book wont change
Expected Behaviour
I think there are 2 different expected outcomes:

sorting the book will only affect the last updated time of the book, with no changes to the page
sorting the book will affect the last updated time of the book, also updating the page last updated time and user

Screenshots or Additional Context
No response
Browser Details
Chrome
Exact BookStack Version
23.08.3

Thanks @LittleBennos, but I'm going to close this off since the fundamental request has already been raised in prior issue #1777.


Microsoft.AppxPackage.Targets causes build failures

Customer reported the following issue:
Description: When attempting to run a build using MSBuild in a batch file, projects are failing due to the following exception:
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\AppxPackage\Microsoft.AppXPackage.Targets(283,22): error MSB4086: A numeric comparison was attempted on "10.0.11000.0" that evaluates to "10.0.11000.0" instead of a number, in condition "'$(TargetPlatformVersion)' >= '10.0.11000.0'". [MyProject.csproj]

This appears to be caused by the following segment within Microsoft.AppXPackage.Targets:
 <PropertyGroup>
 <SdkIsRS1OrLater>False</SdkIsRS1OrLater>
 <SdkIsRS1OrLater Condition="'$(TargetPlatformVersion)' >= '10.0.11000.0'">True</SdkIsRS1OrLater>
 </PropertyGroup>

Switching the VisualStudioVersion flag in the batch file from 14.0 to 12.0 causes this batch file to build successfully. Additionally, it was running fine with 14.0 prior to the upgrade to Update 3.
Strangely, this appears to build fine when running directly inside Visual Studio.
Visual Studio Version: 14.0.25421.03
Repro Steps:Create a batch file with the following script. When running the build, the error will surface.
 @echo off
 set CLRHome=%SystemRoot%\Microsoft.NET\Framework\v4.0.30319
 set MSTestDir=%vs140comnTools%..\IDE
 Set VisualStudioVersion=14.0
 set Path=%Path%;%CLRHome%;%MSTestDir%

 MSBuild ./LocalBuild.proj /v:m /clp:Summary; /m:%multiCore% /nr:False /nologo

Impact: Builds are unable to run with ToolsVersion 14.0. Any developer running exclusively 2015 updating to Update 3 will be unable to build the solutions from the batch files.

Starting point: investigate changes on Property functions before/after Dev14 Update 3.

I have attached a sample application that is seeing this issue. With VS2015 Update 3 installed, execute RunLocalBuild.cmd within the solution folder.
Commenting the line in the .targets file (noted above) stops this issue occurring.
DemoApp1.zip

I have also encountered this with the following simple repro steps:

Create a UWP app in Visual Studio
Load the unmodified .csproj file using the constructor for `Microsoft.Build.Evaluation.Project' which takes a string for file name
Observe the exception

@vchelaru How are you referencing MSBuild in the project that builds the code in which you do the project load? Do the references point to version 14.0, or 4.0?
There are two copies of MSBuild on most machines--one (4.0) installed with the .NET Framework (removing it would be a breaking change in the framework, so we leave it there, stale), and one (14.0) installed with Visual Studio or the Build Tools installer. New functionality is available only in the up-to-date assembly, and version-number comparisons were added in the 12.0 timeframe (I think).

@rainersigwald thanks so much for taking the time to respond.
I am referencing the following assembly:
#region Assembly Microsoft.Build, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
// C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework.NETFramework\v4.6\Microsoft.Build.dll
#endregion
So you're saying that I could reference the .dll that comes with Visual Studio to work around this issue? If so, does that mean I will have to distribute the .dll with my app? I can't count on users of my app to have VS installed, so I can't use the VS-installed version in its install location.

@vchelaru That is a stale version of MSBuild. To understand UWP projects within your app, you'll have to use the newer binaries. To get MSBuild 14.0 installed, you can use the Build Tools 2015 Update 3 installer--you don't have to have all of VS installed.
Right now, that installer is the only way to get MSBuild installed. For the next release, we should have official NuGet packages that you could consume.

Just to confirm, this does appear to be an MSBuild version issue. If I replace the top of the batch file from
@echo off
set CLRHome=%SystemRoot%\Microsoft.NET\Framework\v4.0.30319
set MSTestDir=%vs140comnTools%..\IDE
Set VisualStudioVersion=14.0
set Path=%Path%;%CLRHome%;%MSTestDir%

MSBuild ./LocalBuild.proj /v:m /clp:Summary; /m:%multiCore% /nr:False /nologo

to
@echo off
call "%vs140comnTools%\VsDevCmd.bat"

MSBuild ./LocalBuild.proj /v:m /clp:Summary; /m:%multiCore% /nr:False /nologo

This problem resolves.

Hello,
We're also repo'ing this in our area; to work around it we've commented out Line 283 in Microsoft.AppxPackage.Targets:
<!--<SdkIsRS1OrLater Condition="'$(TargetPlatformVersion)' >= '10.0.11000.0'">True</SdkIsRS1OrLater>-->
Thank you

I fixed this by specifying ToolsVersion="14.0" in the .csproj, rather than some old 4.0 or 12.0 I had on some projects.

Are there any updates to this? we are currently dealing with a broken TFS 2012 build server due to this.
We have a Xamarin PCL project with UWP, WinPhone (8.0, Silverlight) & Android projects.
If I don't set the tools version, I get this error in the UWP project.
C:\Program Files (x86)\MSBuild\Microsoft\WindowsXaml\v14.0\Microsoft.Windows.UI.Xaml.Common.targets (6): A numeric comparison was attempted on "$(TargetPlatformMinVersion)" that evaluates to "10.0.10240.0" instead of a number, in condition "'$(TargetPlatformIdentifier)' == 'UAP' And '$(TargetPlatformMinVersion)' != '' And '$(TargetPlatformMinVersion)' > '10.0.10587.0'".
If I set the tools version (/tv:14.0) I get the following (x4) as well as the one above.
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\AppxPackage\Microsoft.AppXPackage.Targets (283): A numeric comparison was attempted on "10.0.11000.0" that evaluates to "10.0.11000.0" instead of a number, in condition "'$(TargetPlatformVersion)' >= '10.0.11000.0'".
Has anyone actually got this working on a build server?
We have actually ended up trying to install VS2015 on the build server, but this still doesn't work, as well as installing the MSBuild tools update 3 from the link above.

@StacyHenwood If you are using v14 of the tools you must use MSBuild v14 (ie the one from the Visual Studio 14.0 tools folder) to build your solution.
I suspect your TFS Build server will either be using MSBuild v12 from Visual Studio 12.0 tools folder or v4.x from the .net framework. I don't recall how this is controlled for your build but it might the the ToolPath property.

@Nyami Thanks for the information. That's what I thought I was doing, by setting the /tv:14.0 and /p:VisualStudioVersion=14 settings in the MSBuild parameters in the build template (TfvsTemplate.12.xaml). Note that this template does not have a ToolsPath option, so I cant just update it there.
When I look in the logs I see some very strange things.
Near the start I see the following:
Run optional script before MSBuild
MSBuild auto-detection: using msbuild version '14.0' from 'C:\Program Files (x86)\MSBuild\14.0\bin'.
But further on, where the errors are, I see this (note: wrong tools version):
Run MSBuild
C:\Program Files (x86)\MSBuild\12.0\bin\MSBuild.exe /nologo /noconsolelogger "C:\Builds\2[SNIP]
My google-fu must be weak today, because I cannot find out how to make the build server use the correct version of the tools with that template.

I think the path for MSBuild will be set on the "Run MSBuild" activity within TfvsTemplate.12.xaml, if this is the case you should update the template.

@Nyami I have previously looked at that stackoverflow article, and discovered that it is not using the TfvcTemplate.12.xaml template - it is using an older one, and the parameters have changed significantly.
The TfvcTemplate.12.xaml template does not have the ToolPath parameter within the Run MSBuild parameter.
I appreciate your continued help.

Ok, so I have managed to get the correct version of MSBuild to run in my builds.
Its definitely a hack, but its the only way I could get this to work properly.
I have changed the following Registry keys:
HKLM\SOFTWARE\Wow6432Node\Microsoft\MSBuild\12.0\MSBuildOverrideTasksPath
HKLM\SOFTWARE\Wow6432Node\Microsoft\MSBuild\ToolsVersions\12.0\MSBuildToolsPath

from:
C:\Program Files (x86)\MSBuild\12.0\bin\
to:
C:\Program Files (x86)\MSBuild\14.0\bin\
I realise that this is most definitely not going to be the recommended way of doing this, but there seems to be no way of forcing TFSBuild to use the correct version any other way.

What is the status of this - is it still work in progress? Still experiencing this issue with VS15 Update 3, and have managed to resolve by commenting out line 283 in Microsoft.AppxPackage.Targets as per a previous recommendation.

Closing this as there's nothing more we can do from our side. The entry-point MSBuild must be a new enough MSBuild.


Add co-simulation in sim pass

support FST as output for sim pass
"n" parameter now explicitly use clock cycles
new parameter "r" to use FST as input for co-simulation
"sim", "sim-cmp", "sim-gate" and "sim-gold" to select co-simulation mode
"start","stop" or "at" parameter to specify time (default unit is ns, can use anything from zs to s)
"scope" to specify top scope in input sim file

This seems to be using zlib even with ENABLE_ZLIB=0 and broke emcc build.

@yuyichao Thanks for reporting this, FST files library is using zlib directly, so in making this respect ENABLE_ZLIB will make this option not available in emcc build, which is probably fine for most users.

It is actually possible to build with the -s USE_ZLIB=1 emcc flag. However, ENABLE_ZLIB (and the emcc flags) aren't included by default which is the problem.
If I read the code correctly, isn't zlib just one of the output options? Shouldn't other formats still work even if zlib is disabled?

Yes, zlib is one of the output/input options but problem is that external API used  is not made with ZLIB as optional, so I  have to see about effort of making it optional.

I see. Yeah, enabling/disabling the whole feature depending on enable_zlib should be fine for now. The user can still manually enable it and supply the right flag to pull zlib to build.


[Cumulus Tutorial] Fix collator block formation

Updating the tutorial documentation for Connecting a Parachain Adding the bootnodes parameter to collator node start command. This connects the embedded relay node to local relay chain so blocks will form on the parachain after registered and an epoch has passed.
Linked issue: https://github.com/substrate-developer-hub/substrate-docs/issues/707

This is a good addition, but maybe overkill for local testing , as mdns discovery for most people works out of the box, and no bootnodes are needed. Can you comment on how you are trying to spin these up and your network configurations? It could be your firewall is causing issues?

@NukeManDan You are right, the MacOS firewall seems to be blocking my local mdns between the collator and my local relay. I'll update my PR to instead be a note about if the relay isn't connecting to try turning off firewall or setting the bootnodes parameter.


Delete files in import folder after sucessful import

CSAF files that have been imported during startup of the server remain in the import folder. The server will try to import them during every start.
Probably it would be better to delete the successful imported files. Errors during import could be logged in a file, so a user can see, why they haven't been imported.

I'm not sure whether deleting is the right way but maybe moving to a different folder, e.g. already_imported? :thinking:


[WIP]Wrap delegate

An alternative PR that fixes #984.
@zvirja, @moodmosaic, this is the approach that I'd originally advocated in https://github.com/AutoFixture/AutoFixture/pull/986#issuecomment-368730314:

I believe we could return a faked delegate that by default returns the automatically generated value. Then users would never know that anything changed, unless they were examining the type of returned delegate (which seems silly), or they attempted to configure it as a Fake, in which case I'd think they'd be pleased with the result.

@zvirja has already indicated that this approach is not desirable, although I thought that the reasoning was mostly around a perceived clash with what's normally called "Configured" customizations where the faked (mocked/substituted/rhinoed) objects are configured to return values from AutoFixture. I still think that this approach doesn't conflict, since it's all about changing the abilities of specimens that would previously have been supplied by AutoFixture kernel, not by AutoFakeItEasy.
All that's a bit of an aside, as I'm mostly pushing this because for a long time I couldn't even figure out how to accomplish this effect, and now that I have, I was hoping for a critique or this approach and/or discussion of an alternative approach.
All the juicy changes are in my second commit; the first one just grabs DelegateSpecification as was created in #986. But in 291e3ee, the approach I took was to, when FakeItEasy supports faking of delegates, to introduce a FakeItEasyRelay into customizations that would relay delegate requests to Fake<TDelegate>, as well as a builder that would create a Fake<TDelegate> by using DelegateGenerator to produce a regular AutoFixture-supplied delegate specimen, and then wrapping that delegate in a Fake (so the behaviour still looks like the AutoFixture delegate until you configure it).
Explicitly referencing the DelegateGenerator felt a little dirty, but I couldn't otherwise figure out how to pass the request for the delegate along to the AutoFixture kernel without having be picked up by the FakeItEasyRelay (or falling prey to the recursion guards).
Would my approach be considered reasonable, or is there a more idiomatic approach that I'm missing?

@zvirja, I'm perfectly content to proceed as in #986 (which I see you've merged; thanks for that!). After all, your project, your rules. :wink:. I'm (mostly) just chasing information here (even though I retain an emotional attachment to the "wrapping" approach). To that end, if you have time to continue the conversation with me, I'd appreciate it. If not, say so, and I'll stop bothering you, and with no hard feelings!

it might happen built-in delegates might not behave like faked ones (e.g. for the ref/out parameters),

I suppose this is possible. I don't think there'd be an issue, but it'd be easy to test.

It might be unclear why after the update their own delegate builder suddenly stopped to work.

Yes! This is the main thing I was trying to ask about. I puzzled over the AutoFixture API and could not figure out a way to provide a customization that would cause a Fixture to allow my new builder to wrap an otherwise-configured specimen value. I'm looking for a way to say in the builder "forward the request to the downstream builders (the next links in the chain of responsibility), and then act on the result". The closest concept I found within AutoFixture was the Pipeline, but even that only modifies values produced by the builder passed to the Pipeline object. Everything I tried (e.g. recording the fixture during customization and using it, or instead using Resolve) resulted in passing a request back up to the full fixture, where I'd invariably get a recursion error. Explicitly targeting DelegateGenerator was the only thing I could think of that would work, even though I knew it was wrong. Even though I expected the approach I pushed here to be rejected, it's become a puzzle, and I desperately want to know how to do it, even if I shouldn't.

I'm (mostly) just chasing information here (even though I retain an emotional attachment to the "wrapping" approach).

Given that we already merged the other approach, there is a gap - faked delegates do not return auto-values. If you wish, I'd like to ask you to implement the Configured mode for the FakeItEasy integration - it would be really awesome to have it. I love how it's done in the NSubstitute and would recommend that as a sample:

automatically generate value as a method result/properties/fields. Return the same result for the same args;
auto generate the ref and out parameters;
the same for delegates;
fully support the open generic methods;

If you wish, please create a PR for this feature. Nobody requested it so far, but it would be cool to have it.

Even though I expected the approach I pushed here to be rejected, it's become a puzzle, and I desperately want to know how to do it, even if I shouldn't.

Indeed, if you put your builder into the Customizations or ResidueCollectors collections, your builder will suffer from the recursion issues if you perform context.Resolve(request).
Probably, as a workaround, you can the Behaviors collection. In fact, it acts like as a pure Chain Of Responsibility pattern. Behavior allows you to proceed with the request and later somehow interact with the result. In your case, you can allow fixture to create the delegate using the default builder, wrap result into the faked delegate and return it. That is the best I could advise with the current kernel implementation and I don't see other easy options. If you try it, I'm almost confident it will work.
However, even with that approach, I'm still stick to the already merged way - I love the consistency it provides 😌

If you wish, please create a PR for this feature. Nobody requested it so far, but it would be cool to have it.

Sure. I'll warn you, though. I'm going on vacation in about a week and may not be able to submit any code until early April. If that's not a problem, I'll proceed.

If that's not a problem, I'll proceed.

Definitely not a problem. We never had this feature - so it's fine to wait more.
Have a happy vacation 💃
Closing this PR as we are not going to merge it.


TypeScript: Add menu item should have more options

This is an NTVS project menu

Ordinary TypeScript projects also have these further options in another item group just below:

TypeScript File
HTML Page
JavaScript File
Style Sheet

If we do this, we should also do something similar for JavaScript files.

The new set of options seem to be cropping up in duplicate in TypeScript (i.e. not NTVS) projects:


Mobile: fix return on hardware keyboards after backgrounding on iOS

@keybase/react-hackers r? @jzila
@patrickxb found that return-to-send on a hardware keyboard in chat on iPad wasn't working anymore after backgrounding and reforegrounding the app.  It's because our isOpen bool is incorrectly thinking there's a soft keyboard open after the reforeground.
We don't actually need to be testing isOpen at all on iOS -- it was a workaround for an Android behavior where soft keyboards emulate hardware keypresses -- so this PR switches to only considering the state of isOpen on Android.

I tested it and it works!


chore: 🚀 Remove Workflow Debugging

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Team 2 Upgrades

Description
During the day, the NPC (penguin) enters and you have 10 seconds to decide whether you want to interact with it. You can only interact with the NPC by clicking on the penguin when the hover box pops up. Once you interact with the NPC, the game pauses, and a UI popup appears with an upgrade decision and YES/NO buttons. A message of not enough gold will show if you don't have enough gold to purchase the upgrade. The NPC will only come once a day at a random time.
Each upgrade will have a cost - monetary or moral - and a duration.
Examples (optional)

Speed up movement
Faster cooking time after fulfil a certain number of orders
Double gold from orders
Loan 100 gold

Some Related Feature and Task Tickets

#540
#295
#291
#292

Dependencies

Gold (team 6)
Order System (team 4) #8
Integrate Dockets with Customers (team 4) #142
Moral System (team 8) #102

Milestones

[x] Make upgrade classes - Speed boots, rage, extortion, loan (22/09)
[x] Apply random upgrade (from keybind) (29/09)
[x] Upgrade decision UI (27/09)
[x] Upgrade NPC vendor (2/10)

Completion Deadline: Oct 3
Members

Joanna (@Joanna-Wang105)
Calvin (@CalvinJohn99 )
Willem (@willembor )
Alex (@Ashom2 )
Ashwin (@AshHarikrishna )
Daleela (@daleelakhairil )


NHK World vods posted after September 26th, 2023 are broken

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I'm running yt-dlp version 2023.09.24 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
Canada (but presumably worldwide)
Provide a description that is worded well enough to be understood
It appears NHK World has changed their video player recently, as well as the underlying infrastructure. As a result, episode VODs posted after the transition can't be downloaded by yt-dlp.
There is a Reddit post with other people who have noticed the issue: https://reddit.com/r/NHKWorldFans/comments/16tkdyu/new_video_player_on_nhk/
Examples of URLs that still work for now:
https://www3.nhk.or.jp/nhkworld/en/ondemand/video/4026201/ (Document 72 Hours, Sept. 12)
https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2049133/ (Japan Railway Journal, Sept. 14)
Examples of URLs that don't work:
https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2049134/ (Japan Railway Journal, Sept. 28)
https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2073139/ (J-Arena, Sept. 29)
https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2093047/ (Zero Waste Life, Sept. 29)
The problem is that the Piksel movie-s.nhk.or.jp URL (https://movie-s.nhk.or.jp/v/refid/nhkworld/prefid/nw_vod_v_en_2049_134_20230928233000_01_1695913422) returns a "no data found" message from the server, instead of the usual HTML output the Piksel extractor uses to retrieve the app token and other stuff. A quick scan of the assets API endpoint (https://movie-s.nhk.or.jp/ws/ws_asset/api/67f5b750-b419-11e9-8a16-0e45e8988f42/mode/json/apiv/5?sortdir=desc) shows nothing new since September 26th, and attempting to retrieve episode details using that API for the affected episodes returns no results (compare https://movie-s.nhk.or.jp/ws/ws_asset/api/67f5b750-b419-11e9-8a16-0e45e8988f42/mode/json/apiv/5?title=%25umineko%25&sortdir=asc&start=0&end=99 which returns the Document 72 Hours episode above that works fine, to https://movie-s.nhk.or.jp/ws/ws_asset/api/67f5b750-b419-11e9-8a16-0e45e8988f42/mode/json/apiv/5?title=%25hokuriku%25&sortdir=asc&start=0&end=99 which returns two Cycle Around Japan episodes but NOT the Japan Railway Journal episode that also includes "Hokuriku" in the title).
It is possible to retrieve video streams from the new episodes, though I can't find any 1080p versions anymore. On the NHK World VOD pages, a script calls api01-platform.stream.co.jp/apiservice/getMediaByParam/, which returns a JSON response with a movie_url object containing several m3u8 URLs. These contain a 320x180, 640x360 and two 1280x720 stream playlists. Dropping one of the playlist URLs directly into yt-dlp works fine, though I do get a warning "Live HLS streams are not supported by the native downloader. If this is a livestream, please add "--downloader ffmpeg --hls-use-mpegts" to your command".
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2049134/', '-vU']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[088add956] (pip)
[debug] Python 3.10.8 (CPython AMD64 64bit) - Windows-10-10.0.22621-SP0 (OpenSSL 1.1.1q  5 Jul 2022)
[debug] exe versions: ffmpeg git-2020-08-09-6e951d0, ffprobe 6.0-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2<IP_ADDRESS>, mutagen-1.45.1, sqlite3-3.37.2, websockets-10.3
[debug] Proxy map: {}
[debug] Loaded 1886 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version<EMAIL_ADDRESS>Current version<EMAIL_ADDRESS>yt-dlp is up to date<EMAIL_ADDRESS>[NhkVod] Extracting URL: https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2049134/
[NhkVod] 2049-134: Downloading JSON metadata
[Piksel] Extracting URL: https://movie-s.nhk.or.jp/v/refid/nhkworld/prefid/nw_vod_v_en_2049_134_20230928233000_01_1695913422
[Piksel] nw_vod_v_en_2049_134_20230928233000_01_1695913422: Downloading webpage
ERROR: [Piksel] nw_vod_v_en_2049_134_20230928233000_01_1695913422: Unable to extract app token; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "C:\Users\userX\AppData\Local\Programs\Python\Python310\lib\site-packages\yt_dlp\extractor\common.py", line 715, in extract
    ie_result = self._real_extract(url)
  File "C:\Users\userX\AppData\Local\Programs\Python\Python310\lib\site-packages\yt_dlp\extractor\piksel.py", line 82, in _real_extract
    app_token = self._search_regex([
  File "C:\Users\userX\AppData\Local\Programs\Python\Python310\lib\site-packages\yt_dlp\extractor\common.py", line 1263, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)

new system
prod: {
     postMessageTargetOrigin: window.location.origin,
     allowDomainListJsonUrl: "https://movie-a.nhk.or.jp/world/player/config/domain-list.json",
     geoApiUrl: "https://geocontrol1.stream.ne.jp/nhk-a-geo/check.jsonp",
     playerBaseUrl: "/nhkworld/common/player/tv/vod/world/player/",
     eq: {
         apiUrl: "https://api01-platform.stream.co.jp/apiservice/getMediaByParam/",
         subtitleApiUrl: "https://api01-platform.stream.co.jp/apiservice/getSubtitleList/",
         token: "NDc4NThCNTkxQzFCNkQ3ODA4NjcwNTZGREYzNURBNzM=",
         customMetadataNames: {
             domesticFlag: "domesticFlag",
             embedType: "embedType",
             aaType: "aaType"
         }
     },

in https://movie-a.nhk.or.jp/world/player/js/movie-player.js (minified)
https://api01-platform.stream.co.jp/apiservice/getMediaByParam/?token=NDc4NThCNTkxQzFCNkQ3ODA4NjcwNTZGREYzNURBNzM=&type=json&optional_id=nw_vod_v_en_2049_134_20230928233000_01_1695913422&active_flg=1
stream metadata
optional_id is not optional if you want to download the right programme

lots of m3u8s
the auto ones are the ones you want
site uses auto_pc, but mb_auto has more info and a higher-bitrate 720p format (in addition to the lower-rate one)
the js has a subtitleApiUrl but im not sure if it's actually needed, they appear to be included in the m3u8

for this link  - https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2049134/
(i used brave browser for this test ) press F12 to get into the debug menu - go to sources tab - scroll down the tree until you come to movieplayer<PHONE_NUMBER>52605262 (www3.nhk.or.jp/) in the drop down, look for a file that starts
with ?opid=nw_vod now juts right click on the file and select open in a new tab,
and you can now play the video. now use the extension called "the stream detector" - set it to yt-dlp  to grab the m3u8  and download the video.

While English-language videos at /en/ now download correctly I'm seeing that for example daily noon and evening news videos in Japanese at https://www3.nhk.or.jp/nhkworld/ja/ondemand/video produce:
[debug] Command-line config: ['-v', 'https://www3.nhk.or.jp/nhkworld/ja/ondemand/video/0451269387/']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[377e85a17]
[debug] Python 3.11.5 (CPython x86_64 64bit) - Linux-6.5.5-arch1-1-x86_64-with-glibc2.38 (OpenSSL 3.1.3 19 Sep 2023, glibc 2.38)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.12.0, certifi-2023.07.22, sqlite3-3.43.1, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1886 extractors
[NhkVod] Extracting URL: https://www3.nhk.or.jp/nhkworld/ja/ondemand/video/0451269387/
[NhkVod] 0451-269: Downloading JSON metadata
ERROR: list index out of range
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1567, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1702, in __extract_info
    ie_result = ie.extract(url)
                ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 715, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/nhk.py", line 205, in _real_extract
    return self._extract_episode_info(url)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/nhk.py", line 77, in _extract_episode_info
    episode = self._call_api(
              ^^^^^^^^^^^^^^^
IndexError: list index out of range

Not sure if these videos were ever intended to function correctly. When I first tried this a few days ago I couldn't download them and simply assumed it had the same root cause mentioned in this ticket.
These videos are intended for Japanese out of country so downloadable only outside of Japan. The site sends different HTTP response bodies depending on whether or not it perceives a request source IP address to be within Japan or outside of Japan. When outside of Japan the page shows:

By my understanding the videos highlighted with red border don't have any retention, NHK only ever offers the current day's video for download. These two examples are today's 7 pm news (ニュース7 aka nyusu 7) and noon news (正午のニュース aka shogo no nyusu).
"Within" Japan you'll get:

Basically asking you to to use you NHK Plus account to watch a show you missed or to get yourself an NHK World Premium subscription.
Is the list index out of range situation intentional for videos from /ja/ URLs?

@Contik list index out of range is not intentional. Could you open a new issue about this, please?

@bashonly sure thing, opened issue #8303


Profile "Playing" is not showing multiple instances anymore

When i had multiple instances of Visual Studio open it used to show each file under my profile activities, but now it only shows one.
Does discord modified how it works?

Most likely, yes. Discord shows only one instance of the application.
I checked on other applications, and I also can only one instance.

@Reavert i tested with multiple instances of VSCode and it does show all of the, i have 4 instances of Visual Studio 2022 (preview) open but it shows only one.

You see preview of two different extensions that you have installed. This is not "multiple instances" of the same extension.
The first extension is from the Crawl developer, the second is from leonardssh.
This extensions have different names, icons, and most importantly, the Discord application ID, which they use to interact with Discord.
That's why you see them.


lecternTrade throws error when it shouldn't(?)

whenever the villager has two trades which both don't have an enchanted book:

the script breaks and says this:

This has been fixed, thanks for the report!


Renovate configuration improvements

Follow up for #130
Let's change some settings in renovate config to provide a better user experience:

Enable auto merge by default when approved and can be merged
Run tests/e2e tests automatically for renovate PRs
As I remember, renovate can create a single PR to update multiple dependencies. Maybe we need to evaluate this setting?

@hiddenmarten What do you think about?

Enable auto merge by default when approved and can be merged

Automerge will require additional effort because of GitHub limitations, take a look here, anyway, I agree that it will be helpful.
I think it's better to auto-merge only dependencies with minor and patch updates to prevent breaking changes from major updates.
Automerge should be scheduled once a week, e.g. on weekends, as it isn't overwhelming for developers and CI.

Run tests/e2e tests automatically for renovate PRs

Sure, it would be nice

As I remember, renovate can create a single PR to update multiple dependencies. Maybe we need to evaluate this setting?

Yep, we can do it

Some additional improvements after the first renovate run:

bump Golang version in GHA together with Dockefile
bump Kubernetes version in Makefile together with Kubernetes go libraries.


Publish process should detect if LUIS is required but absent from publishing profile

Currently, if you publish a bot that uses LUIS, but the publishing profile does not have a LUIS resource, you get a misleading error:
Error - failed to bind luis prediction resource to luis applications
Yes, of course it failed! Because we have no keys set.
Rather than allowing this to fail, the publishing process should determine if all required resources are present and direct user to the "edit publishing profile" process if one or more is missing.
This would also apply to QNA maker.

Dupe of #7740


No Edit Server

I'd like the option to make a server where nobody else can dig or place blocks. (unless I make them a mod?) Then I could set up a platformer level where everyone else has to try to jump to the end without falling into lava. I could share a link to randos on twitter who can't mess up my level. Like Mario Maker. Also, springs.

This is in


Implement Cluster Scaling Threshold

Description
To ensure we do not react to fleeting changes in cluster utilization and capacity requirements, Replicator should evaluate utilization and cluster scaling constraints and require the cluster to violate these constraints a specified number of times before opting to take a scaling action.
This could be implemented as a -cluster-scaling-threshold configuration flag and associated configuration file directive with a sensible default of 3.
Under this new behavior, if the cluster is determined to be under heavy CPU utilization, the cluster would need to violate the CPU constraint 3 consecutive times before Replicator would choose to take scaling action.
This is a significant design change and commentary is not only welcome but encouraged.

Initial config thoughts:

config parameter as described setup in the standard way
within structs.State struct 'ClusterScalingRequests' or similarly named int type is created

Initial process thoughts:

if a cluster scaling event is requested the 'ClusterScalingRequests' is incremented and checked against the threshold
if the threshold is passed, the cluster will scale out
if the threshold is not passed, replicator will skip and wait for another run
if on the next round will still want to scale, point 1 and 2 is repeated
if on the next round the cluster does not want to scale, we ensure 'ClusterScalingRequests' is set to 0 and continue


Fix incompatibility of BacklogQuota

Motivation
there is  a change between pulsar2.7 and master
command
bin/pulsar admin namespaces get-backlog-quotas $tenant/$namespace

pulsar2.7 returns
destination_storage    BacklogQuotaImpl(limit=102400, limitTime=-1, policy=consumer_backlog_eviction)

but pulsar 2.9 returns
"destination_storage    BacklogQuotaImpl(limitSize=102400, limitTime=-1, policy=consumer_backlog_eviction)"

So we need a compat, let old REST/sdk user, can set and get limit
Verifying this change

add org.apache.pulsar.broker.resources.NamespaceBacklogQuotaCompatibleTest for test

Documentation
Check the box below and label this PR (if you have committer privilege).
Need to update docs?

[x] no-need-doc

It's a compatibility bug fix.

@merlimat @lhotari @eolivelli @codelipenghui PTAL, thanks

@Shoothzj I noticed the fix only fixes the compatibility issue when getting backlog quota, I think another compatibility issue is when using a new version client >= 2.8 but the broker which < 2.8, since the broker-side don't have limitSize and the client side will not pass the limit to the broker, so that the broker will use 0 as the value of value, it also should be a compatibility issue.
It's better to add a compatibility test to make sure we will not introduce breaking change for this part.
Thanks for the great work.

@codelipenghui PTAL again, thanks

@eolivelli @linlinnn PTAL, and should this cherry pick to 2.8 and 2.9 ? thanks

Are we breaking compatibility with 2.8 ?
2.7 is far away, and the current release is 2.8...
I wonder if it is better to restore compatibility with 2.7 and break against 2.8 or is it better to keep compatibility with 2.8 and add another point to the "breaking changes" ?
because IIUC we broke compatibility in 2.8 and now (we are at 2.9) it is too late

I have now understood better the final outcome of this patch.
I will cherry pick to branch-2.9 and prepare (again) 2.9.1


Final classes prevent Unit testing

I have been trying to run some unit tests on logic code that I have written using Fetch classes, Fetch, Request and RequestInfo.
But because they are final, they are not mockable.
Just wondering if they could be not-final, so that mocking and testing could be possible.

@Mithrandir21 The easiest way to do this would be to fork the project and make the classes you need non final.


SS 3.2 version

Could you make a branch with a composer.json that is ready for 3.2? The current version wants definitely a 3.1 version...

I would also appreciate this doing, or please advise if this module is being depreciated for SilverStripe v3.2 projects in favour of using https://github.com/Marketo/SilverStripe-Performable-Sitemaps

Have relaxed the constraint, it should now be ~3.1 which will allow compatibility with 3.2


multiple viewers can not connect to the same master?

I opened the JS demo test page on two terminals, using different client ID to connect to the same master, and only one terminal can display the screen at the same time. Also on AWS console test, occasionally will burst such errors, what is the reason for this?

What browser(s) are you using?

Closing due to inactivity.


Validation framework

I've created some base work for validating and responding to the user.
Client-side validation will use the javascript functions outlined in manager.js.
Server-side validation will respond by setting an error in the session and showing that error, then deleting it as it is shown. One thing I have not figure out how to do yet is how to get the form values through to the form after server-side validation fails in the case of a new object. (i.e., creating a new calendar, server-side borks, I lose all my form data) May not be the most important thing though since you really can't use this app without JS.
Anyway, wanted to send this code out to get a quick look at things before I start mass-producing front-end and back-end validations.

Looking good!  A few things to keep in mind:

The accessibility of form validation can be complicated and is usually overlooked.  The key is to make sure that the user's attention is brought to the validation message and that they can quickly navigate to the element that failed validation: http://webaim.org/techniques/formvalidation/
We do have a validation library in the framework, but it is old and I'm not sure if it is worth using.  www.unl.edu/wdn/templates_4.0/examples/#formvalidator.html


Errors happened when click on the small icon at the right of the address bar on Chrome

What Operating System?
Windows 10 Eduation

What steps will reproduce the issue?

Install the GenderMag Recorder's Assistant by downloading it from GitHub.
A small icon of it appeared at the right of the address bar. Click on it, and an “Errors” button appeared.

Click on the “Errors” button, and it showed the following errors:

What would you expect to be the outcome?
Clicking on the small icon will lead users to the homepage of GenderMag Recorder's Assistant, and no errors would show up.

All these details will help people to fix any potential bugs?
It seems the feature corresponding to the small icon was omitted during the development process. Since the icon appears directly after users install the GenderMag Recorder's Assistant, and users are likely to click the icon, its feature should be fullfilled.

Temporary fix in 93204f8


Split is undefined

I tried doing a simple split on a string, and it returned that the function was undefined. Something like the following:
r.table('companies').filter(r.row('name').downcase().split('test').count().ge(2)).run().then(function (result) {
    callback(result);
});

It returned that the function split was undefined. I got around it for now by doing a filter in pure JavaScript since it's a small dataset, but it may be affecting others.

Arg, thanks, it looks like I just forgot it.
Fix coming.

Great, thank you!

Thanks @badams2527 for the report. It's fixed in 1.16.12.


Fix: Add accept timeout to WebSocketTestSession

Pull Request Checklist

[X] New code has 100% test coverage
[ ] (If applicable) The prose documentation has been updated to reflect the changes introduced by this PR
[ ] (If applicable) The reference documentation has been updated to reflect the changes introduced by this PR

Description
I found a bug where WebSocketTestSession never throws a timeout if websocket.accept is not sent by the server.
This is a proposed fixed which passes the connect timeout value for the TestClient to the receive method. I also added a test case for it.
Thanks!
Kyle
Close Issue(s)

@all-contributors, please add @smithk86 for testing.


Things to discuss about "world" export target

I implemented the "world" export target (for issue #14) but we haven't decide how to handle some cases. Current implementation is very simple.
The "world" export target requires exactly one of the two properties "worldPath" or "worldName". Providing both of them will result in an error.
Problems

In current implementation "worldPath" can lead anywhere. It just creates the some additional folders in that path and exports files there. The folders are "<worldPath>/behavior_packs/<project_name>_bp" and "<worldPath>/resource_packs/<project_name>_rp". I'm not sure if that's ok. Maybe the export path should be relative to .../com.mojang/minecraftWorlds folder?
The exporter doesn't create the behavior_packs.json and resource_packs.json in the world. Maybe it should?
If there is behavior_packs.json and/or resource_packs.json file in the world and it has a reference to an existing BP/RP should the exporter export files to the path of that reference? What if the file points at a pack inside development_behavior_packs/development_resource_packs.

IMO exporter should create *_packs.json if it doesn't exists and if it exists, it should append packs, that are installed if they're not already added. It shouldn't care whether there are already other packs added etc.
As for worldPath on one hand I think it's redundant and can be removed, but on the other, if we add *_packs.json support, then we can't provide it for exact export type.

I think its fine... right?


⚠ Turnos 24x7 has degraded performance

In b01a509, Turnos 24x7 (https://reservas-dev.alternativasinteligentes.com/) experienced degraded performance:

HTTP code: 200
Response time: 2765 ms

Resolved: Turnos 24x7 performance has improved in 4e529ef.


Throw exception if instantiating IOSDevice on non-mac os platform

Description
Add an assert to verify we are not instantiating IOSDevice class on non-Mac OS platform. Doing this uncovered that most of the rest of the tests in this file were actually going down this path, so I fixed that too.
Related Issues
fixes #36008
Tests
I added the following tests:
I added a new test to verify the assert error on non-mac, and then fixed the many of the other tests to be on mac platform.
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
[x] I signed the [CLA].
[x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
[x] I updated/added relevant documentation (doc comments with ///).
[x] All existing and new tests are passing.
[x] The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require Flutter developers to manually update their apps to accommodate your change?

[ ] Yes, this is a breaking change (Please read [Handling breaking changes]). Replace this with a link to the e-mail where you asked for input on this proposed change.
[x] No, this is not a breaking change (shouldn't be).

Are we sure that we don't end up going down this code path on windows/linux machines?

I can say with confidence that within the tool code the only place this gets instantiated is in the IOSDevices class, which has bool get supportsPlatform => platform.isMacOS;. It's possible there's some other tool implementation that instantiates it, which is why I split this out into its own PR.
Looking at the origin of the platform check, it goes back over 3 years (here).


Support Manual Motor Control

For checking if set up is right

I think this gets wrapped up in the ESC mixer


Allow usage of ESLint v7

Current version of eslint-plugin-aura has an outdated dependency on<EMAIL_ADDRESS>which doesn't support the latest major version of ESLint (v7 which is close to a year old).
This results in broken builds:
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! Found<EMAIL_ADDRESS>npm ERR! node_modules/eslint
npm ERR!   dev<EMAIL_ADDRESS>from the root project
npm ERR!   peer eslint@"^6 || ^7" from<EMAIL_ADDRESS>npm ERR!   node_modules/@salesforce/eslint-config-lwc
npm ERR!     dev<EMAIL_ADDRESS>from the root project
npm ERR!   6 more (@salesforce/eslint-plugin-aura, ...)
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer<EMAIL_ADDRESS>|| ^4.0.0 || ^5.0.0 || ^6.0.0" from<EMAIL_ADDRESS>npm ERR! node_modules/@salesforce/eslint-plugin-aura/node_modules/eslint-plugin-compat
npm ERR<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR!   node_modules/@salesforce/eslint-plugin-aura
npm ERR!     dev<EMAIL_ADDRESS>from the root project

Dropping these changes as I've been told that his library is not officially supported and that we should use the force:lightning:lint CLI command.

@pozil actually it is now officially supported, but I am not entirely sure 😓 @trevor-bliss what'd you say?

Yes, you're right. There was a debate but the outcome is that it is officially supported but it needs to evolve.
I'll reopen the issue.


Make the "My name" button in to a dropdown menu

Making in to a dropdown menu allows us to have a place for things that are not specific submission or project.
By moving "Log out" in to this menu we also free up some space.
Start with these items:

[ ] My account
[ ] Activity feed (used to be in the notification bell, only for staff)
[ ] My flagged (only for staff)
[ ] Wagtail admin (only for staff)
[ ] Log out

Might be a good idea to change the url for the activity feed from "/activity/notifications/" in to something better. They are not really notifications.

@theskumar @frjo @wes-otf Here are the various versions of user menu button and dropdown. Please check and provide your opinions.

Simple transparent button with border.

Button without border

User circle icon with text and without border(just like 2nd but circle icon)

Only Icon as a button(no text)

Same as 4th with header text in dropdown(text is same as it was in the button).

Well presented options! I lean towards 3 or 5.
Option 5 looks the best to me, but users might find it more intuitive with 3?


Possible to add scholar_id array to Publication objects that get populated upon Author fill

The author_id or scholar_id array which contains the Google Scholar IDs of the author(s) of the Publication would be very useful if possible to include in the Publications array attached to an Author.
Currently, it seems the only way to get this, is to get the search_authors results, fill the Authors (including 'publications'), then search for each publication and get the result of that, because even calling fill on the publication object directly returned still does not give one the author_id array.

TL;DR: It's a limitation of Google Scholar, not just scholarly.
The way you mentioned is exactly how you can find theauthor_ids of a publication's authors in Google Scholar, and scholarly as a scraper library is implemented to do exactly that. Since Google Scholar doesn't link the profiles of all the authors to the publication object reachable from a Google Scholar profile, scholarly can't fetch it directly either. Even when it links Google Scholar profile to publication searches, it does so only for the first few authors and is not guaranteed to give a complete list author_id of all coauthors.
Rather than searching for publication (which is subject to being blocked), you can get the complete list of coauthors' names for a publication, search for the author's profile by name using the search_author API, verify that this publication is listed on their GS profile if you like to be sure and then append that scholar's author_id to the list. This would be a much preferred way without triggering captchas. But all of this must come from your/application's end, not from scholarly itself.

I put together a notebook based on some code I had. This shows that it's simple for an application to do what you want, but complex for the library to do it for you.
https://colab.research.google.com/drive/1BIvZoe-mp_z9M-zuX__LCN3tNgO7Qsyt?usp=sharing

Thanks for quick/prompt reply and explanation (and of course all the work on this great lib), despite what was probably not the best description on my end looks like you knew exactly what I was referring to.

@arunkannawadi  sorry one more question -
So on the FILLED author objects with the coauthors section - is there any way around the limit of 20 coauthors? I see this getting printed "Fetching only the top 20 coauthors" and no param for changing?
And is there any sort of sorting or selection mechanism for the list that does come through there? e.g. can we say those are their top 20 coauthors based on total # of citations across jointly authored papers, or is that far from the truth?

Please see the optional dependencies section in our README. You'll need to install geckodriver or chrome-driver to get around the limitation of getting only 20 coauthors.
The list you see without it, is the list of 20 (max) coauthors that shows up in one's Google Scholar profile. I am not entirely sure about the ordering, but it's static as far as I know.
In my notebook, I could have followed the approach of going through the coauthor's Google Scholar profile instead of searching for profiles by names. I did not do that because adding co-authors to a profile needs manual intervention. Google Scholar only suggests adding coauthors, and does not add them automatically. This means many people (including me) have at best only a partial list of coauthors, even amongst those who have a Google Scholar profile.

Very useful information, thanks again. I had devised a way to efficiently build a network graph using only the search_author_id and fill author methods, without need for proxy, but seems the main limitation (even if able to pull more than the 20 coauthors from the profile) is that the list is still incomplete and fully a function of how well they've updated it.

You should use search_author methods when you know there are coauthors that are not listed. The main limitation in that case will be authors who don't have a public Google Scholar profile.


Problem "electron-prebuilt-compile" not a range (got "^8.2.0")

Hi Sr.
Thanks for your amazing job.
In this case, I have a issue and idk how to solve it.

My packaje.json
{
  "name": "MultImp",
  "productName": "MultImp",
  "version": "1.0.0",
  "description": "Relleno de datos meteorológicos mediante imputación múltiple",
  "main": "src/index.js",
  "scripts": {
    "start": "electron-forge start",
    "make": "electron-forge make"
  },
  "keywords": [],
  "author": {
    "name": "Miguel Angel Gonzalez",
    "email"<EMAIL_ADDRESS>  },
  "license": "MIT",
  "repository": {
    "type": "git",
    "url": "https://github.com/zarathucorp/shiny-electron-template-windows"
  },
  "dependencies": {
    "axios": "0.27.2",
    "electron-squirrel-startup": "^1.0.0",
    "esm": "^3.2.25",
    "execa": "^5.1.1",
    "pidtree": "^0.6.0"
  },
  "devDependencies": {
    "@babel/core": "^7.22.20",
    "@babel/plugin-transform-async-to-generator": "^7.22.5",
    "@babel/preset-env": "^7.22.20",
    "@babel/preset-react": "^7.22.15",
    "@electron-forge/cli": "^7.2.0",
    "@electron-forge/maker-zip": "^7.2.0",
    "electron": "^26.2.1",
    "electron-prebuilt-compile": "^8.2.0",
    "fs-extra": "^11.1.1"
  }
}

My forge.config.js
module.exports = {
  makers: [
    {
      name: '@electron-forge/maker-zip'
    }
  ]
};

In addition, about Innoextract:

Thanks for your help!

Hi,
I don't know where electron-prebuilt-compile from and nvm too.
I can't help you with processes that aren't covered in this template.
Thanks.


DataVec Python: incorrect address returned for python object + no validation

So I'm trying to run the datavec-python tests on locally on Windows, in IntelliJ.
Happens for both CPU and CUDA.
Windows 10, Python 3.7.4 is also installed and on the Path via Anaconda (though happens without Anaconda installed too).
Perhaps it's a system configuration issue - I'm pretty sure I ran these tests OK on my old machine.
Running on branch sa/new-datatypes-pythonexecutioner - https://github.com/KonduitAI/deeplearning4j/pull/241
As far as I can tell, the python object (numpy array) is valid - note the [[1 1 1]\n [1 1 1]] in the debugger - but PyLong_AsLong(pyAddress) is returning an address of -1 for me.
Additionally this isn't picked up as an invalid address in the DataVec NumpyArray; the first time we see that a problem has occurred is the cudaMemcpyAsync from the invalid address for CUDA (or a JVM crash for CPU). We should have at least basic validation on the Java side.

Failed on [FFFFFFFFFFFFFFFF] -> [0000002A01EA2600], size: [48], direction: [1], dZ: [1]

java.lang.RuntimeException: cudaMemcpyAsync failed; Error code: 1

	at org.nd4j.jita.allocator.pointers.cuda.cudaStream_t.synchronize(cudaStream_t.java:43)
	at org.nd4j.linalg.jcublas.buffer.BaseCudaDataBuffer.<init>(BaseCudaDataBuffer.java:140)
	at org.nd4j.linalg.jcublas.buffer.CudaLongDataBuffer.<init>(CudaLongDataBuffer.java:49)
	at org.nd4j.linalg.jcublas.buffer.factory.CudaDataBufferFactory.create(CudaDataBufferFactory.java:691)
	at org.nd4j.linalg.factory.Nd4j.createBuffer(Nd4j.java:1127)
	at org.datavec.python.NumpyArray.setND4JArray(NumpyArray.java:94)
	at org.datavec.python.NumpyArray.<init>(NumpyArray.java:63)
	at org.datavec.python.NumpyArray.<init>(NumpyArray.java:82)
	at org.datavec.python.PythonObject.toNumpy(PythonObject.java:341)
	at org.datavec.python.PythonType$7.toJava(PythonType.java:208)
	at org.datavec.python.PythonType$7.toJava(PythonType.java:201)
	at org.datavec.python.PythonExecutioner.getVariable(PythonExecutioner.java:197)
	at org.datavec.python.PythonExecutioner.getVariables(PythonExecutioner.java:205)
	at org.datavec.python.PythonExecutioner.exec(PythonExecutioner.java:244)
	at org.datavec.python.Python.exec(Python.java:262)
	at org.datavec.python.TestPythonExecutioner.testNDArrayLong(TestPythonExecutioner.java:220)

This is using ctypes, which isn't the most efficient API out there. If we use NumPy's C API everywhere, I don't think we'd need to use use ctypes anyway?

So I've tried Java 8 (OpenJDK 8) in addition to Java 11 that I was using when I opened the issue.
Same result.
Tried a couple of other random things, all that didn't help:

Not inheriting the system environment variables (PATH etc)
Setting OMP_NUM_THREADS=8
Clearing JavaCPP cache

All result in the same -1 address and hence the JVM crash (CPU) or memcpy issue (CUDA).
So still not sure what's up here...

This may be Windows specific somehow.
The same machine (3990x) on Linux (Clear Linux 32370) runs the tests fine on CPU - no crashes, I'm getting a valid pointer address.

@farizrahman4u Can we try to replace ctypes with numpy everywhere?

@saudet Done: https://github.com/KonduitAI/deeplearning4j/pull/241

@AlexDBlack can you pull sa/new-datatypes-pythonexecutioner and try again?

So, retesting this: I've run a bunch of times each on CPU and CUDA on windows, for branch sa/new-datatypes-pythonexecutioner - no crashes :tada:
I am however seeing a test failure on TestPythonExecutioner.testByteBufferInplace (IIRC Shams flagged this already; now that I can run without crashing I'll look into it)

sa/new-datatypes-pythonexecutioner isn't merged yet, but it's close... I'll close this issue as resolved.


Add the quotation marks in the statistics activity log against items under file shares

Storage Explorer Version: 1.16.0
Build Number: 20201203.3
Branch: main
Platform/OS: Windows 10/ CentOS 7.6.1810 (Core) / MacOS Catalina
Architecture: ia32/x64
Regression From: Not a regression
Steps to Reproduce

Expand one storage account -> File Shares.
Create a file share -> Upload one folder -> Right click the folder -> Click 'Selection Statistics'.
Observe the activity log -> Check whether the statistics item is surrounded by the quotation marks.

Expected Experience
The statistics item is surrounded by the quotation marks.
Actual Experience
The statistics item is not surrounded by the quotation marks.

Additional Context

This issue also reproduces for the action 'Directory Statistics'.
The screenshot of activity log for the action 'Table Statistics' against one table.

Verified on the build main/20211126.7. Fixed.


refactor: Remove TS errors in share-credentials route

Description of change
Fix 8 TS errors when running svelte-check.
Links to any relevant issues
Part of #184.
How the change has been tested
Test if route still renders. npm run check should show 8 errors less.
Definition of Done checklist
Add an x to the boxes that are relevant to your changes.

[x] I have followed the contribution guidelines for this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

PR against the wrong branch.


Added JsCreatePromise to JsRT

+@curtisman , this is the context https://github.com/Microsoft/ChakraCore/issues/2541

I'm a little confused about what resolveFunction and rejectFunction output are and how I would use them.
Let's say JsCreatePromise is equivalent to new Promise((res, rej) => {...}), would this make more sense?
JsCreatePromise(_In_ JsValueRef executor, _Out_ JsValueRef *promise)

executor could be a native function created by JsCreateFunction.

@liminzhu

I'm a little confused about what resolveFunction and rejectFunction output are and how I would use them.

These are essentially the same functions passed to the executor. Another method to fetch them might be like this:
var resolve, reject;
function executor(res, rej) { resolve = res; reject = rej; };
var promise = new Promise(executor);

Goal would be to allow our native code to resolve the promise whenever it wants to. One use might be an API which reads a file from the network and returns a promise resolved with the contents of that file. Native code can construct the promise using JsCreatePromise and hold-on to resolveFunction for that promise. Assume the native-side performs some processing to fetch the result data and when that is done it can call the resolveFunction with that result data which will handle resolving the promise in the engine. I guess the key thing here is that the native code can resolve the promise in an async way without having to create an executor function which would manually suck out the resolve/reject functions.

Makes much sense now. Thanks @boingoing !

LGTM. I just would like to mention that it is not required to have both reject() and response() methods. And if possible, it would be good to accept multiple methods accepted because people can catch multiple types of errors on diff methods. For example:
mypromise.then(...).then(...).catch(...).catch(...) <- that is totally viable.

@galvesribeiro I think you might have it backwards, the resolve and reject parameters are out parameters that will contain pointers to the functions that the native code will call when it has either resolved or rejected the promise.  These are equivalent to the parameters that come into the executor method when creating a promise in JavaScript.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise
For continuations you would just use the JsRT functions to get the property (then, catch, etc.) and then call it normally with JsCallFunction.
In C#:
JavaScriptPropertyId thenId = JavaScriptPropertyId.FromString("then");
JavaScriptValue thenFunction = promise.GetProperty(thenId);

JavaScriptValue resolveFunc = JavaScriptValue.CreateFunction(ResolveCallback);
JavaScriptValue rejectFunc = JavaScriptValue.CreateFunction(RejectCallback);
JavaScriptValue thenPromise = thenFunction.CallFunction(promise, resolveFunc, rejectFunc);

Since it's just a JavaScript function, you can choose to leave out the second parameter when calling it.

Oh! Right, I understood it completely wrong, sorry.  Good work!

Closing this one and retargeting to release/2.0 in #2594.


Error in importing Client

Hi, I have this error:
Traceback (most recent call last):
  File "***.py", line 1, in <module>
    from idex.client import Client
  File "/usr/lib/python2.7/site-packages/idex/client.py", line 10, in <module>
    from ethereum.utils import sha3, ecsign, encode_int32
  File "/usr/lib/python2.7/site-packages/ethereum-2.3.1-py2.7.egg/ethereum/__init__.py", line 11, in <module>
    from . import slogging  # noqa
  File "/usr/lib/python2.7/site-packages/ethereum-2.3.1-py2.7.egg/ethereum/slogging.py", line 6, in <module>
    from ethereum.utils import bcolors, is_numeric
  File "/usr/lib/python2.7/site-packages/ethereum-2.3.1-py2.7.egg/ethereum/utils.py", line 11, in <module>
    import rlp
  File "/usr/lib/python2.7/site-packages/rlp/__init__.py", line 1, in <module>
    from . import sedes  # noqa: F401
  File "/usr/lib/python2.7/site-packages/rlp/sedes/__init__.py", line 1, in <module>
    from . import raw  # noqa: F401
  File "/usr/lib/python2.7/site-packages/rlp/sedes/raw.py", line 9, in <module>
    from rlp.atomic import Atomic
  File "/usr/lib/python2.7/site-packages/rlp/atomic.py", line 4
    class Atomic(metaclass=abc.ABCMeta):
                          ^
SyntaxError: invalid syntax

My source:
from idex.client import Client
client = Client(***, ***)
currencies = client.get_currencies()

log("Currencies: " % currencies)
exit(0)

Hi @sunchaser68 can you try updating to v0.3.0 and see if that helps.