Please let me connect to a different username on Gravatar

I am unable to connect to my account over on Gravatar. As a result, my NuGet profile pic looks like :hankey: :cry:

As I do not know what email this username (Happypig375) is connected to on Gravatar, I tried every email but failed. Is there a way to connect to a Gravatar account which username is different from the one on NuGet as I want to create a new one?

@Happypig375 , we get the image from Gravatar based on the e-mail address. You can see the e-mail address on the account page (https://www.nuget.org/account) under "Email Address" section. Next step will be to login to gravatar.com with this e-mail and make sure the picture is amazing 😄

Wow! That is great news! Hope it is not fake news :wink:
I'll wait a bit and hope my avatar replaces this abomination.

Wow! Thanks! My avatar suddenly looks like 🌈 now 😆


Consider Different Directory to Store Blog Media In

Consider using some sort of folder on the webserver to store images in, maybe a subdirectory of the blog markdown files?

Using a separate R2-based CDN seems to be fine - most issues faced were because requests didn't originate from *.rooray.xyz


people: Add version parameter for medium sized avatar urls.

We need to append the version parameter when constructing the urls for medium-sized images so that the browser updates the image in real time when the user uploads a new avatar.
Discussion: CZO
Fixes #25558.

Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[x] Explains differences from previous plans (e.g., issue description).
[x] Highlights technical choices and bugs encountered.
[x] Calls out remaining decisions and concerns.
[x] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[x] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[x] Visual appearance of the changes.
[x] Responsiveness and internationalization.
[x] Strings and tooltips.
[x] End-to-end functionality of buttons, interactions and flows.
[x] Corner cases, error conditions, and easily imagined bugs.

@timabbott Updated the PR to make 1 the default avatar_version in case we don't have it.

I added some comments and tweaked this to use 0 for the fake avatar version, because that seems safer.
One thing I notice while reading this that we will need to fix as part of making people.js use typescript is that the people.test.js module uses fake user objects that don't have the right format.
I think the long-term fix for that has to be that we create a make_user function in the test library that takes the parameters you want to set, and fills in the rest with default values, so that every user object used in tests has the right type, and then bulk-migrate all of our tests using user objects to use that function. Probably worth a discussion in #frontend on strategy for this.


Define Coding Style

Hi @stackos,
Can you provide coding styles that you use on cpp files and CMake files ?
I use cmang-format with Google style for cpp files and for CMake files I use cmake-format tool on Visual Studio Code that look great.
Thanks

I had commit a clang-format config file for cpp,
but no plans to use cmake for the time being.

Thanks @stackos
For cmake I can make a pull request later


social login icons in RTL

Bug report
Title
social login icons in RTL
Issue Description
social login icons in RTL should be started from right side
Preconditions
1. framework Version- master

Steps to reproduce
1. step1- customer login in RTL

Actual result

fixed https://prnt.sc/tb3lzp


connection lost while in THINKING STATE will get MainAPP stuck

Hi,
I would like to point out   2 behaviors that I noticed using V1.0.2:

After some time in idle I can see that AVS gets disconnect and reconnected all the time. Is that a expected behavior? Isn't AVS suppose to have some kind of keep alive mechanism to avoid that?

If disconnection happen while device is in THINKING stage, device gets stuck into that forever and never moves back to IDLE.
So next keyword interaction will fail because barge-in is not permitted while in Thinking/busy state.

Logs from AVS SDK:
2017-08-25 22:14:26.164 [  b] 0 DialogUXStateAggregator:setState:from=LISTENING,to=THINKING
###########################
Thinking...
###########################
UIManager::onDialogUXStateChanged state = THINKING
InteractionManager::onDialogUXStateChanged to THINKING
######## QcomSoundTrigger::stopRecording() 0
AudioManager::StopRecorder() 0
2017-08-25 22:14:26.174 [ 21] I InProcessAttachmentReader:readFailed:reason=SDS is closed
2017-08-25 22:14:26.877 [ 21] 0 CurlEasyHandleWrapper:reset:responseCode=HTTP_RESPONSE_SUCCESS_NO_CO
NTENT
2017-08-25 22:16:27.922 [ 21] 0 MessageRouter:connectionStatusChanged:reason=SERVER_SIDE_DISCONNECT,
newStatus=PENDING
#############################
Connecting...
#############################
2017-08-25 22:16:28.398 [ 22] 0 MessageRouter:connectionStatusChanged:reason=ACL_CLIENT_REQUEST,newS
tatus=CONNECTED
2017-08-25 22:16:28.400 [  6] I StateSynchronizer:requestingContext:reason=connectionStatusChanged,r
eceivedStatus=CONNECTED
###########################
Thinking...
###########################
2017-08-25 22:16:28.400 [  7] 0 ContextManager:updateStateLocked:action=updatedState,state={"volume"
:30,"muted":0},namespace=Speaker,name=VolumeState
2017-08-25 22:16:28.400 [  7] 0 ContextManager:buildContextSuccessful:context={"context":[{"header"
:{"namespace":"Speaker","name":"VolumeState"},"payload":{"volume":30,"muted":0}},{"header"
:{"namespace":"AudioPlayer","name":"PlaybackState"},"payload":{"token":"amzn1.as-ct.v1.Domain
:Application:DailyBriefing:TTS#ACRI#ssml#ACRI#DailyBriefingPrompt.6c0513cc-ada9-4d2d-b861-1e80dc7
fe6dc:ChannelItem:1:0","offsetInMilliseconds":0,"playerActivity":"STOPPED"}},{"header":{"na
mespace":"SpeechSynthesizer","name":"SpeechState"},"payload":{"token":"amzn1.as-ct.v1.Domain:
Application:DailyBriefing:TTS#ACRI#DailyBriefingPrompt.6c0513cc-ada9-4d2d-b861-1e80dc7fe6dc:Say:
DAILYBRIEFING:DailyBriefingEpilogue","offsetInMilliseconds":0,"playerActivity":"FINISHED"}},{"
header":{"namespace":"Alerts","name":"AlertsState"},"payload":{"allAlerts":[],"activeAlerts"
:[]}},{"header":{"namespace":"SpeechRecognizer","name":"RecognizerState"},"payload":{"wakewo
rd":"ALEXA"}}]}
2017-08-25 22:16:28.401 [  7] 0 EventBuilder:buildJsonEventString:messageId=d7d07817-7aeb-4512-992d-
95fab5a0af8f,namespace=System,name=SynchronizeState
2017-08-25 22:16:33.439 [ 22] I StateSynchronizer:messageSentSuccessfully
2017-08-25 22:16:33.439 [ 22] 0 CurlEasyHandleWrapper:reset:responseCode=HTTP_RESPONSE_SUCCESS_NO_CO
NTENT
###########################
Thinking...
###########################
2017-08-25 22:18:36.025 [ 22] 0 MessageRouter:connectionStatusChanged:reason=SERVER_SIDE_DISCONNECT,
newStatus=PENDING
#############################
Connecting...
#############################
2017-08-25 22:18:36.624 [ 23] 0 MessageRouter:connectionStatusChanged:reason=ACL_CLIENT_REQUEST,newS
tatus=CONNECTED
2017-08-25 22:18:36.625 [  6] I StateSynchronizer:requestingContext:reason=connectionStatusChanged,r
eceivedStatus=CONNECTED
###########################
Thinking...
###########################
2017-08-25 22:18:36.625 [  7] 0 ContextManager:updateStateLocked:action=updatedState,state={"volume"
:30,"muted":0},namespace=Speaker,name=VolumeState
2017-08-25 22:18:36.625 [  7] 0 ContextManager:buildContextSuccessful:context={"context":[{"header"
:{"namespace":"Speaker","name":"VolumeState"},"payload":{"volume":30,"muted":0}},{"header"
:{"namespace":"AudioPlayer","name":"PlaybackState"},"payload":{"token":"amzn1.as-ct.v1.Domain
:Application:DailyBriefing:TTS#ACRI#ssml#ACRI#DailyBriefingPrompt.6c0513cc-ada9-4d2d-b861-1e80dc7
fe6dc:ChannelItem:1:0","offsetInMilliseconds":0,"playerActivity":"STOPPED"}},{"header":{"na
mespace":"SpeechSynthesizer","name":"SpeechState"},"payload":{"token":"amzn1.as-ct.v1.Domain:
Application:DailyBriefing:TTS#ACRI#DailyBriefingPrompt.6c0513cc-ada9-4d2d-b861-1e80dc7fe6dc:Say:
DAILYBRIEFING:DailyBriefingEpilogue","offsetInMilliseconds":0,"playerActivity":"FINISHED"}},{"
header":{"namespace":"Alerts","name":"AlertsState"},"payload":{"allAlerts":[],"activeAlerts"
:[]}},{"header":{"namespace":"SpeechRecognizer","name":"RecognizerState"},"payload":{"wakewo
rd":"ALEXA"}}]}
2017-08-25 22:18:36.625 [  7] 0 EventBuilder:buildJsonEventString:messageId=1d12c824-7f10-4b48-abdd-
3dcca7fb5f7e,namespace=System,name=SynchronizeState
2017-08-25 22:18:40.312 [ 23] I StateSynchronizer:messageSentSuccessfully
2017-08-25 22:18:40.312 [ 23] 0 CurlEasyHandleWrapper:reset:responseCode=HTTP_RESPONSE_SUCCESS_NO_CO
NTENT
###########################
Thinking...
###########################
2017-08-25 22:20:41.668 [ 23] 0 MessageRouter:connectionStatusChanged:reason=SERVER_SIDE_DISCONNECT,
newStatus=PENDING
#############################
Connecting...
#############################
2017-08-25 22:20:42.149 [ 24] 0 MessageRouter:connectionStatusChanged:reason=ACL_CLIENT_REQUEST,newS
tatus=CONNECTED
###########################
Thinking...
###########################
2017-08-25 22:20:42.149 [  6] I StateSynchronizer:requestingContext:reason=connectionStatusChanged,r
eceivedStatus=CONNECTED
2017-08-25 22:20:42.149 [  7] 0 ContextManager:updateStateLocked:action=updatedState,state={"volume"
:30,"muted":0},namespace=Speaker,name=VolumeState
2017-08-25 22:20:42.150 [  7] 0 ContextManager:buildContextSuccessful:context={"context":[{"header"
:{"namespace":"Speaker","name":"VolumeState"},"payload":{"volume":30,"muted":0}},{"header"
:{"namespace":"AudioPlayer","name":"PlaybackState"},"payload":{"token":"amzn1.as-ct.v1.Domain
:Application:DailyBriefing:TTS#ACRI#ssml#ACRI#DailyBriefingPrompt.6c0513cc-ada9-4d2d-b861-1e80dc7
fe6dc:ChannelItem:1:0","offsetInMilliseconds":0,"playerActivity":"STOPPED"}},{"header":{"na
mespace":"SpeechSynthesizer","name":"SpeechState"},"payload":{"token":"amzn1.as-ct.v1.Domain:
Application:DailyBriefing:TTS#ACRI#DailyBriefingPrompt.6c0513cc-ada9-4d2d-b861-1e80dc7fe6dc:Say:
DAILYBRIEFING:DailyBriefingEpilogue","offsetInMilliseconds":0,"playerActivity":"FINISHED"}},{"
header":{"namespace":"Alerts","name":"AlertsState"},"payload":{"allAlerts":[],"activeAlerts"
:[]}},{"header":{"namespace":"SpeechRecognizer","name":"RecognizerState"},"payload":{"wakewo
rd":"ALEXA"}}]}
2017-08-25 22:20:42.150 [  7] 0 EventBuilder:buildJsonEventString:messageId=70a27867-9b70-44a3-b74d-
e22e38e8b5e3,namespace=System,name=SynchronizeState
2017-08-25 22:20:43.601 [ 24] I StateSynchronizer:messageSentSuccessfully
###########################
Thinking...
###########################
[1] Callback event received: 0
[1] Event detection timestamp<PHONE_NUMBER>144
event_handler_thread: Success to acquire qti lock
[1] session params 0x25e978, 5700
capture LAB data
Inside capture_lab_data
sample rate 16000, channels 1, samp sz 2, duration 5700
, total_bytes_to_read 182400
QcomSoundTrigger::onKeyWordDetected - KEYWORD DETECTED = 10
QcomSoundTrigger::onKeyWordDetected() - m_client->notifyOfWakeWord()
lab capture file : /data/SVA/lab_capture_file_1
2017-08-25 22:27:24.605 [  9] E AudioInputProcessor:executeRecognizeFailed:reason=Barge-in is not permitted while busy

Hi @julianobRibeiro,

After some time in idle I can see that AVS gets disconnect and reconnected all the time. Is that a expected behavior? Isn't AVS suppose to have some kind of keep alive mechanism to avoid that?

That is not an expected behavior.  There is an internal ping which should keep the connection alive.  Every 5 minutes you should  see something like the following:
2017-08-28 14:52:56.589 [  5] 0 HTTP2Transport:sendPing:pingStream=0x0 2017-08-28 14:52:56.661 [  5] 0 HTTP2Stream:receivedRequestId:value=x-amzn-requestid\: ... 2017-08-28 14:52:56.661 [  5] 0 HTTP2Transport:handlePingResponse 2017-08-28 14:52:56.661 [  5] 0 CurlEasyHandleWrapper:reset:responseCode=HTTP_RESPONSE_SUCCESS_NO_CONTENT
How long are you waiting in idle before you see the disconnect?  And by "all the time", is it a constant/continuous stream of connect/disconnect, or is it periodic?  And if periodic, how long between disconnects?
Is there any chance another user is trying to connect using the same credentials as you?  AVS will only allow one connection at a time with a given set of credentials, and you can get into a ping-pong of connect/disconnect if another user connects using duplicate device credentials.

If disconnection happen while device is in THINKING stage, device gets stuck into that forever and never moves back to IDLE. So next keyword interaction will fail because barge-in is not permitted while in Thinking/busy state.

It looks like AIP is not detecting message send failures in all cases.  We will address this in a future release.  In the mean time, I expect you will not encounter this state once the repeated connect/disconnect issue is resolved.
Ken

Hi Ken,
Thanks for fast response.
I don`t see HTTP2Transport:sendPing:pingStream messages on my log. maybe it is because ping is configured for each 5 min in IDLE and my disconnection/connection is happening every 2 MIN.
By "all the time" I mean that every 2 min I am seeing same log messages that I gave you showing:
MessageRouter:connectionStatusChanged:reason=SERVER_SIDE_DISCONNECT,newStatus=PENDING
and then
2017-08-28 16:01:23.942 [  e] 0 MessageRouter:connectionStatusChanged:reason=ACL_CLIENT_REQUEST,newStatus=CONNECTED 2017-08-28 16:01:23.942 [  6] I StateSynchronizer:requestingContext:reason=connectionStatusChanged,receivedStatus=CONNECTED
Any idea what could be causing this disconnection every 2 min?
Thank you,
Juliano

Hi Juliano,
I'd like to take a look at what we're logging on the server side.  Could you send me your x-amzn-requestid?  It is printed if you have DEBUG0 or higher logging enabled, and should show up in the Connecting phase, before the Sample App banner prints.
Thanks,
Ken

HI ken,
Is that the print that youa re talking about?
https://github.com/alexa/avs-device-sdk/blob/master/ACL/src/Transport/HTTP2Stream.cpp#L277
Even after removing #ifdef DEBUG, I can not see it while executing my APP.
Regards,
Juliano.

Hi Juliano,
Once small point of correction here.  A server side disconnect (followed by a reconnect) is expected about once an hour.  It is used to allow for load balancing and updating of servers.
Another data point that may be helpful:  How long in to your session are you seeing the disconnect?  Is that timing consistent?
Also, just removing the #ifdef DEBUG is not enough.  That #ifdef is there to avoid the overhead of the preparing for that log line in non-DEBUG builds.  To enable debug logging you need to define ACSDK_DEBUG_LOG_ENABLED, which is done by default for DEBUG builds.
Hope this helps,
-SWH

Hi,
Here is the x-amzn-requestid from test that I have just executed.
Every time that I initiate my sample APP it gets disconnected after 2 minutes if there is no activity happening on device.

2017-08-30 17:34:38.899 [  5] 0 HTTP2Stream:receivedRequestId:value=x-amzn-requestid: 069b7ffffe236
048-00000d21-00027e94-565b7ac40e2b144c-316a45bf-3

Thank you in advanced,
Juliano Ribeiro

Hi SWH,
Were you able to do any trace using x-amzn-requestid that I have provided.
I am still trying to figure out why I am getting SERVER_SIDE_DISCONNECT every 2 minutes while in IDLE.
Regards
Juliano

HI @julianobRibeiro,
For your second point, see what I commented here for a temporary solution.
Thanks,
Sanjay

Thank you for the suggesting I will try that.
Is there a way to understand why SERVER is sending DISCONNECT messages every 2min?
That can impact on our device performance or even fail some use case interaction depending on timing when they are issued.
Regards,
Juliano

Hi @julianobRibeiro,
Apologies if you already responded to this question but I didn't see a response. Is there any chance another user is trying to connect using the same credentials as you? We will investigate with the server side team to see what the issue may be.
Thanks,
Sanjay

Hi Sanjay,
There is only one user at a time.
When we try 2 users using same credential at same time we are not even able to connect.
On the issue that I am reporting we are receiving DISCONNECT from SERVER every 2 min, besides that client works fine.
REgards
Juliano

Hi @julianobRibeiro,
Just wanted to follow-up regarding the issue where the device gets stuck in THINKING during a disconnect. Are you still having this issue in the latest version?
Let me know and if you are still experiencing difficulties I can work on it.
Fede


Add Python Black code formatter CI integration

Add CI command for the Python Black code formatter and fail if code is not formatted correctly.
Also includes a run on the code base and makes the overall style more consistent.
Fixes #599

Black itself is working fine now in the CI pipeline, however there seems to be a problem regarding the API integration tests:
  ● /v1/analyses › Submit › issues

    expected 200 "OK", got 400 "Bad Request"

      at Test.Object.<anonymous>.Test._assertStatus (node_modules/supertest/lib/test.js:268:12)
      at Test.Object.<anonymous>.Test._assertFunction (node_modules/supertest/lib/test.js:283:11)
      at Test.Object.<anonymous>.Test.assert (node_modules/supertest/lib/test.js:173:18)
      at assert (node_modules/supertest/lib/test.js:131:12)
      at node_modules/supertest/lib/test.js:128:5
      at Test.Object.<anonymous>.Request.callback (node_modules/superagent/lib/node/index.js:718:3)
      at parser (node_modules/superagent/lib/node/index.js:906:18)
      at IncomingMessage.res.on (node_modules/superagent/lib/node/parsers/json.js:19:7)

Unhandled rejection Error: Invalid login: 535 Authentication was rate limited. Check again in 299 seconds
    at SMTPConnection._formatError (/home/mythril-api/node_modules/nodemailer/lib/smtp-connection/index.js:606:19)
    at SMTPConnection._actionAUTHComplete (/home/mythril-api/node_modules/nodemailer/lib/smtp-connection/index.js:1335:34)
    at SMTPConnection._responseActions.push.str (/home/mythril-api/node_modules/nodemailer/lib/smtp-connection/index.js:366:26)
    at SMTPConnection._processResponse (/home/mythril-api/node_modules/nodemailer/lib/smtp-connection/index.js:762:20)
    at SMTPConnection._onData (/home/mythril-api/node_modules/nodemailer/lib/smtp-connection/index.js:558:14)
    at TLSSocket._socket.on.chunk (/home/mythril-api/node_modules/nodemailer/lib/smtp-connection/index.js:709:51)
    at emitOne (events.js:116:13)
    at TLSSocket.emit (events.js:211:7)
    at addChunk (_stream_readable.js:263:12)
    at readableAddChunk (_stream_readable.js:250:11)
    at TLSSocket.Readable.push (_stream_readable.js:208:10)
    at TLSWrap.onread (net.js:601:20)

This happens multiple times so that succeeding tests fail due to auth rate limiting.

Merged the latest changes in master and merged. Ready to review :+1:


How to exit rancher graceful?

What kind of request is this (question/bug/enhancement/feature request):
Question
Steps to reproduce (least amount of steps as possible):
Result:
Other details that may be helpful:
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): rancher-server 2.0
Installation option (single install/HA): single install

Cluster information

Cluster type (Hosted/Infrastructure Provider/Custom/Imported):  Imported
Machine type (cloud/VM/metal) and specifications (CPU/memory): VM
Kubernetes version (use kubectl version):

(paste the output here)

Docker version (use docker version):

(paste the output here)

How to exit rancher graceful? Exiting the server seems doesn't clearn the environment, it left pods/contianers running there.
Is there a way to clear it all after the rancher server exit?

closed as dup


Update to express 4

As this is one of the most popular articles on your website (and extremely helpful) you should seriously consider updating the code base for the passport-local strategy.

It looks like this was done, though there is a branch still using express 3.x. Also, this is outdated, in order to make it work, you will basically need to convert 4.x to 3.x by installing all the depreciated modules. If you want security, write your own middleware. If you want something fast, this could still be good but it's not so straightforward.


Multiple-yaxis-scales, 3 series with 2 scales demo was broken by 3.47.0

New Pull Request
Fixes relating to yaxis.seriesName array feature added at revision 3.47.0:

The forward and reverse yaxis to series maps had errors.
Anything that indexed into various global or context arrays (yaxis array, series working arrays, baseline or ratio arrays, etc), needed to be mapped through the new forward or reverse yaxis to series maps.

Fix historical issue with goals and annotations being drawn when the axis is hidden or outside the grid area when zoomed or panned, or where rect area annotation should be clipped.
Miscellaneous:

Remove yaxis.min: 0 from the bar axes in sample as not required.
Fix several calls to CoreUtils.getLogVal(b,d,seriesIndex) that were missing the 'b' (base) argument.
Fix what may have been an historical issue with log charts and return 0 for negative or zero values passed to getLogVal(), not just values that equal 0. Negative values appear to cause the chart to misalign with the axis.
wrong point annotation x position in sparkline chart.

Fixes #2757
Fixes #3073
Fixes #3421
Fixes #3553
Fixes #4081
Fixes #4241 (maybe)
Fixes #4323
Type of change

[X] Bug fix (non-breaking change which fixes an issue)

Checklist:

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[X] My changes generate no new warnings
[X] New and existing unit tests pass locally with my changes

This replaces the previous PR #4329 that was incorrectly based.

@junedchhipa Can you give me a clue re the baseLineY value that is used in translating into SVG grid coordinate space. I haven't been able to determine how that value is derived but it throws out the alignment of logarithmic plots and yet is ok for linear plots. At the moment I'm just substituting zero for log plots.


Grammar fix

gh

Thanks for your interest in contributing to GraphicsFuzz. I assume this might have been an accidental PR, as the branch being merged only contains a commit that I (paulthomson) authored. As noted above, you will need to sign the CLA before any contributions can be accepted. Also note that we develop in the dev branch. Thus, please start your development by checking out the dev branch and also make sure the "base branch" when doing a PR is manually changed to dev. Thanks. I will close this PR for now, but feel free to ask further questions, open another PR, or correct this PR.


Add DevOps Mutualization Project Operations Manager Role

Description
This pull request add the Project Operations Manager role to the DevOps Mutualization repository.
Added and Updated Files
The files that were added / updated within the DevOps Mutualization repository are ...

README.md
docs/open-roles/README.md
docs/open-roles/project-operations-manager.md

Commits

add project operations manager role 256d54e
Merge branch 'master' into add-project-role 5a4ae94
update open roles README 25808ea
update SIG README 6847da7
update SIG README 6b2ca95
update SIG README a9b83b1
update SIG README 644bc84
update SIG README 8403dd3
update SIG README 410979b
update job titles 7fb731e
update job titles 9d1483c

Merged following @ashukla13's approval below ...

looks good


Update version to 1.0.20-3 on master

Update version to 1.0.20-3

🚀 [Deployed](https://github.com/Expensify/Expensify.cash
/actions/runs/765130740) 🚀 to
staging on Mon Apr 19 2021 at 22:41:56 GMT+0000 (Coordinated Universal Time)

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Multiple target squares not working correctly

When targetting two tokens within range, it gives the warning to say that no squares are available to move and attack from when this is clearly not the case.
Have tried to get it to work both manually and with the shift override.

Another example

Sorry I haven't noticed this issue before as issues do not transfer to forks. This module was taken over by me June 16 2023 and is now at this repository. I hope this issue has been resolved since, but if it still persists, please open it again at this page.


Test

User story

As a {}
I want / need {}
So that {}

Acceptance criteria

Criterion 1
Criterion 2
...

Definition of done (DoD)

Added only after week 5
The same for all features
Here goes the project specific part

DoD general criteria

Feature has been fully implemented
Feature has been merged into the mainline
All acceptance criteria were met
Product owner approved features
All tests are passing
Developers agreed to release

Not a feature issue, hence marking closed


Consider replacing the function or lambda with a reference to an exported function

Hello, I'm facing some problems to build my own library.
I don't know if here is the right place to post this error because I'm almost sure that is my mistake but I'm not sure.
I created a library with a provider and after I published this library at npm, when I'm importing my library inside a new angular-cli project, i'm facing this library.
I'm importing this library with .forRoot() static method and the library.
ERROR in Error encountered resolving symbol values statically. Calling function 'SampleModule', function calls are not supported. Consider replacing the function or lambda with a reference to an exported function, resolving symbol AppModule in /home/luishmcmoreno/Projetos/test3/src/app/app.module.ts, resolving symbol AppModule in /home/luishmcmoreno/Projetos/test3/src/app/app.module.ts
Can someone help me?

Are you linking the library with npm link if you are i was having the same issue and I fixed this removing the node_modules folders from the library folder

@luishmcmoreno — Is it possible to share your code? Thanks!

Hey, thank you @izifortune but I'm not linking.
@jvandemo sure!
https://gist.github.com/luishmcmoreno/b968290035770c63b6243c35ba3cb6a8

I'm getting the same error, did you solve this?

Not yet.
When I run ng serve, the problem happens but if a save some file again (live reload) the problem is gone. But the problem is ng build --prod don't work.

are you using that forRoot function? if yes, can you place your code here? I solved the problem here, but I don't know if it is the same case

Yes I'm using .forRoot() method and I didn't resolved yet.
The code is here: https://gist.github.com/luishmcmoreno/b968290035770c63b6243c35ba3cb6a8

it's not the same case, I was passing a parameter to forRoot function, and it was throwing the error... did you try to verify if everything is fine on the 'dist'?

I think everything is ok but I'm not sure.
The dist is here: https://www.npmjs.com/package/@pluritech/auth-service

I think I opened a very similar issue #128

Hi there I'm having the same problem ,
I'm using npm link only for the dist folder , later I'm consuming the library in an Ionic 2 project and everything works fine until i do the --prod in the project , any clue ?  help will be really appreciated

I fixed the issue changing the custom provider , with a export function ! , is a ngc problem and nothing related to this generator ! , thanks !

@ljcp — Thank you for sharing your solution, much appreciated. 👍
@luishmcmoreno — Can you please verify if that also works for you?
Thanks!

I didn't understand the solution. But you can label this as solved. I chose another specific way to solve my problem.
Thank you all.

@luishmcmoreno — Would you be able to share your solution (if possible)?
It would be great if others can benefit from your solution as well. Thanks!

Isn't a solution. I didn't using your library in this case.

@luishmcmoreno — Ah, ok, thanks for the update! 👍


fix: use export = for typescript module definition

I actually wasn't sure if this would work but this allows the following in typescript.
import * as chalk from 'chalk' // (new versions of ts don't let you use chalk as a function 🙌 )
import chalkDefault from 'chalk'
import chalkTypescriptRequire = require('chalk')// (legacy typescript style)

😑
This will ~fix~ allow intellisense in commonjs projects in vscode (#246)
This may break some people's typescript type usages.

Oh and also re-enables import { red } from 'chalk' in ts I know there was some discussion around intent with that...
side note: I wish that the way 99% of commonjs modules were written had a parallel in ES modules 😩. so much import * as someFunction from 'somewhere' we'll be "transpiling" forever...

@sindresorhus, We keep getting user reports running into this issue. is there any thing we can help with to get this issue addressed?

This may break some people's typescript type usages.

Chalk is a very popular package. I'm not sure I want to risk it.

Oh and also re-enables import { red } from 'chalk' in ts I know there was some discussion around intent with that...

This has already been discussed in great detail over at #215.

What's the verdict here? I'm very out of the loop on typescript stuff so I'm not much help.
If this is an opinionated change that will break existing users then I echo @sindresorhus's concerns.

Closing. I'm not interested in risking breaking people's usage for something that is really a bug in VSCode. I also don't want to enable import { red } from 'chalk'.


Movie list totally empty

Hi !
I start using https://metamanager.wiki/en/latest/  recently, and since, my movie list is totally empty in pledo (with last version 1.3.0 also)). I don't touch TV show library and this one is still OK.
I can't find interesting log, hint, trace to know what is the problem.
I have this line that say there is 0 movie, its wrong.
pledo  |       Syncing movies: Found 0 movies in library Movies from server REDACTED
Is it possible that now my movie "entry" are too huge, contains too much metadata?
Do you have any idea for debugging this?
Thanks

Strange, this is kind of similiar to #13.
Your assumption could be possible, the sync requests movies and tv shows in batches, with a batch size of 200 movies with one request. Maybe this exceeds plex media servers limits on movies with too much metadata. I'll think about a different, more dynamic approach.

Metadata sync is now in smaller batches and kind of concurrently, so it shouldn't even be slower than before. Do you mind do try the new version 1.3.1 again?

It works ! Thanks 💯 🥇


Add operations to manage zones

[x] main
[x] 4-3-stable

This would provide functionality like:

iadmin mkzone
iadmin modzone
iadmin modzonecollacl
iadmin rmzone
iadmin lz

This functionality is needed by the zone management tool.

Equivalent functionality for iadmin modzonecollacl is partially implemented, but disabled until further notice.
See #290 for additional details.


Ehcache java.lang.reflect.InaccessibleObjectException Unable to make field private final byte[] java.lang.String.value accessible: module java.base does not "opens java.lang" to unnamed module @2cd76f31

( ehcache version org.ehcache:ehcache:3.9.9 java 11)
WARN 8418 --- [nio-8080-exec-2] org.ehcache.sizeof.ObjectGraphWalker     : The JVM is preventing Ehcache from accessing the subgraph beneath 'private final byte[] java.lang.String.value' - cache sizes may be underestimated as a result
CacheConfiguration<String,CachedValueWrapper> cacheConfiguration= CacheConfigurationBuilder
.newCacheConfigurationBuilder(String.class, CachedValueWrapper.class,
newResourcePoolsBuilder().heap(100, MemoryUnit.MB))
.withExpiry(ExpiryPolicyBuilder.timeToLiveExpiration(Duration.parse("PT30S")))
.build();
I am new to caching and was wondering if this is due to insufficient space in the heap memory?

This is due to the more recent JVMs enforcing strong encapsulation and preventing reflective access to JVM internals. You'll need to either open up reflective access using (potentially numerous) --add-opens command line switches, or move to using a count based heap resource (e.g. heap(10000)) so that Ehcache doesn't need to size the cached elements.


Fix typo in type cast

It's probably a typo, since type cast is done via .type(img1.dtype)

hi @BloodAxe thanks for reaching us out. However, this is not a typo and it's made in purpose since it can handle both cases similar to the C++ api. According to an internal discussion in the pytorch slack, .to is more idiomatic In addition, it's being mentioned that .type() is legacy code from the old version. ight be deprecated at some point.
Said that, I close this. I encourage you to keep in touch with us for other potential contributions.


Dynamic expressions not working with new URL(.., import.meta.url)

Bug report

What is the current behavior?
When a URL is created with a dynamic expression, webpack does not transform it:
const language = detectVisitorLanguage();
const url = new URL(`./locale/${language}.json`, import.meta.url);
const response = await fetch(url.href); // fails at runtime as url has not been transformed

If the current behavior is a bug, please provide the steps to reproduce.

I can pull together a small example repo if helpful, please LMK.
What is the expected behavior?

I expected it to behave like imports with dynamic expressions - i.e. I expected webpack to have included all potential /locale/* files in the build, such that the correct file could be loaded at runtime.
const language = detectVisitorLanguage();
const response = await import(`./locale/${language}.json`);

As a workaround, the code can be written like so:
const language = detectVisitorLanguage();
let url;
if (language === "fr-FR") {
  url = new URL("./locale/fr-FR.json", import.meta.url);
} else {
  url = new URL("./locale/en-US.json", import.meta.url);
}
const response = await fetch(url.href);

Other relevant information:
webpack version:  5.36.2
Node.js version: 14
Operating System: macOS 11.2.2
Additional tools: N/A

Other workaround - you can generate file like (or create plugin for it):
export default const fr_FR = new URL('./locale/fr-FR.json', import.meta.url);
export default const en_US = new URL('./locale/en-US.json', import.meta.url);

Bug, your example make sense

But it should be not hard, we need to add URLContextDependency (like ImportContextDependency https://github.com/webpack/webpack/blob/master/lib/dependencies/ImportContextDependency.js) and put it here https://github.com/webpack/webpack/blob/master/lib/dependencies/URLPlugin.js

Still make sense I think

Is it actually completed? I still see the described behaviour in Webpack 5.91.0 while Vite works perfectly fine with dynamic URLs.


add debug image variants

Fixes https://github.com/authzed/zed/issues/141
I've basically ported changes in https://github.com/authzed/spicedb/pull/750

I've also updated Dockerfile to use distroless.dev/static. The original PR did not update that file.
I've noticed the build section of .goreleaser.docker.yaml deviates from the one in SpiceDB. I've left untouched, but wonder if we should align them

It deviates a bit because it needs to run on macOS to generate the binary because zed needs cgo for to link against the keychain library.


-t/--stream, -l/--lattice in #whichExtractionMethod

As of #141.
CLI seems to still only use -r/--spreadsheet and -n/--no-spreadsheet. Change that.

https://github.com/tabulapdf/tabula-java/commit/6e1b540e8efdaeb6585ad7e13905cf58421111a1

Oh good catch, thanks @criztovyl!


Make configuration files more accessible

Some of our dependency injection configuration files look a bit more difficult than they need to.
Take for instance https://github.com/solid/community-server/blob/0644f8d24517b88018f85941d5b74b94c3a443f3/config/presets/ldp/permissions-extractor.json#L7-L14.
It could be a bit easier if one-line objects were just one line:
{
  "CompositeAsyncHandler:_handlers": [
    { "@type": "MethodPermissionsExtractor" },
    { "@type": "SparqlPatchPermissionsExtractor" }
  ]
}

Also, we might want to add some aliases to get rid of the @ keywords, so we can just write id and type.
Could we avoid @graph?
Plus, let's introduce a prefix for urn:solid-server:default: (perhaps css or default?) to remove verbosity.
Also, perhaps we should lint our JSON files to enforce some of these?
Tasks

[ ] make simple objects one-liners
[ ] add aliases for @ keywords
[ ] add prefix for urn:solid-server:default:
[ ] check if we can get rid of @graph
[ ] set up linting

I would love to work on this one. I have a weird passion for well-written configs and have a vested interest in making sure it's easy to configure using Kubernetes. Can you provide more context or documentation on how the current config process works, and what files I should check out?

Sorry, I meant specifically in the codebase. For example, what is a RegexRouterRule or a MemoryResourceStore? Where can I learn what all these different dependency injection values mean?
It seems like this dependency injection framework is super powerful, but it makes it difficult for me to have any sense of what these config values do or even how to configure and launch the app. Have you given any thought to how to simplify things and lower the barrier to entry for new developers? It might be as simple as just writing documentation for all the different config values, or wrapping the configs in a layer of syntactic sugar that can be circumvented by advanced users.

Looks like this ticket covers the documentation part, so I guess that doesn't exist yet. :D
https://github.com/solid/community-server/issues/295

I guess that doesn't exist yet. :D

It does—#295 discusses documenting the configuration files themselves, not the components they use.

I meant specifically in the codebase. For example, what is a RegexRouterRule or a MemoryResourceStore? Where can I learn what all these different dependency injection values mean?

They are components in the codebase:

https://github.com/solid/community-server/blob/v0.3.0/src/storage/routing/RegexRouterRule.ts#L9-L17
https://github.com/solid/community-server/blob/v0.3.0/src/storage/accessors/InMemoryDataAccessor.ts#L24

We can generate TSdoc for them (#434), which will make them easier to find and use.

Variables can also be useful here: https://github.com/solid/community-server/pull/296#issuecomment-748675469 / https://github.com/LinkedSoftwareDependencies/Components.js/issues/23

Awesome, this is amazing context thanks!


Link to Bluetooth Assigned Numbers is outdated

New link is https://www.bluetooth.com/specifications/assigned-numbers/baseband

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: aa52ee84-78eb-85eb-55d0-9a2163f95799
Version Independent ID: d81d2cfd-2f50-ccbd-ec6f-96d73897b8ed
Content: Bluetooth Registry Entries
Content Source: windows-driver-docs-pr/bluetooth/bluetooth-registry-entries.md
Product: windows-hardware
GitHub Login: @nabazan-microsoft
Microsoft Alias: windowsdriverdev

Thanks, @sahikon
Fixed in source. Should be live soon.


Add basic function, tests, and sample file

Added basic function based on read-clipboard ps1 structure. Added some pester tests.
To run tests I switched into public and ran invoke-pester .\Read-OleDbData.Tests.ps1
Seems to be failing the build tests, but I didn't want to add it into the main project files before review.

Cool, yes, just checking. Proceed at the pace you are comfortable with. Thanks for the PR!

These pass locally, but I need to add an exclusion for when the ace driver isn't there. I don't think it would be appropriate to add ace in as a dependency. More like something you should get a warning on if it's not there and you run it, and something the tests should check for and then just skip testing if it's not there.

@dfinke all tests passing.

I went ahead and created a very basic module for this for the time being.

Btw, I asked Jaykul, and he created a #importexcel channel on his PowerShell discord server. It's free, check it out. https://discord.com/invite/powershell

Awesome! I joined, thank you.

I like the way you used $ACEnotWorking to skip tests. Haha, I was writing a question here about running on Linux, then looked at the function and you handle it, terrific! I'll make more time to review when I can.
Thanks for the PR, looking forward to trying it all out.

Thinking out loud.
It'd be cool to use Read-OleDbData out of the box like this. Thoughts?
Read-OleDbData ./test.xlsx 'select ROUND(F1) as [A1] from [sheet3$A1:A1]'

Thinking out loud.
It'd be cool to use Read-OleDbData out of the box like this. Thoughts?

Read-OleDbData ./test.xlsx 'select ROUND(F1) as [A1] from [sheet3$A1:A1]'

I think so. Probably want to change the name to something like Read-XlsxUsingOleDb. Could include another command that way that just packages it up. The raw command could just be an internal really. Could also setup the params to work like that.
The unfortunate thing with no connection string is some of the ways of how oledb will treat excel depends on the connection string. So if you want headers (I'm not using them here) have to update the connection string, if you want to treat things as native types or not, more changes, etc.
I would say that as a broader use module, all of those are probably useful to someone. Whatever you think would be good for an mvp is good with me. It isn't too tough (I think) to support more use cases, especially the ones we mentioned.

added a helper method with the simpler syntax. definitely makes the calls shorter! i figured adding a helper method for this would still give people the ability to do basically anything with the underlying command, but default to the more typical use case the way you noted.
PS C:\gh\ImportExcel> Read-XlsxUsingOleDb .\__tests__\ImportExcelTests\TestData1.xlsx "select * from [sheet1`$b3:b3]"

F1
--
 4

PS C:\gh\ImportExcel> cd .\__tests__\ImportExcelTests\
PS C:\gh\ImportExcel\__tests__\ImportExcelTests> Read-XlsxUsingOleDb TestData1.xlsx "select * from [sheet1`$b3:b3]"

F1
--
 4

PS C:\gh\ImportExcel\__tests__\ImportExcelTests> Read-XlsxUsingOleDb TestData1.xlsx "select 1"

Expr1000
--------
       1

"helper" function work great. Especially when layering is in play. Read-XlsxUsingOleDb, I think the function names needs a nudge. Not sure what, mayge along the lines of Read-Excel... works with Import, and Export. I see why *UsingOleDb is needed. Wish Query was an accepted verb, Query-Excel feels like it is a great fit. A possible fallback is Invoke-ExcelQuery.

Yep, it's very literal at the moment. No reason that it has to be, especially since it's a helper. Can just redirect people to the underlying function in the help if they need it. And I don't know that it's really a requirement that folks using the method actually know that it's using oledb. It's easy enough to add to help for anyone who's curious.
I use 'read' for this instead of 'invoke' just based on the stuff in the ms docs:

Use the Read verb to open a resource and extract information contained within.

Use the Invoke verb to perform synchronous operations, such as running a command and waiting for it to end.

That being said, Invoke is consistent with Invoke-Sqlcmd, so would definitely seem natural. The only issue there may be that someone may want to 'update' something. While technically that would work here, I probably wouldn't use a dataadapter and fill to a datatable with that and would maybe add some flags to use appropriate object types.
Can always note the use cases in the examples and if anyone asks, can add more functionality as needed. I personally only use this for reads as the writing has been a bit janky for my taste. In places where I write, I just use the regular ImportExcel cmdlets and avoid the oledb, or for some older jobs I still use epplus until I move them over to the module.
I am also totally fine with using a 'non-accepted' verb like Query-Excel, especially since it's a 'helper' function anyway. Or could also alias it. I have a utility function I use in a lot of my jobs that has a function name of Add-PrefixForLogging so it's nice and compliant with the right verbs, but I alias it as 'L' for logging and that's what I actually use. =P

I think from the accepted verb option, Invoke-ExcelQuery is probably the most intuitive.
I added it as it's own function just to isolate it, but maybe it should just be an optional -Query param on Import-Excel? From a 'helper' standpoint, that seems like it may be more intuitive. I'm not sure how much demand there is for something like this, but if you think it would be useful, that may be a better place for it.

Yeah, that was my thought as well. I figured my use was kind of an edge case and if a bunch of people asked about it, maybe later it would make sense. I think it's easy enough to leave the underlying helper stuff and then just later implement a helper switch if that made sense.
Oledb itself is not terribly descriptive/clear on some errors especially with the query formation. I suppose exhaustive examples could take care of that, but probably no way to cover every single instance and no real way to be sure people
Would read them.
I am game for whatever you think is best.

Thanks. On adding -Query I like the idea, need to think it thru from a support standpoint. Could get lots of question on many levels, driver installs etc.

i think the best approach on this, if it was added, would be to likely link to a readme of some kind or maybe a wiki page with a lot of details and examples. I mean you could put them in help, but maybe just some basic ones and then have a lot more on a page on the repo somewhere. that way someone could, presumably, submit a bug or feature request or ask for help on there. or maybe even refer them to discord or something.
I have never tried to setup a module where you can trigger the get-help -online stuff, but i'm thinking like that maybe.
current basic module I have installed locally looks like this:
╰─ get-help read-oledbdata                                                                                ─╯

NAME
    Read-OleDbData

SYNTAX
    Read-OleDbData [[-ConnectionString] <string>] [[-SqlStatement] <string>]

ALIASES
    None

REMARKS
    None

but I think I'd want the remarks section to look more like a standard command, or at least have a link or something to follow like this:
REMARKS
    Get-Help cannot find the Help files for this cmdlet on this computer. It is displaying only partial help.
        -- To download and install Help files for the module that includes this cmdlet, use Update-Help.
        -- To view the Help topic for this cmdlet online, type: "Get-Help Copy-Item -Online" or
           go to https://go.microsoft.com/fwlink/?LinkID=2096990.

Some Things
Good stuff!

I don't see Invoke-ExcelQuery in the psd1 in FunctionsToExport
Looks like on the rename, the call to Read-OleDbData got renamed to Invoke-ExcelQuery
Did these tests run for you locally, it should fail
What happens if folks don't have don't have the .NET installed with that driver?

Do the tests get skipped when the CI pipeline runs on Windows?
I made those first two change. I'd prefer you update the PR.
Then I was able to do the below. Very cool.
Import-Module ./importexcel.psd1 -force

$xlFile = 'D:\mygit\ImportExcel\__tests__\Read-OleDbDataTests\Read-OleDbData.xlsx'

Get-ExcelSheetInfo $xlFile | ForEach-Object {
    Invoke-ExcelQuery $xlFile "select $($_.Index)"
}

we must have been typing at the same time as i submitted my other reply just as this one popped up. let me take care of these.

I don't see Invoke-ExcelQuery in the psd1 in FunctionsToExport

Fixed! I also added the underlying function to the exported functions.

Looks like on the rename, the call to Read-OleDbData got renamed to Invoke-ExcelQuery

Fixed!

Did these tests run for you locally, it should fail

Yes, and I thought that was kind of odd. I originally thought it was because I was running in a ps where I had run the ps1 originally, but after killing it, it was still fine. I poked around and found this in the ImportExcel.psm1:
foreach ($directory in @('Private', 'Public', 'Charting', 'InferData', 'Pivot')) {
    Get-ChildItem -Path "$PSScriptRoot\$directory\*.ps1" | ForEach-Object { . $_.FullName }
}

And since I was force loading the module up top, it seemed to reload these each time. I just snagged that top matter when I was poking around in the other Public methods looking for one to use as a sort of a template.

What happens if folks don't have don't have the .NET installed with that driver?

They'll just get the errors thrown out. If no oledb classes, they'll get some lib error from dotnet about a bad class. If no provider, they'll get a bad provider error I believe. I don't currently have any machines that are missing the dotnet libs or the provider so I am not 100% sure what the errors look like. They'll also get a variety of oledb errors if there is poorly crafted sql. I currently just let them come out as I prefer to see the errors. We can try and catch some of those and provide some guidance, but I just let them bubble out so the error is easier to search for. If I was going to catch them, it might be better to just give a link to a wiki or readme with frequent issues that could be updated without having to update the method itself.

Do the tests get skipped when the CI pipeline runs on Windows?

yes because there won't be any providers installed like that. it just does a basic test to open the file and if that doesn't work, it will set the flag re:ace working. you have to have the access 2010 runtime installed to use it, although i see there is now a 2016 version too:

2010
2016

It's been a few years, but I think I've only installed the 2010 one.
I kind of viewed this use as an edge case. If you think more people would use it then maybe it's worth doing some work to get the provider stuff in as part of module installation. Or maybe, at least, upon the first run of this command some checks and an auto install. I'm not 100% sure what that will look like as I only use this functionality on servers I control, so I just have the prereqs documented as part of the installation procedure.
whatever you think makes the most sense works for me.

Thanks for iterating on this, much appreciated. I think out of the box, the tests should fail on windows to catch those sort of errors. Otherwise if people try to use it, they'll just stall and move on. I'm not sure of the best way to install the driver during the testing.

My pleasure. There is a module called get-ace that uses ace to read data. I haven't used it or played with it but let me take a look at it. Maybe there is some precedent there. If not I'll see about cooking something up.

i wasn't able to find the original 'ace' module, but i did find someone who had a slightly modified copy of it here:

https://github.com/DBremen/PowerShellScripts/tree/master/modules/ACE

and it looks like their solution was just to add a note in the help that calls out that you must have the drivers installed in order for the module to work.
i did find a post on SO where someone had a function to do conversion using ace:
https://stackoverflow.com/questions/58244399/using-powershell-with-microsoft-ace-oledb-12-0-to-convert-between-csv-xml-xls-xl
basically, they did a check up front like this, again with some simple help:
        $oledb = (New-Object system.data.oledb.oledbenumerator).GetElements() | Sort -Property SOURCES_NAME | where {$_.SOURCES_NAME -like "Microsoft.ACE.OLEDB*"} | Select -Last 1
        If ($oledb -eq $null) {
            Write-Output "MICROSOFT.ACE.OLEDB does not seem to exist on this computer."
            Write-Output "Please see https://www.microsoft.com/en-us/download/details.aspx?id=54920"
            Write-Output "This can also happen if you have not installed the 32 or 64 bit version "
            Write-Output "and you are running this script in the architecture that is missing it."
            Write-Output "Solution is to do silent install on missing driver"
            break
        } else {
            $Provider=$Oledb.SOURCES_NAME
            Write-Verbose "Provider $Provider found on this computer"
        }

I'm not sure about the auto-install as it might need the ability to run as admin. probably could find and attach the dlls to load if needed, but I'm not sure if there are any distribution restrictions on them.
on this:

I think out of the box, the tests should fail on windows to catch those sort of errors.

do you not think the skip is enough for local? if someone runs the tests locally, they'll see that they are skipped. maybe easier to just add a write-host or something that states the reason for skipping in those tests so it's more apparent. then adding the same check (probably something more like the one listed above) in the cmd with some clarifying text. I'm just not sure about the maintainability of something that includes this runtime, in terms of including an install vs a comment with direction to the install link from MS.

Good stuff, I'm playing with the Invoke-ExcelQuery. A few things:

For the local skip, leave as is
I like the up front check. Easier for the user to know before they get errors
Auto installs need to be avoided. You need to prompt a user before installing etc
I'm ok with an auto install in the tests. I control that with the CI pipeline. Knowing if a function name was misspelled etc is good to catch in the tests before publishing the module
Could you add a InvokeExcelQuery directory in the Examples folder and add an example
Do you blog? This would be a good blog post. Something simple, not elaborate.

Experimenting with this
Something along these lines would be good for the example and a post. The xl file is attached.
$q = 'select * from [sheet1$A:A]'
$q = 'select * from [sheet1$]'
$q = 'select * from [sheet1$A2:E11]'
$q = 'select F2,F5 from [sheet1$A2:E11]'
$q = 'select * from [sheet1$A2:E11] where F2 = "Grocery"'
$q = 'select F2 as [Category], F5 as [Discount], F5*2 as [DiscountPlus] from [sheet1$A2:E11]'

Invoke-ExcelQuery .\testOleDb.xlsx -Query $q | Format-Table

testOleDb.xlsx

Good stuff, I'm playing with the Invoke-ExcelQuery. A few things:

For the local skip, leave as is

👍

I like the up front check. Easier for the user to know before they get errors

👍 Agreed. Will add something

Auto installs need to be avoided. You need to prompt a user before installing etc

👍

I'm ok with an auto install in the tests. I control that with the CI pipeline. Knowing if a function name was misspelled etc is good to catch in the tests before publishing the module

👍 Sounds good. I'll take a stab at it.

Could you add a InvokeExcelQuery directory in the Examples folder and add an example

👍 Yep. I'll pull all of the other examples and tests in there and work under this name.

Do you blog? This would be a good blog post. Something simple, not elaborate.

👍 Yep. Occasionally, anyway @ ashbrook.io =P Definitely!

Experimenting with this
Something along these lines would be good for the example and a post. The xl file is attached.
$q = 'select * from [sheet1$A:A]'
$q = 'select * from [sheet1$]'
$q = 'select * from [sheet1$A2:E11]'
$q = 'select F2,F5 from [sheet1$A2:E11]'
$q = 'select * from [sheet1$A2:E11] where F2 = "Grocery"'
$q = 'select F2 as [Category], F5 as [Discount], F5*2 as [DiscountPlus] from [sheet1$A2:E11]'

Invoke-ExcelQuery .\testOleDb.xlsx -Query $q | Format-Table

testOleDb.xlsx

Looks good to me. Will update appropriately.

Good stuff, I'm playing with the Invoke-ExcelQuery. A few things:

For the local skip, leave as is

i'm not sure how to adjust the tests to check for local only. i didn't see any tests that seemed to indicate that, but i didn't go through every one.

I like the up front check. Easier for the user to know before they get errors

I left the readoledb one as is just as a check for linux/macos because you could technically use it for any oledb source. i could make this an internal method and then only support the Invoke-ExcelQuery command as public, but i figured i would just leave it as public and then just in case someone wanted to do something funny they could still use it.
i updated the invoke cmd to check for the provider and give a warning message before returning with the ms url for the ace install
i also updated the invoke command to have the same check for the system.data.oledb call which should throw an error if no oledb is installed or the system.data space isn't there.

Auto installs need to be avoided. You need to prompt a user before installing etc

👍

I'm ok with an auto install in the tests. I control that with the CI pipeline. Knowing if a function name was misspelled etc is good to catch in the tests before publishing the module

i couldn't find a direct link to download the exe from ms. i can download the exe and add it to the repo and see about installing it from there. i can take a stab at the install for ci. i haven't ever included an exe auto install in a pipeline, so you may see a million build messages as I mess around with things. if you have an idea on how to do it, let me know otherwise I'll play around with it. I found this thread which seems to have a similar situation and does have some commands someone suggested: https://stackoverflow.com/questions/65081902/how-to-install-a-odbc-driver-on-the-azure-integration-runtime-machine. But the commands are basically similar to what you have in the installpowershell script which is what i was going to use as a template, although I'm not exactly clear on how that is invoked =P
for now, i switched to the newer test check so there is a warning in the tests during discovery that will state why the tests are being skipped

Could you add a InvokeExcelQuery directory in the Examples folder and add an example

👍 Added

Do you blog? This would be a good blog post. Something simple, not elaborate.

👍 Once it's done. =P

Thanks for all of that. I'd prefer errors when the tests run, but it's no worth the effort. Yeah, for the installpowershell script, I vaguely remember taking that out at one point 🤷♂
Having the warning in the function is good enough. If more is needed down the road, it'll get addressed.
Again, thanks for all of this. I want to let it sit for a day or so, I'll come back to it and exercise it. See what pops out ;-)

sounds good. could always do some negative tests. i haven't tried bubbling up or doing some global/script level var sets in tests, but i think that might do the trick. right now i'm setting that skip var. i think some testing would guarantee it for sure, but i think the pester tests are in an ordered list so could maybe set the $skip for later tests based on the earlier ones. i guess that doesn't buy much more than is happening here, but then it's done in the testing loop instead of in discovery.
there's got to be a way to add ace into the pipeline to test it. i didn't want to go nuts fiddling with it unless you thought it was worth it. i don't love that the testing basically gets skipped in CI, just not sure exactly how to load ace into that pipeline.

Agreed. I have not tried installing anything other than ps modules in the test run. At this point, it is an MVP. If you'd like, create another PR for experimentation, the AZDO is free, because this is a public project.

I'll leave that for after the blog posting. Once the PR is integrated, I need to update the importexcel in house and update some jobs to swap over to this function instead of the custom read-oledb function i have now. =)

I'm seeing about getting this working in a sandbox instance. Seems to be good to go, but the installer is pretty large so seeing about sticking it in a repo on here just to host it, rather than including it. Not sure how well that will work, but would be nice to have the ability to not skip the tests. I can't see a way to get the downloader from ms directly in a script, and the exe didn't seem to really want to install silently. I was able to just extract the msi from the exe file and it seems to work properly.
I'm basically using the same method in your InstallPowershell.ps1 script. See below for output from win10 sandbox:
PS C:\Users\WDAGUtilityAccount> (New-Object system.data.oledb.oledbenumerator).GetElements().SOURCES_NAME
SQLOLEDB
MSDataShape
ADsDSOObject
Windows Search Data Source
MSDASQL
MSDASQL Enumerator
SQLOLEDB Enumerator
MSDAOSP
PS C:\Users\WDAGUtilityAccount> $MsiPath = (gci C:\Users\WDAGUtilityAccount\Desktop\Downloads\2016_AccessDatabaseEngine_X64\aceredist.msi).fullname
PS C:\Users\WDAGUtilityAccount> Start-Process 'msiexec.exe' -Wait -ArgumentList "/i $MsiPath /quiet"
PS C:\Users\WDAGUtilityAccount> (New-Object system.data.oledb.oledbenumerator).GetElements().SOURCES_NAME
SQLOLEDB
MSDataShape
Microsoft.ACE.OLEDB.12.0
Microsoft.ACE.OLEDB.16.0
ADsDSOObject
Windows Search Data Source
MSDASQL
MSDASQL Enumerator
SQLOLEDB Enumerator
MSDAOSP
``

I got it up on github using lfs, but after looking at the way that works, we'll run out of bandwidth pretty fast. Only 1GB for free and the install is almost 100mb. We could host it somewhere else, but I think the right way to do things is probably to have a custom image setup with the ace drivers installed.
I have always used a process similar to what we're already doing where I just use a base image and modify it. I have built some custom docker containers in the past for something like this, but I think that may all be a bit much for an MVP.
I'm going to just leave it as is for now unless you think otherwise. =)

This is great! Thanks for add it . I tweaked the example to output the $query. I'm going to merge and publish it. I have your github and twitter links, let me know of any other links to add, blog, linkedin etc.
Thanks again. Plus, great work. A pleasure to collaborate :-)

Thank you, the feeling is mutual!
linkedin: https://linkedin.com/in/royashbrook
blog: https://ashbrook.io
=)


fix: subjectId type

Fix for subjectId type
Closes #45

@simoneb No clue 😞 , It looks everything fine, @actions.core and @actions.github weren't updated, they have the same version, I didn't find the docs version but I believe they updated the section about addComment  to use ID! instead of String! almost sure that was String! otherwise it should never have worked.

Weird. I realized now that there is a changelog but it doesn't contain any mention of this change.
https://docs.github.com/en/graphql/overview/changelog


Matrix representation of circuit

What is the expected enhancement?
It would be great to be able to inspect the unitary matrix of a circuit that can be represented as such. Currently one can go through Operator to get a representation but it's not obvious to a user how to do this.
A circuit.to_matrix method or similar would make this much clearer and easily accessible. However, not all circuits have a matrix representation and thus this would not be universal.

I don't think this should be a circuit method:

It hides the fact you are doing an expensive simulation to determine the matrix
Many circuits won't have a matrix representation
For many circuits the the matrix will typically be too large for available memory (around >14 or so qubits)

It would be better to better document to the user how to use the Operator class  or unitary simulator to do this

I think that matrix representations can be very useful during the development of an algorithm, for small sets of qubits, and also during the first exercises of a student that is learning how quantum gates alter the states. At least, for me it was :)
Documenting it would make it less accesible for new developers, I think it is something that should be available for use in the first tutorials about quantum gates. So, implementing it and discouraging its use under the circumstances you describe, would help advanced users understand when it should no longer be used.
Just my opinion though :)

This is already available in the Operator class. It is one extra line of code:
from qiskit.quantum_info import Operator
Operator(circuit)

but it does proper error handling, ability to further analyze that operator, compose two circuits via composing their operators, etc.
So I don't think this shortcut should exist; we should encourage the use of Qiskit classes. This probably points to the Operator class not having been exposed to users as well as the QuantumCircuit class. This should be fixed via tutorials, etc.

Yes, it's available but both me and a collaborator did not find it on our own so something is lacking. I'm also not sure how constructing an Operator would be better than constructing an Operator inside a to_matrix method, though the former is a bit more specific with regards to what's happening.

Also, if the feature is so hidden that requires a tutorial for users to know about it and how to use it, maybe a modification is indeed required :) Plus, the feature is almost developed and ready for review, I only have to find time to fix a circular dependency. https://github.com/Qiskit/qiskit-terra/pull/3110

Is this issue still active? FWIW I agree with @chriseclectic's answer above, I think it would be better to not hide a potentially expensive and not scalable operation in a seemingly innocent circuit method 🙂
Also, getting the matrix is now in the basic circuit tutorial: https://qiskit.org/documentation/tutorials/circuits/01_circuit_basics.html#Unitary-representation-of-a-circuit.

closing this issues as wont fix. Please, reopen if there are new arguments for a QuantumCircuit.to_matrix method.

The operator approach generates the matrix but it also permutes the qubit order. Afaik there is no way of setting the qubit order to match the convention. I have illustrated the problem I have encountered here, the solution is specific to attaining the controlled gate but there is no general approach.
There needs to be an inherent way of generating the matrix of a QuantumCircuit without relying on Operator as an intermediate.

Qiskit's endianness convention is well-defined and common in programming; Operator doesn't permute the qubits, it just respects the little-endian convention that Qiskit uses.  If you want to reverse the bit order in a circuit such that calling Operator(circuit) produces a matrix in apparent big-endian convention, you can use QuantumCircuit.reverse_bits() first, or you can use Operator.reverse_qargs() after the operator is created.  (Alternatively, you can also just use a little-endian convention in your matrix work if you're using Qiskit, or you'll always be fighting our convention.)
Generating a matrix from a circuit somewhat efficiently (if it's even possible) is a very non-trivial operation, and given that we've supported Operator doing that, I stand by the arguments that were made two and four years ago about not adding another way of doing it.


Geotable plot

This is a PR to bring into pysal/dev work to implement easy plotting for geotables. Not to be merged yet but rather to create a place to centralized the discussion around design details and so on. My idea is that, once we're happy with it we merge and, ideally, we have it in by the time @ljwolf's GSoC project gets merged as well so we can have a big splash on simplifying workflows with spatial data.
So far, this implements basic machinery to plot geotables with both a matplotlib and a bokeh backend and contains a color.py module with palette support that depends on bewer2mpl. There's a notebook (geotable_plot) explaining the new functionality.

Coverage decreased (-2.04%) to 83.239% when pulling dc0ae2a1060d7bf6a277878a2a09e27640447a38 on darribas:geotable_plot into 336cfd00c99280b761264962716db3ff84e2e34f on pysal:dev.

brewer2mpl mentions that it's superseded by paletteable, should we switch?
@sjsrey @ljwolf

Ahh, good call!
I think it's smart to switch to palettable since it involves much more than just brewer, but am not sure of how much would need to be refactored. Probably just colors.py? And, maybe changes in the ipywidgets colorpicker?
I could probably author the necessary changes over the weekend, but I can review any stuff this week.

yes, I think all of the refactoring should happen in color.py. I really like the modular approach you took in there. At the end of the day, the API should be: pass in an indexed series of classes and a palette, receive an equally indexed series of colors. I also really like the option to select the style of the colors (hex, rgb, etc.). bokeh only seems to take hex, so the default currently set is also great.

I like the colors.py stuff too, but that's :100: @sjsrey
I agree on the API. Not sure how the style of colors matters in the backend, other than what each package expects naturally. Maybe that'll be more apparent as I dig in.

re. the color style (or unit), I think matplotlib takes essentially anything, but bokeh only hex. I like leaving it open because we might end up using more backends and it seems to be pretty straightforward.

For choroBrewer I moved a lot of the interface building code out of the notebook and into its own module. My thinking is this gives us another use case for the plotting of the geotables so it might help flesh out the design of viz.
I am +1 on moving to palette from brewer2mpl if someone wants to do the porting. I think all the changes would be isolated in color.py

Coverage decreased (-0.1%) to 85.184% when pulling 8a3837f71fde38bca13d07811d13c63192487666 on darribas:geotable_plot into 336cfd00c99280b761264962716db3ff84e2e34f on pysal:dev.

Sweet. It looks pretty straightforward. palettable.colorbrewer looks like it supports almost exactly the same stuff as brewer2mpl.

Coverage decreased (-2.04%) to 83.244% when pulling c1d1b185f9746e43cfe66c2f321b0d314a6a0ac8 on darribas:geotable_plot into 336cfd00c99280b761264962716db3ff84e2e34f on pysal:dev.

Latest commit switches to palettable in color.py. It provides the same range of palette support but it changes the API so the user only needs to pass the name of the palette, not the type (which is redundant in color brewer). Additionally, there's an option to pass a palettable palette class instead that allows to easily tap into all the other options in the library (e.g. Wes Anderson palettes).
I think this is pretty good as it gives direct and easy access to all colorbrewer, plus additional ones without much effort. My only concern is we are tied to palettable, which is not terrible but does not allow us to use, for instance, viridis or other seaborn/matplotlib palettes.
Next thing we should think of implementing is the legend. @sjsrey do you think it'd be easy to adopt a similar approach to that you used in geopandas?

Coverage decreased (-2.04%) to 83.239% when pulling 296595c999b97a8fad29a2dd62a14e690d3ab574 on darribas:geotable_plot into 336cfd00c99280b761264962716db3ff84e2e34f on pysal:dev.

This change breaks choroBrewer.ipynb
/home/serge/Dropbox/p/pysal/src/pysal/pysal/contrib/viz/mapping.py in geoplot(db, col, palette, classi, backend, color, facecolor, edgecolor, alpha, linewidth, marker, marker_size, ax, hover, p, tips, figsize, **kwargs)
    611     '''
    612     if col:
--> 613         lbl,c = value_classifier(db[col], scheme=classi, **kwargs)
    614         if type(palette) is not str:
    615             palette = get_color_map(palette=palette, k=c.k)

/home/serge/Dropbox/p/pysal/src/pysal/pysal/contrib/viz/mapping.py in value_classifier(y, scheme, **kwargs)
     58     classification   : Map_Classifier instance
     59     """
---> 60     c = classifier[scheme.lower()](y, **kwargs)
     61     return (pd.Series(c.yb, index=y.index), c)
     62

TypeError: __init__() got an unexpected keyword argument 'dtype'

Good catch @sjsrey. I think the last commit should fix it!

@darribas +1 on the fix

Coverage decreased (-2.04%) to 83.239% when pulling bf93acbff704702798e4441b908907fa1405777f on darribas:geotable_plot into 336cfd00c99280b761264962716db3ff84e2e34f on pysal:dev.

@darribas on the legend question, yes that makes sense as a next step. Have to give some thought to legend design/options on the one hand, and then implementation on the other.
Also, as a next step, for choroBrewer I want to explore adding options for specification of a classifier so the user can view the impacts of that choice along side different cmaps.

Coverage decreased (-0.1%) to 85.145% when pulling 20530cf26552ed6c5684f3388ee78cc022d52b5a on darribas:geotable_plot into 336cfd00c99280b761264962716db3ff84e2e34f on pysal:dev.

@sjsrey Sounds good. I wonder if we could merge this PR into dev and split the task into two issues: one for legends and another one for choroBrewer, since they are relatively distant projects. Since the plotting relates to the GSoC project, what do you think about merging this now @ljwolf?

I don't think we should merge as we likely should do some more testing before something goes into dev (which becomes the next rolling release basically). We haven't checked other supports (lines, points) yet and once we put things in dev, the api is basically set so if we decide later we need to change it because of good reasons, we have breakage.
I can see starting a possible different branch for choroBrewer and the legend effort, -  but I don't see it as too distant from the plotting/GSoC project as it is a consumer of the latter. I think it is useful to have a "consumer" perspective help inform the design of the lower level viz effort.

Right. Sounds good. Lines and points should work just as well, the plotting is hooked up and because it's modular, it's separated from the coloring so no additional work is required. But that doesn't mean it can't have bugs of course. Probably a good idea to test a bit more.

That's right, I've been very liberal with dependencies. I think a check when the function is called would be the cleanest way to go about it.
Related to this, it's got me thinking: I know we don't test on contrib but, given the different parts geotable plotting depends on (e.g. bokeh) are moving very quickly, would it make sense to set up tests that we can at least run locally for sanity? If so, how'd we test plotting functionality?

Coverage decreased (-4.8%) to 80.437% when pulling 025fc39bdd232d0a78c30234e44f2b3aa3316cdf on darribas:geotable_plot into 336cfd00c99280b761264962716db3ff84e2e34f on pysal:dev.

Coverage decreased (-2.8%) to 82.515% when pulling fea380e6e4f0aea327b81bb5b32630c541849f26 on darribas:geotable_plot into 336cfd00c99280b761264962716db3ff84e2e34f on pysal:dev.

Coverage decreased (-2.8%) to 82.515% when pulling fea380e6e4f0aea327b81bb5b32630c541849f26 on darribas:geotable_plot into 336cfd00c99280b761264962716db3ff84e2e34f on pysal:dev.

@sjsrey Just fixed the notebook for geotable_plot. I think it was a mismatch that didn't have to do with any of the actual plotting. It should run fully as of now. We should clean it up a bit and make it comprehensive (in that it covers all the existing functionality) and make a requirement to jupyter nbconvert --execute geotable_plot.ipynb and have that run smoothly before pushing any change.

Coverage decreased (-2.8%) to 82.515% when pulling 0dd820a21ec86f7740b5954a492491f4c9c869f2 on darribas:geotable_plot into 336cfd00c99280b761264962716db3ff84e2e34f on pysal:dev.

Coverage decreased (-2.8%) to 82.515% when pulling 4b1efa1693dcd23ae593445b096fc9a3553bb0b2 on darribas:geotable_plot into 336cfd00c99280b761264962716db3ff84e2e34f on pysal:dev.

Coverage decreased (-0.09%) to 82.423% when pulling 1d737803b64b4a397eb532e641b0685366e75c3a on darribas:geotable_plot into b18c2cff22abb00a4f69fedc66ed10b013b63bee on pysal:dev.

We need to update the requirements information before merging this, as I think the number of dependencies for this module has increased.

Just pushed an update to the dependency list for the contrib.viz module. Since we have a few things in there, and not everything is needed for everything, I've attempted to break it into components. Maybe @sjsrey and @ljwolf can have a look (since they were the contributors to some of those parts) to see if this works. Alternatively, we can list everything needed to access all the functionality on viz.

Coverage increased (+0.6%) to 83.073% when pulling 62583877d5ccea0976109195d6999f5a7eb10869 on darribas:geotable_plot into b18c2cff22abb00a4f69fedc66ed10b013b63bee on pysal:dev.

The print statements should still use the warnings.warn command, instead. But, that's not a showstopper.
Am I right on the combinations of packages that we need to check work?

[ ] matplotlib, palettable, pandas
[ ] bokeh, pandas
[ ] none

Thanks @ljwolf , :s/print/warn done!
bokeh also requires pandas as the coloring is done in the same way as with matplotlib on geoplot. What do you mean by none?

like, if the user only has scipy/numpy, it'll fail on import rather than at runtime?

Right, but right now pandas and matplotlib will fail at import.

Coverage increased (+0.6%) to 83.073% when pulling 85b4db70cab7635e7b6deeb3f89a5e871c1578d9 on darribas:geotable_plot into b18c2cff22abb00a4f69fedc66ed10b013b63bee on pysal:dev.

Cool, :shipit:

Looking into the issue with the brexit map under bokeh that @sjsrey noted, I think the bokeh backend will be limited for complex geometries.
To reprise, the issue was that a call to plot the brexit data in bokeh finishes computing, but crashes before display due to something somewhere in the Javascript rendering.
#this will crash your notebook
maps.geoplot(db, 'Pct_Leave', k=7, alpha=0.5, classi='fisher_jenks', backend='bk')

This is surprising, since there are only ~380 constituencies. In contrast, geotable_plot can show NAT in bokeh just fine :frowning_face:
NAT = ps.pdio.read_files(ps.examples.get_path('NAT.shp'))
maps.geoplot(NAT, 'HR90', alpha=.5, k=7, classi='fisher_jenks', backend='bk')

This led me to look at the vertex counts.
brexit_verts = db.geometry.apply(lambda x: len(x.vertices))
brexit_verts.unique()

The british constituency shapefile has some pretty amazingly detailed polygons, with the Highlands constituency using ~360k points alone. The worst in NAT is under 200 vertices. Thanks mostly to the intense detail with which the Scottish part of the dataset is stored, it takes around 3.5 million points to plot all constituencies, whereas NAT is only 80k points to plot. Extrapolating a bit, it looks like BokehJS or the notebook Javascript chokes at around 2 million points in the bokeh plot.

To help the user, we could warn if the plot "size" is too large and the rendering likely to fail, but I'm not sure if the point limit you can find in the brexit data will hold for other datasets.

Thanks very much for this @ljwolf. I think this is not necessarily a bug in our side but just a characteristic or rendering on the browser these days at least. I wonder also if this is bokeh only or any renderer that directly attempts to put that many points on a browser will choke.
I'd be inclined not to set any message because: a) it's probably hard to come up with a number as it might depend on browser, machine specs, etc. and b) it's likely to change over time as bokeh and web technologies improve. What do you think?

I agree, looks like a JS issue. I'm not sure when matplotlib will fail, but I'm sure if you try to plot some massive number of points, it'll start to seize up in the image display.
You're also right that it's not worth it to find that limit & display when we think it might fail. I'm game to consider the concerns we discussed from the dev call answered and this to get merged!

Yes, I think at some point any visualization approach is bound to fail if it naively tries to plot all of the data. At that point, approaches like datashader are probably more useful.
+1 on merging as far as I'm concerned, provided executing the guide notebook returns no error.

Just for future reference, this notebook could be an example on how to plot lines with a lot of points and not get bokeh to choke:
https://anaconda.org/jbednar/trajectory/notebook

Thanks @ljwolf for digging into this and finding the reason why that particular shapefile was raising the problem. I agree this is not a bug on our side and I'm +1 on merging once the issues you mentioned get  resolved. This will be really nice to have in the library!

Coverage increased (+0.6%) to 83.074% when pulling da440550366f6e0b3b294eae1c90b069d7b0451a on darribas:geotable_plot into b18c2cff22abb00a4f69fedc66ed10b013b63bee on pysal:dev.

Coverage increased (+0.6%) to 83.074% when pulling da440550366f6e0b3b294eae1c90b069d7b0451a on darribas:geotable_plot into b18c2cff22abb00a4f69fedc66ed10b013b63bee on pysal:dev.

Coverage increased (+0.6%) to 83.074% when pulling da440550366f6e0b3b294eae1c90b069d7b0451a on darribas:geotable_plot into b18c2cff22abb00a4f69fedc66ed10b013b63bee on pysal:dev.

Coverage increased (+0.6%) to 83.074% when pulling da440550366f6e0b3b294eae1c90b069d7b0451a on darribas:geotable_plot into b18c2cff22abb00a4f69fedc66ed10b013b63bee on pysal:dev.

Coverage increased (+0.6%) to 83.074% when pulling da440550366f6e0b3b294eae1c90b069d7b0451a on darribas:geotable_plot into b18c2cff22abb00a4f69fedc66ed10b013b63bee on pysal:dev.

I think the only thing left here is decide whether we want to bring in @ljwolf's work to make qualitative palettes automatically set k if they have a fixed number of colors. There was a PR for that but can't find it now...

That PR is here.
I'll take a look at bringing that up to speed.

Revised PR on that is here.

OK, that's merged! If tests pass (which they should), are we good to merge? I think so.

Coverage increased (+0.6%) to 83.074% when pulling 1d19ea356d3c4a61c40e83a91ac828db79e626da on darribas:geotable_plot into b18c2cff22abb00a4f69fedc66ed10b013b63bee on pysal:dev.

Coverage increased (+0.6%) to 83.074% when pulling 1d19ea356d3c4a61c40e83a91ac828db79e626da on darribas:geotable_plot into b18c2cff22abb00a4f69fedc66ed10b013b63bee on pysal:dev.

Coverage increased (+0.6%) to 83.074% when pulling 1d19ea356d3c4a61c40e83a91ac828db79e626da on darribas:geotable_plot into b18c2cff22abb00a4f69fedc66ed10b013b63bee on pysal:dev.

Coverage increased (+0.6%) to 83.074% when pulling 1d19ea356d3c4a61c40e83a91ac828db79e626da on darribas:geotable_plot into b18c2cff22abb00a4f69fedc66ed10b013b63bee on pysal:dev.

Coverage increased (+0.6%) to 83.074% when pulling 1d19ea356d3c4a61c40e83a91ac828db79e626da on darribas:geotable_plot into b18c2cff22abb00a4f69fedc66ed10b013b63bee on pysal:dev.


spanner: backup and restore examples and tests

Towards: https://github.com/GoogleCloudPlatform/ruby-docs-samples/issues/537

@skuruppu Tests are failing due to GOOGLE_CLOUD_SPANNER_TEST_INSTANCE and GOOGLE_CLOUD_SPANNER_PROJECT are not exported in kokoro test environment.  If you have permission to add it, please add it.

@skuruppu Tests are failing due to GOOGLE_CLOUD_SPANNER_TEST_INSTANCE and GOOGLE_CLOUD_SPANNER_PROJECT are not exported in kokoro test environment. If you have permission to add it, please add it.

I believe it's already exported in the Kokoro script. I don't think we made changes recently to the configs.

@jiren have you been able to fix the tests?

@jiren have you been able to fix the tests?

Looking into it.

Interesting, now it seems to be just failing with the same error for Ruby 2.4 but passing on 2.5.

@jiren any luck with the tests? We would like to get this submitted within a day or two. Let me know if there's anyone I need to follow up with if you're stuck.

@jiren any luck with the tests? We would like to get this submitted within a day or two. Let me know if there's anyone I need to follow up with if you're stuck.

No luck, I have tried using ruby 2.4 on local machine and it is working. I am going to try on Docker image of kokoro.

@skuruppu Now all test cases are passed. Moved before / after callback code to spec helper.

Kokoro running only latest ruby version as per script.
https://github.com/GoogleCloudPlatform/ruby-docs-samples/blob/651d4dc52a1fbdc01b403188142307ea9157020d/spec/kokoro-run-all.sh#L5-L8
Ruby 2.4
RUN_ALL_TESTS set to "" so it failed because env variable will not set for spanner.
https://github.com/GoogleCloudPlatform/ruby-docs-samples/blob/651d4dc52a1fbdc01b403188142307ea9157020d/spec/kokoro-run-all.sh#L69-L72
+ [[ -n '' ]]

Log output
https://source.cloud.google.com/results/invocations/4f5d473e-e069-4478-87a0-63b0725b07bc/targets;collapsed=/cloud-devrel%2Fruby%2Fruby-docs-samples%2Fpresubmit%2Fruby-2.4/log
Ruby 2.5
RUN_ALL_TESTS set to 1 so it will set the env variables for spanner.
Log output
https://source.cloud.google.com/results/invocations/924fe7c1-fa2a-44e3-9f45-52a43d53ebe8/targets/cloud-devrel%2Fruby%2Fruby-docs-samples%2Fpresubmit%2Fruby-2.5/log
+ [[ -n 1 ]]
+ export GOOGLE_CLOUD_SPANNER_TEST_INSTANCE=ruby-test-instance
+ GOOGLE_CLOUD_SPANNER_TEST_INSTANCE=ruby-test-instance
+ export GOOGLE_CLOUD_SPANNER_PROJECT=cloud-samples-ruby-test-0
+ GOOGLE_CLOUD_SPANNER_PROJECT=cloud-samples-ruby-test-0

@TheRoyalTnetennba, @jiren did some digging and it seems like the RUN_ALL_TESTS env var is not set properly for Ruby 2.4. If you're able to help us fix this, we would really appreciate it.

Actually, I just checked the build output. No tests are now run for Ruby 2.4. Is this the intention?

Actually, I just checked the build output. No tests are now run for Ruby 2.4. Is this the intention?
Are you able to create a PR for a new simple change to see if tests for 2.4 run on master? Because if it does, then we need to fix the issue in this PR that causes the tests to not run. Otherwise, we need to fix the issue on master.

I think it is intended to do so (only run ITs on the latest ruby version 2.5):
https://github.com/GoogleCloudPlatform/ruby-docs-samples/blob/3c9046b6d823338daa18badc6ea51a019c3aab3b/spec/kokoro-run-all.sh#L4-L6

Oh I see, thanks @hengfengli, I didn't know.


When you make a mistake for a new TestingGround, the Hashes becomes a HashWithIndifferentAccess

Duplicate of #30. :wink:

Duplicate of #30. :wink:

:blush:


Please invite me to the community

Name
Vlad Chernykov
Additional Context
Please invite me to Github Organization

Invitation sent to join the GitHub Organization. Welcome to Infraform 🎉Please check your mail to accept the invitation sent by GitHub! 🤖About usInfraform is a remarkable open-source solution tailored for serverless platforms, aimed at empowering creators to craft their infrastructure solutions. 💻 We're thrilled to welcome you into our vibrant community! ✨ We eagerly anticipate your valuable contributions! 🤓 Don't forget to explore our community on Github and connect with us on LinkedIn 💜 Feel free to spread the word about our community to like-minded individuals interested in development, DevOps, technology, and collaborative contributions. 👉💜👈


Segmentation fault in ovs-vswitchd during dump-bpf-map

When I try to dump BPF map, ovs-vswitchd gets SIGSEGV. As a result ovs-vswitchd is terminated.
To trigger this issue, I need to do following steps:

Create test environment - two pairs of interfaces with type veth:

$ ip netns add test0
$ ip link add port0 type veth peer name peer0
$ ip link set port0 netns test0
$ ip link set peer0 up

$ ip netns exec test0 sh << EOF
ip addr add "<IP_ADDRESS>/24" dev port0
ip link set port0 up
EOF

$ ip netns add test1
$ ip link add port1 type veth peer name peer1
$ ip link set port1 netns test1
$ ip link set peer1 up

$ ip netns exec test1 sh << EOF
ip addr add "<IP_ADDRESS>/24" dev port1
ip link set port1 up
EOF

Create OVS switch with ports of type afxdp:

$ sudo ovs-vsctl -- add-br br0 -- set Bridge br0 datapath_type=netdev
$ sudo ovs-vsctl add-port br0 peer0 -- set interface peer0 type="afxdp"
$ sudo ovs-vsctl add-port br0 peer1 -- set interface peer1 type="afxdp"

Load uBPF program into switch and insert data into (any) map:

$ sudo ovs-ofctl load-bpf-prog br0 1 tunneling.o
$ sudo ovs-ofctl show-bpf-prog br0
NXT_BPF_SHOW_PROG_REPLY (xid=0x4):
The number of BPF programs already loaded: 1
id 1:   loaded_at=2020-03-07 T23:00:43
	map_ids 0,1
$ sudo ovs-ofctl update-bpf-map br0 1 1 key 14 0 16 172 value 0 0 0 0

I used tunneling.p4 as a uBPF program.

Dump content of the map:

$ sudo ovs-ofctl dump-bpf-map br0 1 1 hex
ovs-ofctl: talking to unix:/usr/local/var/run/openvswitch/br0.mgmt (End of file)

As the result in the terminal, where ovs-vswitchd has been run, I got following logs:
2020-03-07T21:59:02Z|00051|memory|INFO|handlers:1 ports:3 revalidators:1 rules:5
Segmentation fault (core dumped)

When I repeat this scenario with gdb, I got following logs for ovs-vswitchd:
2020-03-08T14:33:57Z|00049|memory|INFO|handlers:1 ports:3 revalidators:1 rules:5

Thread 1 "ovs-vswitchd" received signal SIGSEGV, Segmentation fault.
0x0000555555932d90 in ofpmsg_update_length (buf=buf@entry=0x8ec40)
    at lib/ofp-msgs.c:884
884	    struct ofp_header *oh = ofpbuf_at_assert(buf, 0, sizeof *oh);
(gdb) bt
#0  0x0000555555932d90 in ofpmsg_update_length (buf=buf@entry=0x8ec40)
    at lib/ofp-msgs.c:884
#1  0x0000555555787783 in ofconn_send (ofconn=0x613000001000,
    counter=0x60600008e100, msg=0x8ec40) at ofproto/connmgr.c:1489
#2  ofconn_send_reply (ofconn=ofconn@entry=0x613000001000, msg=0x8ec40)
    at ofproto/connmgr.c:1026
#3  0x00005555556e08f4 in handle_dump_map (oh=<optimized out>,
    ofconn=0x613000001000) at ofproto/ofproto.c:6525
#4  handle_single_part_openflow (type=<optimized out>, oh=<optimized out>,
    ofconn=0x613000001000) at ofproto/ofproto.c:8892
#5  handle_openflow (ofconn=ofconn@entry=0x613000001000,
    msgs=msgs@entry=0x7fffffffd280) at ofproto/ofproto.c:9077
#6  0x0000555555787f33 in ofconn_run (
    handle_openflow=0x5555556d9b80 <handle_openflow>, ofconn=0x613000001000)
    at ofproto/connmgr.c:1321
#7  connmgr_run (mgr=<optimized out>,
    handle_openflow=handle_openflow@entry=0x5555556d9b80 <handle_openflow>)
    at ofproto/connmgr.c:356
#8  0x00005555556c5f47 in ofproto_run (p=<optimized out>)
    at ofproto/ofproto.c:1894
#9  0x000055555568ae16 in bridge_run__ () at vswitchd/bridge.c:3246
#10 0x000055555569de55 in bridge_run () at vswitchd/bridge.c:3305
#11 0x000055555567d934 in main (argc=<optimized out>, argv=<optimized out>)
    at vswitchd/ovs-vswitchd.c:127

I am running into the same issue, have you found a solution?


Sending a lot of requests

Hi. It's strange, but every 2-4 seconds a request is sent to infura. All I did was connect the DAPPProvider. Don't know what could be the reason? Or is it ok?

same issue, and most of them mean to call the 'eth_blockNumber' method

I noticed that this doesnt happen in version 0.10.1, I didnt check other versions.

This is normal behaviour. useDapp always keeps queried data up-to-date, so it periodically checks if the a new block was mined.
If the interval is too short you can manually set it on the provider like this (replace the provider URL and chainId with your own):
const avaxProvider = new JsonRpcProvider("https://api.avax-test.network/ext/bc/C/rpc")
avaxProvider.pollingInterval = 10_000;

const config: Config = {
  readOnlyChainId: AvalancheTestnet.chainId,
  readOnlyUrls: {
    [AvalancheTestnet.chainId]: avaxProvider
  },
  pollingInterval: 10000,
  notifications: {
    expirationPeriod: 1000,
    checkInterval: 1000
  }
}

If the interval is too short you can manually set it on the provider like this (replace the provider URL and chainId with your own)

Thank you, it`s help

This is normal behaviour. useDapp always keeps queried data up-to-date, so it periodically checks if the a new block was mined.
If the interval is too short you can manually set it on the provider like this (replace the provider URL and chainId with your own):
const avaxProvider = new JsonRpcProvider("https://api.avax-test.network/ext/bc/C/rpc")
avaxProvider.pollingInterval = 10_000;

const config: Config = {
  readOnlyChainId: AvalancheTestnet.chainId,
  readOnlyUrls: {
    [AvalancheTestnet.chainId]: avaxProvider
  },
  pollingInterval: 10000,
  notifications: {
    expirationPeriod: 1000,
    checkInterval: 1000
  }
}

Hey, this doesn't seem to work on my end. I've tried it the exact same way but it seems the intervals are the same even with an interval of 100000.
const provider = new ethers.providers.JsonRpcProvider(
  "https://cloudflare-eth.com/"
);
provider.pollingInterval = 100000;

const config: Config = {
  readOnlyChainId: Mainnet.chainId,
  readOnlyUrls: {
    [Mainnet.chainId]: "https://cloudflare-eth.com/",
  },
  pollingInterval: 100000,
  notifications: {
    expirationPeriod: 1000,
    checkInterval: 1000,
  },
};


Enables outbound traffic by default to allow model downloading

Eventually we may want to bake the models into the image to avoid this workaround. This may cause other challenges with multi-model-serving. Either way, this workaround is necessary for now.

This change is

Ah you're right. Got a little overeager ;)


Add experimental dilithium key pair class

may need to tweak https://github.com/mymindstorm/setup-emsdk a bit.


2.0.0 is incompatible with Wagtail due to requiring beautifulsoup v4.9.1

Hi,
Thanks a ton for your work on django-bootstrap4 — I find it really elegant and useful. And congrats on the 2.0.0 release 🥳
In a Wagtail (2.9) project, I tried updating all our packages using pipenv update. I then got:
ERROR: Could not find a version that matches beautifulsoup4<4.9,<5.0.0,>=4.8,>=4.9.1

As far as I can tell, this is because Wagtail requires beautifulsoup4>=4.8,<4.9 (both in 2.9 and in master), while django-bootstrap4 requires beautifulsoup4 = "^4.9.1".
I can see two possible solutions:

Allow beautifulsoup4 4.8 in django-booststrap4.
Allow beautifulsoup4 4.9 in Wagtail.

It is not obvious to me which solution would be better.

I think django-bootstrap4 should set low demands for external packages, so I'll try and fix it on this side.

Great!
FYI, I found the reasoning behind Wagtail’s versioning constraint in https://github.com/wagtail/wagtail/pull/5817#issuecomment-584580715:

Given that our usage of beautifulsoup is pretty sensitive to breaking changes, I'm wary of including 4.9 and above in the range, so I'll go ahead and set it to >=4.8,<4.9.

CC @gasman

Fixing in https://github.com/zostera/django-bootstrap4/tree/feature/201


Upload WAT plugin documentation to confluence

Put the wat plugin instructions and a copy titled "RTS Plugin documentation" on confluence.

https://www.hec.usace.army.mil/confluence/display/PLUG/Creating+an+RTS+Model+Plugin
Copy with the title renamed.


--debug=explain wrong  with MD5-timestamp

Describe the bug
--debug=explain gives wrong information when using MD5-timestamp decider.
Required information

Link to SCons Users thread discussing your issue
Version of SCons: 3.1.2
Version of Python: 2.7 and 3.6
Which python distribution: python.org
How you installed SCons: pip install scons
What Platform are you on: Windows 7

I observe a strange behavior of --debug=explain, when the MD5-timestamp decider is used. When I change a single file, --debug=explain marks all the "sibling" dependencies as "changed". It does not affect the build itself though, as expected only the necessary files are recompiled.
Consider this SConstruct:
Decider('MD5-timestamp')
Program('hello', 'hello.c')

And the hello.c:
int main(int argc, char** argv) {
    return 0;
}

If I run scons, modify hello.c, and run scons -Q --debug=explain, I get:
scons: rebuilding `hello.obj' because:
           `hello.c' changed
           `C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\cl.EXE' changed
cl /Fohello.obj /c hello.c /nologo
hello.c
scons: rebuilding `hello.exe' because:
           `hello.obj' changed
           `C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\link.EXE' changed
link /nologo /OUT:hello.exe hello.obj

Here, cl.exe is marked as changed. But if there was a .h that the .c included, it would be marked as changed too. Only MD5-timestamp causes this problem, the other deciders work correctly.

Issue is still reproducible on current master (183ced59ecbdf2fc5b7c91324c70ab591bb5d042).
Following modification to test suite:
diff --git a/test/explain/basic.py b/test/explain/basic.py
index 6d41496fd..e37b5be44 100644
--- a/test/explain/basic.py
+++ b/test/explain/basic.py
@@ -99,6 +99,7 @@ cat = Builder(action = [[r'%(python)s', r'%(cat_py)s', '$TARGET', '$SOURCES']])
 one_cat = Builder( action = [[r'%(python)s', r'%(cat_py)s', '$TARGET', '${SOURCES[0]}']])

 env = Environment(tools=[])
+env.Decider('MD5-timestamp')
 env.Append(BUILDERS = {'Cat':cat, 'OneCat':one_cat},
            SCANNERS = kscan)
 env.PrependENVPath('PATHEXT', '.PY')

allows to observe this behavior with:
python3 runtest.py test/explain/basic.py

which among others differences shows python3 as changed:
>            `/usr/bin/python3' changed

Actually it shows ALL file dependencies as changed during execution with --debug=explain. This issue hit me when order of arguments was changing.
It seems to be somehow connected with passing None for parameter node in a call to:
    def changed_timestamp_then_content(self, target, prev_ni, node=None):

I don't know scons internals enough to know why it cause problems as this case seems to be handled.
Actually value of this parameters seems to be used only in MD5-timestamp decider so maybe it should be passed properly during processing output for --debug=explain.

@bdbaddog would you be able to look at it as it seems it could be introduced by you in 8c588f8baf76c0cd491e0589e9d2122787ef2512 ?

If you change the command line in any way, including changing the order of arguments SCons should rebuild.
Unless those parts of the command line which are changing are wrapped in $( $)'s

This issue hit me when order of arguments was changing.

So that rebuild might be valid, though as has been discussed many times --debug=explain is not perfect.
I'll see if I can take a look in the next week or so.

I've mentioned elsewhere, just for recording in this issue: it's "unfortunate" that if the relevant decider decides a rebuild is needed, than the explain logic fires and it tries to decide why - repeating the work. That leads to the potential for inaccuracies in the explaner, since it doesn't have identical code.  I believe we've concluded it was done that way for performance reasons, so that unless you specifically asked for explain, you don't pay any of the cost.

If you change the command line in any way, including changing the order of arguments SCons should rebuild.

Yes, I know and I did not mean to question it in any way. Sorry if I wasn't precise.

This issue hit me when order of arguments was changing.

So that rebuild might be valid,

Yes, it was.

though as has been discussed many times --debug=explain is not perfect.

I wasn't aware of it.
In my case I had a link command with few hundreds of input files and receiving information that all of them are changed did not help at all. And moreover there was no information about command being different but it's another problem. That's why I considered it serious and tried to check if it is reproducible in master and after confirmation provided easy to reproduce testcase. Fixing it currently seem to be to much for me.

I'll see if I can take a look in the next week or so.

Great. Thank you.