I have set up Postgres and Lighthouse and both are running. However, I am facing an issue when trying to run chaind

Steps to Reproduce
Set Up Geth and Lighthouse
I am running two services in different terminals:
1st Terminal (Geth):
geth --http --http.api eth,net,engine,admin --authrpc.jwtsecret ~/secrets/jwt.hex --authrpc.port 8551

2nd Terminal (Lighthouse Beacon Node):
lighthouse bn \
  --network mainnet \
  --execution-endpoint http://localhost:8551 \
  --execution-jwt ~/secrets/jwt.hex \
  --checkpoint-sync-url https://sync-mainnet.beaconcha.in \
  --disable-deposit-contract-sync

Set Up PostgreSQL
I installed PostgreSQL through Homebrew and started it with the following commands:
brew install postgresql
brew services start PostgreSQL

Configure chaind
I configured the chaind YAML file located at ~/.chaind.yml as follows:
# Base log level of the process
log-level: info

# Log output file
log-file: /var/log/chaind.log

chaindb:
  # URL of the PostgreSQL database
  url: postgres://chain:secret@localhost:5432/chain
  max-connections: 16

eth2client:
  # Log level of the Ethereum 2 client module
  log-level: debug
  # Address of the beacon node
  address: localhost:5052

eth1client:
  # Address of the Ethereum 1 node
  address: localhost:8551

blocks:
  enable: true
  address: localhost:5052

validators:
  enable: true
  balances:
    enable: false

beacon-committees:
  enable: true

proposer-duties:
  enable: true

finalizer:
  enable: true

eth1deposits:
  enable: false

summarizer:
  enable: true
  epochs:
    enable: true
  blocks:
    enable: true
  validators:
    enable: false

Run chaind
When I try to run chaind with the following command, it runs but does nothing:
sudo chaind --chaindb.url=postgres://chain:secret@localhost:5432/chain --eth2client.address=http://localhost:5052

The command runs but does not produce any output or fetch the required data.
Request for Assistance: Need to fetch
(I need to fetch data on the withdrawal slot and epoch data of validator index that is shown on the UI of Beacon Chain)
Please provide the correct command to fetch the data and any troubleshooting steps or configuration changes that might be needed.

(I need to fetch data on the withdrawal slot and epoch data of validator index that is shown on the UI of Beacon Chain)

chaind is probably not the best tool for this.  ethdo is likely more useful, for example:
ethdo validator withdrawal --validator=12345
Withdrawal expected at 2024-07-06T00:21:11 in block 9449804

however note that the output is an estimate, as withdrawals can be delayed by missed slots.

I have checked the ethdo repo but couldn't find a command that can provide me with a list of withdrawals slots or epoch of a particular validator index

The command I showed above ethdo validator withdrawal will give you this information (the "block" value in the output).

where should i use this block value?

The block number is the slot in which the withdrawal will occur (assuming no blocks are missed or orphaned between the time you run the command the and block number returned).

I need this info, red box shown in the  SS(either slot or epoch)

ethdo validator withdrawal --validator=1409196
Withdrawal expected at 2024-07-05T11:40:59 in block 9446003

The block shown here is roughly the same as the slot shown in beaconcha.in (we use different methods to estimate the number).  Th epoch can be found by dividing the slot by 32.

how can i get the all he slot numbers

You may need to define your use case a little better here.  Is this data required for a small number of validators, or for all of them?  Do you want to know upcoming withdrawals, or historical ones?
Historical withdrawal data is in blocks, so you could perhaps run ethdo block info for each block you care about and fetch the withdrawals from there:
ethdo block info --json --blockid=9445823 | jq .message.body.execution_payload.withdrawal
[
  {
    "index": "51274027",
    "validator_index": "1406215",
    "address": "0x662d5723210ae19297303d82bd4e6dcac75fe8f8",
    "amount": "18653243"
  },
  {
    "index": "51274028",
    "validator_index": "1406216",
    "address": "0x662d5723210ae19297303d82bd4e6dcac75fe8f8",
    "amount": "18683840"
  },
  {
    "index": "51274029",
    "validator_index": "1406217",
    "address": "0x662d5723210ae19297303d82bd4e6dcac75fe8f8",
    "amount": "18584063"
  },
  {
    "index": "51274030",
    "validator_index": "1406218",
    "address": "0x662d5723210ae19297303d82bd4e6dcac75fe8f8",
    "amount": "18651744"
  },
  {
    "index": "51274031",
    "validator_index": "1406219",
    "address": "0x662d5723210ae19297303d82bd4e6dcac75fe8f8",
    "amount": "18640942"
  },
  {
    "index": "51274032",
    "validator_index": "1406243",
    "address": "0x35cb1491dcf4c0ab6b413afc42298e32f13ff524",
    "amount": "18525182"
  },
  {
    "index": "51274033",
    "validator_index": "1406244",
    "address": "0x35cb1491dcf4c0ab6b413afc42298e32f13ff524",
    "amount": "18623883"
  },
  {
    "index": "51274034",
    "validator_index": "1406245",
    "address": "0xf69c234f91a3cb86242acda97577786481691a51",
    "amount": "18440787"
  },
  {
    "index": "51274035",
    "validator_index": "1406246",
    "address": "0xe0e22593f641b62b1cfe09ae5e05028b4ea20b23",
    "amount": "18667662"
  },
  {
    "index": "51274036",
    "validator_index": "1406247",
    "address": "0x2b78035514401ed1592eb691b8673a93edf97470",
    "amount": "18672245"
  },
  {
    "index": "51274037",
    "validator_index": "1406248",
    "address": "0x6be457e04092b28865e0cba84e3b2cfa0f871e67",
    "amount": "18681916"
  },
  {
    "index": "51274038",
    "validator_index": "1406249",
    "address": "0x6be457e04092b28865e0cba84e3b2cfa0f871e67",
    "amount": "18664385"
  },
  {
    "index": "51274039",
    "validator_index": "1406250",
    "address": "0x210b3cb99fa1de0a64085fa80e18c22fe4722a1b",
    "amount": "63671104"
  },
  {
    "index": "51274040",
    "validator_index": "1406251",
    "address": "0x210b3cb99fa1de0a64085fa80e18c22fe4722a1b",
    "amount": "18405667"
  },
  {
    "index": "51274041",
    "validator_index": "1406252",
    "address": "0x210b3cb99fa1de0a64085fa80e18c22fe4722a1b",
    "amount": "18518530"
  },
  {
    "index": "51274042",
    "validator_index": "1406253",
    "address": "0x2fa3540c237d0955ca97a46749459cd0ab32f982",
    "amount": "18612851"
  }
]

here's my requirement, we have around 56 validator index's  that belongs to this address : 0xf71649551253943fb92cf16cc5c0b23572285965
I am creating automation script that need to claim eth from the slot, so I need all the withdrawal slot numbers of every validator index, that belongs to respective validator index that I am looking for, from starting to current ones

In this case you don't want future information, you want past information.
Your best bet would be to run the ethdo block info command above sequentially for each block and then check the output for your required indices (or withdrawal address).  That will give you the validator index and the amount, which should be enough information for your script to use.

There are various ways in which it is possible to reduce the amount of work if you know the validator indices given that withdrawals are processed in order, however that's beyond the scope of the help I can provide here.  Best to start with a simple approach and then optimize it when necessary.

yeah I have a list of validator indexes
and Getting this error:

The first command tells you when the withdrawal is expected, which is a week in the future.  Obviously such a block has not been produced yet, hence the error on the second command.

So what command i use to get block slots of specific validator index

The information above is to allow you to obtain future slots, and then use that information (which, as mentioned, may alter slightly as you get closer to the appointed time) to read the block after the fact.
I have already explained above how to obtain information for past slots.

May i know now where i am going wrong:

You are using withdrawal rather than withdrawals in your jq path (sorry this was my mistake originally).
Also note that you should be using a local beacon node rather than the fallback; the fallback is rate-limited so you will obtain inconsistent results when it returns a rate limit response rather than the block.

okay, may I know is this the correct way to start the beacon node locally:
Running both the command to setup lighthouse locally
geth --http --http.api eth,net,engine,admin --authrpc.jwtsecret ~/secrets/jwt.hex --authrpc.port 8551
lighthouse bn \   --network mainnet \   --execution-endpoint http://localhost:8551 \   --execution-jwt ~/secrets/jwt.hex \   --checkpoint-sync-url https://sync-mainnet.beaconcha.in \   --disable-deposit-contract-sync

There are many guides out there to set up Ethereum nodes, please use one of those to configure according to your needs.


Support WebFlux

Spring authorization server does not support WebFlux currently, will it support WebFlux in the future?

It is estimated that it will not be compatible with WebFlux until the version is stable.  For the time being, many interfaces need to be adjusted and all functions of OAuth2 need to be improved.

It is estimated that it will not be compatible with WebFlux until the version is stable. For the time being, many interfaces need to be adjusted and all functions of OAuth2 need to be improved.

@xindemeng Closing as duplicate of gh-152. See comment


added unit test for p5.master

added unit test for the methods in p5.master
1.checks for value mastervolume ( set and returned values)
2.checks  for  fading effect of setting masterVolume
3.checks for setting the masterVolume after the given time t
results

@therewasaguy please review this pr!

@therewasaguy  i think we can add it  ?


How to get more information about a 400 ScriptError?

I get the following error for random pages in a crawl (using Undercrawler)
[scrapy_splash.middleware] WARNING: Bad request to Splash: {'description': 'Error happened while executing Lua script', 'info': {'message': 'Lua error: ', 'type': 'LUA_ERROR'}, 'type': 'ScriptError', 'error': 400}
Is there a way to get more specific information about this 400 error?. It seems splash:go throws this error (based on my debugging). I read about the 5 types of error for splash:go (https://splash.readthedocs.io/en/stable/scripting-ref.html#splash-go) - would I see this message for error type number 5 on the referenced page, where Splash can't decide what caused the error?

I'm not sure, but it seems that if error was happening in splash:go, the error would be more specific and there would be a line number. Such an error looks more like an error with the whole lua script happening before the execution, although I'm not sure how it can happen for random URLs, so probably this is also a bad hypothesis.

Actually, I added a log statement right before the splash:go error is caught, and based on that it does look like splash:go is throwing the error. Any ideas on getting more information? Thanks.

@nehakansal I see, that makes sense. I assume this is the line in question https://github.com/TeamHG-Memex/undercrawler/blob/e5104dca5a5d9fe4b9ddd085c7b0935a712ea74d/undercrawler/directives/headless_horseman.lua#L61 - the only idea I have so far is to try replacing
  local ok, reason = splash:go{
    url, http_method=http_method, headers=headers, body=body}
  if #(splash:history()) == 0 then
    error(reason)
  end

with
  assert(splash:go{
    url, http_method=http_method, headers=headers, body=body})

this might give a more detailed error.

Thanks @lopuhin. I updated the code to use assert and interestingly if I use assert on this particular crawl, then I don't get the 400 errors. As soon as I go back to the non-assert code, I get 400 errors on subsequent crawls after a successful one.
To elaborate further - after some amount of time (don't know what's the threshold here), when I run this crawl, the first one is all okay but if I run subsequent crawls on the site soon enough then I get random 400 errors on all the crawls. But I don't see this error behavior with the assert code.
Any clues as to why assert would fix the issue I am seeing?

@lopuhin - any ideas based on my previous comment? Thanks.

Any clues as to why assert would fix the issue I am seeing?

@nehakansal to be honest, this behaviour looks extremely strange to me. Splash should be stateless (apart from caching), spider should also be mostly stateless, the website is not stateless (e.g. bans) but they don't know about the assert. So I'm really puzzled, no good ideas so far.


AutoRefreshingTokenCredential token override issue

Hi.
I have a program that streams some data from an http request to azure storage blob as well as accessing some other api's that also needs to authenticate using the same AutoRefreshingTokenCredential.
This is the logic flow:

Stream all data to storage using put_block (wait to run put_block_list for now)
Access other api's and do some work (uses get_token("<scope>") on the same AutoRefreshingTokenCredential that the storage client uses)
If everything succeeds in step 2, run put_block_list to "finalize" the upload.

The problem I'm facing is that, after running get_token in step 2, step 3. ( put_block_list) returns the following error:
server returned error status which will not be retried: 401

Caused by:
    HttpError {  Status: 401,  Error Code: InvalidAuthenticationInfo,  Body: "b"\xef\xbb\xbf<?xml version=\"1.0\" encoding=\"utf-8\"?><Error>InvalidAuthenticationInfo<Message>Server failed to authenticate the request. Please refer to the information in the www-authenticate header.\nRequestId:<omitted>\nTime:2023-02-21T16:11:53.5576343Z</Message><AuthenticationErrorDetail>Issuer validation failed. Issuer did not match.</AuthenticationErrorDetail></Error>"",  Headers: [   date:Tue, 21 Feb 2023 16:11:53 GMT   content-type:application/xml   server:Microsoft-HTTPAPI/2.0   content-length:402   x-ms-error-code:InvalidAuthenticationInfo   www-authenticate:Bearer authorization_uri=https://login.microsoftonline.com/<omitted>/oauth2/authorize resource_id=https://storage.azure.com   x-ms-request-id:<omitted> ], }

If I skip step 2. then step 3. works.
Another observation I made is that, if I don't use AutoRefreshingTokenCredential, all the steps work:
let creds = Arc::new(DefaultAzureCredential::with_sources(defaul_credentials));
// let creds = Arc::new(AutoRefreshingTokenCredential::new(creds));
return creds;

I'm not really sure what's going on here, but I suspect AutoRefreshingTokenCredential overwrites the current token when I run get_token to call the API's, then when it gets to put_block_list, it uses the token from step 2?
Another thing I'm not sure about is if I even need AutoRefreshingTokenCredential, the documentation is really sparse, all it says is:
Wraps a TokenCredential and handles token refresh on token expiry

Does that mean that, if I don't wrap DefaultAzureCredential in AutoRefreshingTokenCredential, the token might expire? Or does it just keep the token "up to date", so the next time I run get_token or something else, it will be faster?
I hope the explanation is clear enough, if not, I can try to put together some example code.
Also, as a side note, not directly related to this. I'm having trouble calling get_token("<scope>") using managed identity after updating from 0.6.0 to 0.10.0, getting this error message:
Multiple errors were encountered while attempting to authenticate: getting managed identity credential timed out

The code is completely unchanged, all I did was update and rebuild. Downgrading makes it work again.

Thanks for the detailed report.
As you suggest, it looks like the problem is that AutoRefreshingTokenCredential only caches a single token, and the token is at the scope of a particular resource:
https://github.com/Azure/azure-sdk-for-rust/blob/main/sdk/identity/src/token_credentials/auto_refreshing_credentials.rs#L58
There is no check that the resource matches on subsequent requests when the cached token may be returned:
https://github.com/Azure/azure-sdk-for-rust/blob/main/sdk/identity/src/token_credentials/auto_refreshing_credentials.rs#L42
One workaround may be to create separate credentials for your storage client and your other non-storage client.
The "proper" fix is to update the SDK's AzureRefreshingTokenCredential to take account of the resource - perhaps maintaining per-resource tokens.
Your managed identity get_token() issue sounds concerning and needs further investigation - clearly something seems to have regressed.

Thanks for getting back to me so quickly.
If there's anything I can do the help move this forward, please let me know.
Regarding the workaround you proposed, I thought about that, but again, I'm still not sure I need  AzureRefreshingTokenCredential at all. The docs aren't exactly clear on why and when to use it. I assume get_token will fetch a new token once the old one has expired regardless, it will just take longer? My guess is that AzureRefreshingTokenCredential just acts as a cache and makes sure there's always a valid token available when needed, to make requests faster?
As for the get_token() timeout problem on 0.10.0, I will try to isolate the issue to make sure that I'm not doing anything wrong. I might open a separate issue once I can confirm for sure that it is the update that causes the timeout to occur.

In the current implementation, the Credential implementations of get_token() always do the work to fetch a new token, even if the previous token has not expired.  It does no caching of tokens.  I was recently surprised to discover this, and will be raising a new issue to address this as I think we should be caching by default.
For example, see:

AzureCliCredential::get_token()
ClientSecretCredential::get_token()
ImdsManagedIdentityCredential::get_token()

This will clearly impact performance if you are making many API calls.  Using the AutoRefreshingTokenCredential does provide caching and renewing on expiry, but turns out that it suffers from the issue you reported.

As for the get_token() timeout problem on 0.10.0
If this is easily reproducible, it would be very helpful if you could determine which release between 0.6 and 0.10.0 breaks the function for you.  If you determine that it is broken in newer version(s) then yes, please do create a new issue to track it.


[Feature] Agent to automate task of daily social media posting on all social media plateforms, fetching data from the chatGPT prompt based fine tuning agent.

Basically I got the idea of making an agent like to help enterprise or any major business to automate the task of posting on social media on daily basis through agent.
Actually here the two agents will be working for the task to perform, one is for content fetching from prompt engineering for the perticuler niche of business and other will be posting on different social media handles with image and written posts.

See langchain docs. It can be done with that integration easily

Hey there, it seems to be less of an issue of request for crewAI itself and more of something to build with it, I actually built a similar example in the examples repo: https://github.com/joaomdmoura/crewAI-examples/tree/main/instagram_post


feature: document lxml element of proxy classes for advanced users

Proxy classes such as Document, Paragraph, and Table each hold a private reference to the lxml element they correspond to, <w:document>, <w:p>, and <w:tbl> respectively. With these elements, advanced users can call the underlying lxml API directly to develop customized solutions the existing API does not yet support.
Add documentation so advanced users can readily access these elements without consulting the source code.

I wish to use the underlying lxml API to place two table beside each other. For example I want to place table X at the left side and table Y at the right. What table property should I use? Or how do I achieve this?

@progamniac please ask support questions on StackOverflow using the python-docx tag.


Using scrollbarV and Bootstrap makes datatable-body shrink on interaction

I'm submitting a ...  (check one with "x")
[x] bug report => search github for a similar issue or PR before submitting
[ ] feature request
[ ] support request => Please do not submit support request here, post on Stackoverflow or Gitter

Current behavior
When using the Bootstrap theme and enabling body-scrolling using scrollbarH without virtualization makes the datatable-body's calculated height shrink on every interaction.
See the provided stackblitz. When e.g. resizing the window, the body will slowly shrink until only header and footer are visible. This also happens when paging through the table (shrinking by a pixel or two per interaction).
Expected behavior
I'd expect the table's calculated height to not lead to shrinking of the content until they are invisible.
Reproduction of the problem
https://stackblitz.com/edit/angular-uz8cbk
What is the motivation / use case for changing the behavior?
When you have a long table and want to keep header and footer in view, scrollbarV needs to be used. In those scenarios, it is unexpected for the table to gradually shrink.
Please tell us about your environment:

Table version:
This is still an issue with the most recent version.

Angular version:
This is also an issue regardless of angular version (tested both with angular7 / datatable_V14 and angular8 / datatable_V15.

Browser:
all browsers available to me (Edge, Firefox, Chrome)

Language:
all

Same problem!
Any scroll or click event add 3px height to the "datatable-body" element

Same...

same problem here...


0523 api

What problem does this PR solve?
Briefly describe what this PR aims to solve. Include background context that will help reviewers understand the purpose of the PR.
Type of change

[x] Documentation Update

Validated and improved clarity of the API reference. I may have tempered with the intended meaning, please double check.

There is still room for improvement. The API descriptions are still on the thin side, but I think the current version is ready for publication.

Add DEMO link at the first paragraph of README


inconsistency in configuration registry of files.contribution.ts

A key is not spelled the same way in the enum and the enumDescriptions
ModificationsIfAvailable in the first and  ModificationIfAvailable in the second.
I don't know the consequences if any so I won't propose any fix that may break localisation.
See around https://github.com/Microsoft/vscode/blob/main/src/vs/workbench/contrib/files/browser/files.contribution.ts#L324:L324

The second key is for the NLS look up file. If can be named anything, doesn't have to match the enum key.
You can purpose a fix for consistency, but it really doesn't matter.


Stuck at this warning for >20 min: UserWarning: Distributing <class 'dict'> object. This may take some time.

I am trying to use modin to speed up an image processing pipeline that uses pandas dataframes to store images in individual columns together with computed image properties in other columns. I am new to modin, and I was trying it as a drop-in replacement to pandas by changing the import statement to import modin.pandas as pd. However, when I run a function to compute a set of image properties which takes just a few minutes with standard pandas, modin gets stuck at the warning UserWarning: Distributing <class 'dict'> object. This may take some time. for at least 20 min (aborted manually).
I would like to try to narrow down this issue, but I don't know where to start looking and did not find anything on the issue tracker relating to this warning either. The function I am running is rather long, so not sure how helpful it would be to post it here, but it is essentially applying functions over the image columns either individually or together in permutations to compute properties of the images. I am already using joblib to parallelize, but for testing with modin I set the joblib loops to only use a single core.
I am sorry this is not more specific, is there anything that can be said about where to start looking for why I am getting stuck? Or is modin not a good fit for working with dataframes containing images?
I tried this with modin 0.7.3 installed from the conda-forge, and running with the dask backend. I tried installing ray and modin via pip, but could not get it to work (I could try more if you believe it would make a difference). I am on Arch Linux 5.6.15-arch1-1

Hi @joelostblom, thanks for the detailed question! I will explain some details about how Modin works first, then we can try to debug this performance.
When Modin takes in data, it partitions datasets in the way that makes them scalable. So when we get a dictionary object, we have to manually split the data to each partition and this can't (reliably) be done in parallel without making many copies because the data cannot be accessed by all of the Modin workers in parallel. There are still some tricks we can do here to make things marginally faster, but there is a limitation at the Python language level that will be hard to get around.
Ideally, we can ingest the data from a CSV file or some other format. Then all of the workers can access the shared filesystem and read in-parallel. This will be faster than having the data already in memory. Is it possible to read the data in from a file?
The joblib point is another challenge with Modin. If Modin only has 1 CPU it will basically be pandas + some communication overheads, so the more the better here. There are also known issues with Modin working with joblib (#672). I assume you were trying to manually parallelize a large job with pandas before, my recommendation here is to try Modin without joblib.
Your type of workload is a good fit for what we want to do with Modin, so let's see if we can help your case more. A lot of the workarounds people used with pandas to fix certain things are counter-productive in Modin.

Thanks for your reply @devin-petersohn !
Your explanation makes a lot of sense to me. I did not previously understand that modin's parallelization depends is best applied during the reading of data, I thought it was generally when constructing a pandas dataframe. I think this makes sense with the error because I am reading tens of GBs of images into a dictionary that I am then converting to a dataframe and I guess this is where modin/python chokes.
I might try your suggestion with writing these images to a single CSV (probably as flattened arrays then) that I read into modin. I am a bit afraid that the tripled IO (read imgs, write csv, read csv) will negate much of the later performance benefit in terms of overall execution time. Still, I think it is an attractive option to allocate most of the processing time to the initial IO in order to speed up later parts of the pipeline, so I will give this a go (but it might be a little while before I can rewrite that part of the pipeline).
Does this makes sense to you, or do you see any more immediate way of scaling up the image? I have meant to look into storing images in an HDF5 store for other reasons, would this also be a suitable format for reading a dataframe into modin or is CSV preferred?
Once I have this initial dataframe of images, what happens to performance if I construct a second dataframe from the first one (e.g. through a reshape, subset, etc). Would I face similar problems as now since this second dataframe is not created directly from a read operation?

@joelostblom how are you initially reading the images? We have some extensible parsers that would potentially allow us to directly read those in parallel at first. Are you doing something with the dictionary or is it simply needed to create a dataframe? What format are you reading from?
The triple IO can potentially be problematic, and it would depend on what you're doing later. Depending on what you're doing with the dictionary we may be able to avoid this altogether.
HDF5 support is not yet implemented in Modin. There are some limitations to the parent libraries we might depend on and haven't had the time to focus on writing a custom HDF5 engine. Anything we have implemented will work just fine, these can be found on the readme.
Constructing a new DataFrame from another will still be fast. The speed problem is coming from some structure that Modin doesn't control. Python dictionaries are more difficult to distribute because the Python executor that controls it must communicate it directly to other Python executors, so we are limited by how fast Python can loop over the data and send it. After all of this data is split up and distributed, Modin will run fast.

I use imread from scikit-image in a loop using joblib to paralellize over a list of all the file names to read in. The images are in tiff format, and I believe that scikit-image uses the tifffile library under the hood (these are microscopy images, so eventually I might switch to xarray instead of numpy since that seems to provide some conveniences , but this is more of a long term goal).
The only thing I am doing with the dictionary is storing some image metadata (short strings) that is extracted from the file names, but as long as I can store the file name and the image using the modin approach, I could extract this metadata as a separate step. I also downscale the images using scikit-learn right after reading them so that I don't need to store the full size image, but I imagine this would not be a problem to do with the approach you are suggesting (or possible as a separate step afterwards).
Let me know if you need more info or some sample code if it is not clear from the above. Excited to learn how I can speed this up using modin!

I should also say that the image processing I am doing relies on scikit-image and numpy so I would need to eventually have the images in a format that is compatible with those libraries (the scikit-image imread function returns a numpy array).

@joelostblom This helps quite a bit. Some code samples might help here. How many image files are you processing and what does the structure of a row or column look like in the dataframe?
Modin doesn't have a numpy API yet, so this might be a challenge at the current time. We have some plans in the works for this, but (as you can imagine) it is a big undertaking so we're making sure the design is right. We can convert a Modin dataframe to a numpy array at the moment, but it will not be distributed (yet).
We are also building out the extensibility component of the API and custom parsers. This would be a good test of that. There's no good high level API for these at the moment, so it will take some collaboration until we introduce a high level API for custom parsers and functions.

Sorry for the delayed reply @devin-petersohn, I have had some thesis deadlines to tend to over the weekend so got lost in that.
It is unfortunate that there is no numpy API atm, then it sounds like modin might not be the tool I need for my image processing needs right now? In the meantime, I found https://github.com/nalepae/pandarallel which has been working great and fit well with my pipeline since it was mainly a few .apply call on the image columns that constituted the bottleneck for me currently. I believe modin would bring additional benefits when it starts working with numpy images, but for my current application I might stick with pandarallel since I already have a working solution for the performance issues I was experiencing.
In terms of your questions on the data, I am processing anything from 10 to 300-400 images, typically around 100. Then I process these with skimage to crop objects out of the images and measure the properties of these objects. So the initial dataframe has 1-4 big images per row (depending on how many channels we record in the microscope), but the final data frames with measurements has a single of the original images split out into multiple rows, one object per row. This would be the typical structure of the final data frame (truncated from around 20ish columns and hundreds of rows):

major_axis
aspect_ratio
perimeter
area
circularity
img1a
img1b

0
79.103
1.14776
323.492
4222
0.859097
2D np.array
2D np.array

1
89.6788
1.26829
524.653
4836
0.765625
2D np.array
2D np.array

2
78.4372
1.21918
352.599
3909
0.808969
2D np.array
2D np.array

I don't see a particular actionable task in this issue, so I'm going to close it. @joelostblom I hope you are using Modin. Please comment here or in a new issue if you would like help.


Connect-ADOPS -OauthToken from Pipeline

Describe the bug
The System.AccessToken in an Azure Pipeline is valid for some of these API calls but is not possible because of these two endpoints:

https://app.vssps.visualstudio.com/_apis/accounts
https://app.vssps.visualstudio.com/_apis/profile/me

https://github.com/AZDOPS/AZDOPS/blob/e1d7bcadd24ea125405100ba3979370e18f8e95e/Source/Public/Connect-ADOPS.ps1#L63-L66
To Reproduce
Run Azure Pipeline likes the following:
steps:
 - pwsh: |
    Install-Module ADOPS -Scope CurrentUser -Force
    Import-Module ADOPS
    Connect-ADOPS -Organization "$(System.CollectionUri)" -OAuthToken "$(System.AccessToken)"

Expected behavior
There should be no issue to pass the Pipeline App Token to pull some of these resources.

Interesting, and a scenario I haven't tested. Most likely I would guess the scope of the token from the pipeline isn't set correct.
I'll dig in to it to see if and how to fix. Pinging in @PalmEmanuel as well as he may know something from the AzAuth module.

A pipeline token is not that user of ADO so I imagine these routes will also find nothing. The scope require is vso.profile for both.

The main issue isn't finding nothing, the call returns an "log in" page, which we manually throw as a filed login.
This is what happens if we remove ADOPS and run the same manually:

Since the accesstoken is scoped either as proj or org I would say it may be hard, not to mention pretty useless, to even grant it access to external endpoints. How about adding a "SkipVerification" switch that simply bypasses this check and creates the token anyway?
We ( c | sh )ould also include a verbose message to it saying something like "since we wont verify this connection it is up to you to make sure it works."
Would that be a good solution?

I've added a PR to implement this, but would be happy to get some feedback.
Since I haven't implemented pre-releases yet you can test it by cloning the code from the linked PR, #203 , and importing it manually to your repo.
I tested it using the following yaml and it seems to work as expected.
- pwsh: |
    Install-Module AzAuth -Force -Verbose
    Import-Module AzAuth
    Import-Module $(Pipeline.Workspace)/s/adops/Source/ADOPS
    Connect-ADOPS -Organization "$(System.CollectionUri)" -OAuthToken "$(System.AccessToken)" -Verbose -SkipVerification
    Get-ADOPSProject

If you this this is a viable solution I can merge and create a minor release right away.

@bjompen I think this is sufficient!
I had an alternative idea where I would introduce something like Set-ADOPSOauthToken since majority of the cmdlets support passing the organization in.

We had discussions earlier on setting default org, as this was supported in v.1 of the module (with PAT tokens). I don't really remember now why, but there were issues with it and adding it may require a bit more restructure of login behaviour. I am absolutely open to a discussion about it, but lets see what is needed first.


Pix2Pix Code - Generator and Discriminator questions

This repo is amazing!
Ive read the blogs and other post for pix2pix and your code seems to be the most clear but I have some questions about what exactly is going on:

In the "#Downsampling" portion of this code for the Generator, how exactly is any spatial resolution actually being lost (downsampled) if you fix the padding parameter equal to "same"? Technically arent the feature map sizes the exact same for d0 and d7 because of this regardless of stride and filter size? Padding equals same ensures this I thought. Since this part is puzzling I dont see how the "Upsampling" portion of the code (omitted here) is doing any actual upsampling back to d0 spatial resolution since we never got smaller than d0 to begin with. I think I am missing something really obvious here!

    def build_generator(self):
        """U-Net Generator"""

        def conv2d(layer_input, filters, f_size=4, bn=True):
            """Layers used during downsampling"""
            d = Conv2D(filters, kernel_size=f_size, strides=2, padding='same')(layer_input)
            d = LeakyReLU(alpha=0.2)(d)
            if bn:
                d = BatchNormalization(momentum=0.8)(d)
            return d

        def deconv2d(layer_input, skip_input, filters, f_size=4, dropout_rate=0):
            """Layers used during upsampling"""
            u = UpSampling2D(size=2)(layer_input)
            u = Conv2D(filters, kernel_size=f_size, strides=1, padding='same', activation='relu')(u)
            if dropout_rate:
                u = Dropout(dropout_rate)(u)
            u = BatchNormalization(momentum=0.8)(u)
            u = Concatenate()([u, skip_input])
            return u

        # Image input
        d0 = Input(shape=self.img_shape)

        # Downsampling
        d1 = conv2d(d0, self.gf, bn=False)
        d2 = conv2d(d1, self.gf*2)
        d3 = conv2d(d2, self.gf*4)
        d4 = conv2d(d3, self.gf*8)
        d5 = conv2d(d4, self.gf*8)
        d6 = conv2d(d5, self.gf*8)
        d7 = conv2d(d6, self.gf*8)

I am also having a hard time deciphering the code below concerning the discriminator. This patchGan idea is still pretty vague to me (not due to your code), wondering why it even matters but assuming I understood why it was neccessary I am still confused about the discriminator portion of the code since the "validity" variable would have the same spatial resolution as d1, which has the same spatial resolution as img_A/img_B for the same reason I outline above, also there is no explicit downsampling with the disriminator it seems, so there are two reasons why it seems H x W of validity should equal H x W of img_A/img_B, however when we go to train the discriminator we set "valid" and "fake" to a size much smaller than img_A or img_B, in our case img_A and img_B are supposed to be 256,256,3 but disc_patch has dimension 16,16,1. And it is this disc_patch dimension that acts as a target for the discriminator during training but if discrimnator recieves input of 256,256,3 it should return output at the same resolution resulting in 256,256,1 since I cant see anywhere in the code that would get the feature map size down to 16,16,1.  I have more conceptual/fundemental questions about pix2pix in general but I cant ask them until I get past these roadblocks...

     # Calculate output shape of D (PatchGAN)
     patch = int(self.img_rows / 2**4)
     self.disc_patch = (patch, patch, 1) #has dim 16,16,1

    def build_discriminator(self):

        def d_layer(layer_input, filters, f_size=4, bn=True):
            """Discriminator layer"""
            d = Conv2D(filters, kernel_size=f_size, strides=2, padding='same')(layer_input)
            d = LeakyReLU(alpha=0.2)(d)
            if bn:
                d = BatchNormalization(momentum=0.8)(d)
            return d

        img_A = Input(shape=self.img_shape)
        img_B = Input(shape=self.img_shape)

        # Concatenate image and conditioning image by channels to produce input
        combined_imgs = Concatenate(axis=-1)([img_A, img_B])

        d1 = d_layer(combined_imgs, self.df, bn=False)
        d2 = d_layer(d1, self.df*2)
        d3 = d_layer(d2, self.df*4)
        d4 = d_layer(d3, self.df*8)

        validity = Conv2D(1, kernel_size=4, strides=1, padding='same')(d4)

        return Model([img_A, img_B], validity)

    def train(self, epochs, batch_size=1, sample_interval=50):

        start_time = datetime.datetime.now()

        # Adversarial loss ground truths
        valid = np.ones((batch_size,) + self.disc_patch)
        fake = np.zeros((batch_size,) + self.disc_patch)

        for epoch in range(epochs):
            for batch_i, (imgs_A, imgs_B) in enumerate(self.data_loader.load_batch(batch_size)):

                # ---------------------
                #  Train Discriminator
                # ---------------------

                # Condition on B and generate a translated version
                fake_A = self.generator.predict(imgs_B)

                # Train the discriminators (original images = real / generated = Fake)
                d_loss_real = self.discriminator.train_on_batch([imgs_A, imgs_B], valid)
                d_loss_fake = self.discriminator.train_on_batch([fake_A, imgs_B], fake)
                d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)

THANKS FOR YOUR HELP!

Hi,

If you print the shape of the output of the last downsample layer in the generator you will see that the output has the shape (?, 2, 2, 512). You can do this by adding print (d7.shape) in Pix2Pix.build_generator. By setting padding='same' padding will be applied to preserve the spatial resolution of the input given that strides=1. If you use the same setting for padding and set strides=2 padding will be applied in the same way in the previous case, but as stride is set to 2 the spatial resolution of the output is downsampled by a factor of two.

As the input image is propogated through four downsample layers the output shape of the discriminator will be (batch_size, img_height // 2**4, img_width // 2**4, 1) = (batch_size, 256 // 2**4, 256 // 2**4, 1)  = (batch_size, 16, 16, 1).

Thanks for these answers!!

All these years I've thought "same" was ensuring spatial resolution regardless of stride by automatically padding the the input feature maps with zeros...

But I just checked deeper and you are absolutely right "same" is ONLY maintains spatial when stride=1. Since I always use stride equals one, i was confused! Thanks for clarifying this...

Your answer to  number one answers this question of mine as well...

Thanks so much!!!!!!!

Awesome. No problem!

I think I answered my own question about the Patch GAN idea by the way. Is the idea that rather than just letting D output a single number for the entire image as in the traditional sense, you instead force smaller sub patches of the image to be "believable" by D and as a result produce higher quality images from G. This seems to make a lot of sense to me. What is your recommendation for tuning the ouptut spatial resolution of D, is higher better or worse?
It looks like you make use of MSE losses as well is this due to the insights from Least Squares GANs paper (which I thought was fantastic by the way)? I agree with your choice of loss functions just wondering what your rationale is...
THIS CODE IS AMAZING and thanks to your efforts I think I have a 90%+ understanding of what is going with pix2pix.
Concerning this Mode Collapse we always observe is there anything explicit in the code that I am missing that explicitly combats that? Ive heard of this term "mean distance between batch" but I dont know if that actually works well in practice.
Im closing this now but I would love your thoughts still!!
THANKS AGAIN!!!!!!!!!!!!!


Think more about SourcePos

They were created when we were still using antlr, but they play important roles in the backend. Now that we're all-in grammar-kit, maybe the information stored in source pos can be simplified, and we might use text range for error report.

The LSP makes use of them many times. Maybe we can make the lines & columns lazy, but they cannot be removed.
However, Span and LineColSpan can be removed.


Multiple Notification Channel Support for User Account Suspension

Description
The system can be configured to lock a user account after a user account being idle for a predefined period of time. In this scenario, the user will be notified via the preferred communication channel for that user.
Requirement
According to the current implementation, the Identity server can only send email notifications. But there can be a requirement, where the user prefers to receive a notification via an SMS.
User Narrative

User registers an account.
No login activity for a defined period of time.
User account gets locked after the allowed idle account timeout.
The user will be notified via the preferred communication channel if available.

If only the email is available for the user, notify over email.
If only the mobile is available for the user, notify over SMS.
If both the email and the mobile is available,

and if a preferred communication channel is set, notify over the preferred communication channel.
and if a preferred communication channel is not set, notify over the server default notification channel.

Related to https://github.com/wso2/product-is/issues/9754

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


clarify use cases and tests for input data

I've been looking at this and related libraries for processing some JSON stream data, and I was explicitly looking for a tool that would handle non new-line delimited JSON. While this library says that it's for processing new-line json, there isn't any code that handles newline separations, and I believe from looking at the implementation that it should handle streams of multi-line json documents just fine.
It might be nice to add a few test cases and tweak some of the documentation to reflect this. Would you all be interested in this?

Hey! I'm interested for sure but I'm pretty busy for the next week or so. I'll try to get back to this then :)

I did some poking and testing at this over the weekend and even though this seems like it should work (which is the deserializer mode that we're using), it my tests it doesn't seem to work in my (high level) tests. The really confusing wording in the documentation is:

The data can consist of any JSON value. Values need to be a self-delineating value e.g. arrays, objects, or strings, or be followed by whitespace or a self-delineating value.
That seems to cover both newline and other concatenated values. 🤷🏻  My tests were pretty high level so some unittests might be good, I'll poke at that next as I have time.

Ok, I posted that comment and #5 does resolve this.


Add Select Configuration Screen for MATS Emissions

As an ECMPS user, when I click on the MATS navigation item under the Emissions section, I want to be able to see a select configuration screen so that I can properly select the desired data.

[ ] Add select configuration screen view for MATS navigation item

[ ] Able to see the facility and configuration on the screens

Note: This ticket does not include the ability to open the tab

@aprematta Ticket 3962 is ready for 508 testing and has been added to the Emissions Module 508 Testing Epic #4071


Add support for the Einsum operator

Implement ONNX parsing support for the Einsum operator
Resolves https://github.com/migraphx-benchmark/AMDMIGraphX/issues/92

Windows CI is failing.  Seems like...
src\onnx\parse_einsum.cpp:516:21: error: use of undeclared identifier 'ssize_t'

Windows CI is failing. Seems like... src\onnx\parse_einsum.cpp:516:21: error: use of undeclared identifier 'ssize_t'

I replaced ssize_t with int64_t. Seems like it's still failing for some other reason.


chore: update to latest WDIO build

@christian-bromann alpha.369 seems to fail on this test, alpha.367 works fine:
https://github.com/webdriverio-community/wdio-electron-service/blob/682438565e70c26a3a5aeb41edff496fada08ae0/packages/e2e/dom.spec.ts#L17


Can we create Ad-hoc and Development Certificates Using Fastlane ?

New Issue Checklist

[ ] Updated fastlane to the latest version
[ ] I read the Contribution Guidelines
[ ] I read docs.fastlane.tools
[ ] I searched for existing GitHub issues

Issue Description
Complete output when running fastlane, including the stack trace and command used

[INSERT OUTPUT HERE]
Environment

[INSERT OUTPUT HERE]

@bhawin-bigstep Yes, you can create Development and Distribution certs using fastlane 👌


Interactive mode is glitchy on windows regarding whitespace, there's a 10 space wide gap

When I start it on Windows in interactive mode, there is a bunch of whitespace between > and my cursor:

When I start typing, it appends the chars at the cursor but when I press backspace at some point, it "un-indents" the prompt, but now there is a gap in the middle of the prompt!

When I type more, it appends at the end, but when I press backspace again, it shifts the whitespace gap to be directly before the cursor. This gap seems to be always 10 spaces wide:

Also initially:

Might be an issue of the Terminal Emulator. Which one are you using? Windows Terminal or Mintty or else?


api: Add new custom profile field type, "Pronouns"

Fixes: #5517

Marked as draft just because I haven't yet tested this live.

OK -- tested on CZO and all seems good.

LGTM, thanks! Please merge at will.

Thanks for the review! Merging.


Add "s" letter to "Total Broker" in Brokers page

Describe the bug
Please add "s" letter to "Total Broker" within Brokers page
Set up
https://www.kafka-ui.provectus.io/ui/clusters/local/brokers
Screenshots

A thought, change it to "Broker Count" maybe? :)


Train fails

...
Epoch 00010: saving model to /var/tmp/bat/snapshots/resnet50_csv_10.h5
1564 instances of class java-classic with average precision: 0.9435
1650 instances of class vogue-menthe with average precision: 0.9563
mAP: 0.9499
Epoch 11/50
10000/10000 [==============================] - 7936s 794ms/step - loss: 0.2792 - regression_loss: 0.1874 - classification_loss: 0.0191 - masks_loss: 0.0727

Epoch 00011: saving model to /var/tmp/bat/snapshots/resnet50_csv_11.h5
Traceback (most recent call last):
  File "/home/michael/keras-maskrcnn/keras_maskrcnn/bin/train.py", line 313, in <module>
    main()
  File "/home/michael/keras-maskrcnn/keras_maskrcnn/bin/train.py", line 309, in main
    max_queue_size=1,
  File "/home/michael/.local/lib/python3.5/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/home/michael/.local/lib/python3.5/site-packages/keras/engine/training.py", line 1418, in fit_generator
    initial_epoch=initial_epoch)
  File "/home/michael/.local/lib/python3.5/site-packages/keras/engine/training_generator.py", line 251, in fit_generator
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/home/michael/.local/lib/python3.5/site-packages/keras/callbacks.py", line 79, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "/home/michael/.local/lib/python3.5/site-packages/keras_retinanet/callbacks/common.py", line 31, in on_epoch_end
    self.callback.on_epoch_end(epoch, logs=logs)
  File "/home/michael/keras-maskrcnn/keras_maskrcnn/bin/../../keras_maskrcnn/callbacks/eval.py", line 66, in on_epoch_end
    save_path=self.save_path
  File "/home/michael/keras-maskrcnn/keras_maskrcnn/bin/../../keras_maskrcnn/utils/eval.py", line 212, in evaluate
    mask = cv2.resize(mask, (box[2] - box[0], box[3] - box[1]))
cv2.error: OpenCV(3.4.3) /io/opencv/modules/imgproc/src/resize.cpp:4045: error: (-215:Assertion failed) !dsize.empty() || (inv_scale_x > 0 && inv_scale_y > 0) in function 'resize'

It looks like you tried to load an empty image when evaluating at the end of a training epoch

It looks like there is an empty mask in your validation set

my dataset was generated using rendering software. I calculated bboxes from masks:
            mask = cv2.imread(maskPath, cv2.IMREAD_GRAYSCALE)
            where = np.array(np.where(mask>127))
            if where.size > 0:
                y1, x1 = np.amin(where, axis=1)
                y2, x2 = np.amax(where, axis=1)

Strange that the error occurs after several epochs, but that error does usually indicate that a file isn't properly loaded. You could try printing type(mask) before the line where it crashes. I'm guessing it will say NoneType just before it crashes. Maybe if you also print the file where it crashes on, you get some more perspective of what is happening. Maybe it's always happening on the same mask?

This problem dispraiser then I add checks for very small masks:
if abs(x2-x1)<8 or abs(y2-y1)< 8 or (x2-x1)*(y2-y1)<400:

I think small mask, like one dot, can raise this error again.

Hmm even small masks shouldn't cause a crash. Could you make a small dataset containing one image and one object in that image that crash? Then I can test on it further to fix it to ensure it won't crash.

Sorry, I cant do this. Its not public dataset.

If you know how to make the issue happen, you can try to do it on a publicly available image?

I generated several masks, that may cause this error: http://i87.beon.ru/masks.tar .

again get this error. At the same epoch, after saving snapshot. I think, snapshot saving do something wrong. For example, corrupt information about bboxes.
1784 instances of class 111 with average precision: 0.8032
1745 instances of class 222 with average precision: 0.8715
1822 instances of class 333 with average precision: 0.8122
1814 instances of class 444 with average precision: 0.8563
1827 instances of class 555 with average precision: 0.8343
mAP: 0.8355
Epoch 9/50
10000/10000 [==============================] - 12373s 1s/step - loss: 0.5614 - regression_loss: 0.3762 - classification_loss: 0.0587 - masks_loss: 0.1265

Epoch 00009: saving model to /var/tmp/xxx/snapshots/resnet50_csv_09.h5
1784 instances of class 111 with average precision: 0.8047
1745 instances of class 222 with average precision: 0.8764
1822 instances of class 333 with average precision: 0.8262
1814 instances of class 444 with average precision: 0.8571
1827 instances of class 555 with average precision: 0.8250
mAP: 0.8379
Epoch 10/50
10000/10000 [==============================] - 12646s 1s/step - loss: 0.5286 - regression_loss: 0.3544 - classification_loss: 0.0538 - masks_loss: 0.1203

Epoch 00010: saving model to /var/tmp/xxx/snapshots/resnet50_csv_10.h5
1784 instances of class 111 with average precision: 0.8207
1745 instances of class 222 with average precision: 0.8744
1822 instances of class 333 with average precision: 0.8388
1814 instances of class 444 with average precision: 0.8477
1827 instances of class 555 with average precision: 0.8351
mAP: 0.8434
Epoch 11/50
10000/10000 [==============================] - 12794s 1s/step - loss: 0.4994 - regression_loss: 0.3351 - classification_loss: 0.0495 - masks_loss: 0.1148

Epoch 00011: saving model to /var/tmp/xxx/snapshots/resnet50_csv_11.h5
Traceback (most recent call last):
  File "/home/xxxx/keras-maskrcnn/keras_maskrcnn/bin/train.py", line 313, in <module>
    main()
  File "/home/xxxx/keras-maskrcnn/keras_maskrcnn/bin/train.py", line 309, in main
    max_queue_size=1,
  File "/home/xxxx/.local/lib/python3.5/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/home/xxxx/.local/lib/python3.5/site-packages/keras/engine/training.py", line 1418, in fit_generator
    initial_epoch=initial_epoch)
  File "/home/xxxx/.local/lib/python3.5/site-packages/keras/engine/training_generator.py", line 251, in fit_generator
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/home/xxxx/.local/lib/python3.5/site-packages/keras/callbacks.py", line 79, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "/home/xxxx/.local/lib/python3.5/site-packages/keras_retinanet/callbacks/common.py", line 31, in on_epoch_end
    self.callback.on_epoch_end(epoch, logs=logs)
  File "/home/xxxx/keras-maskrcnn/keras_maskrcnn/bin/../../keras_maskrcnn/callbacks/eval.py", line 66, in on_epoch_end
    save_path=self.save_path
  File "/home/xxxx/keras-maskrcnn/keras_maskrcnn/bin/../../keras_maskrcnn/utils/eval.py", line 212, in evaluate
    mask = cv2.resize(mask, (box[2] - box[0], box[3] - box[1]))
cv2.error: OpenCV(3.4.3) /io/opencv/modules/imgproc/src/resize.cpp:4045: error: (-215:Assertion failed) !dsize.empty() || (inv_scale_x > 0 && inv_scale_y > 0) in function 'resize'

Exception ignored in: <bound method BaseSession.__del__ of <tensorflow.python.client.session.Session object at 0x7f557ea91c18>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 696, in __del__
TypeError: 'NoneType' object is not callable

again get this error. At the same epoch, after saving snapshot. I think, snapshot saving do something wrong. For example, corrupt information about bboxes.

Did you try my previous suggestion?

Strange that the error occurs after several epochs, but that error does usually indicate that a file isn't properly loaded. You could try printing type(mask) before the line where it crashes. I'm guessing it will say NoneType just before it crashes. Maybe if you also print the file where it crashes on, you get some more perspective of what is happening. Maybe it's always happening on the same mask?

again get this error. At the same epoch, after saving snapshot. I think, snapshot saving do something wrong. For example, corrupt information about bboxes.

Did you try my previous suggestion?

Strange that the error occurs after several epochs, but that error does usually indicate that a file isn't properly loaded. You could try printing type(mask) before the line where it crashes. I'm guessing it will say NoneType just before it crashes. Maybe if you also print the file where it crashes on, you get some more perspective of what is happening. Maybe it's always happening on the same mask?

No, I used my private dataset.

I add some debug prints. Tomorrow try to get some information.

I added debug print(type(mask), box[2], box[0], box[3], box[1]) and started training on very small dataset with --steps=1 . It worked 50 epoches and finished without errors. Magic...

I start training with debug on full dataset. May be after 2 days I get some useful information.

<class 'numpy.ndarray'> 163 143 642 464
<class 'numpy.ndarray'> 156 151 632 468
<class 'numpy.ndarray'> 837 815 639 470
<class 'numpy.ndarray'> 619 601 635 460
<class 'numpy.ndarray'> 801 725 684 518
<class 'numpy.ndarray'> 132 119 631 466
<class 'numpy.ndarray'> 258 190 634 477
<class 'numpy.ndarray'> 1034 1008 936 767
<class 'numpy.ndarray'> 715 646 635 482
<class 'numpy.ndarray'> 533 457 598 558
<class 'numpy.ndarray'> 615 613 636 472
<class 'numpy.ndarray'> 569 572 637 472
Traceback (most recent call last):
  File "/home/michael/keras-maskrcnn/keras_maskrcnn/bin/train.py", line 313, in <module>
    main()
  File "/home/michael/keras-maskrcnn/keras_maskrcnn/bin/train.py", line 309, in main
    max_queue_size=1,
  File "/home/michael/.local/lib/python3.5/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/home/michael/.local/lib/python3.5/site-packages/keras/engine/training.py", line 1418, in fit_generator
    initial_epoch=initial_epoch)
  File "/home/michael/.local/lib/python3.5/site-packages/keras/engine/training_generator.py", line 251, in fit_generator
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/home/michael/.local/lib/python3.5/site-packages/keras/callbacks.py", line 79, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "/home/michael/.local/lib/python3.5/site-packages/keras_retinanet/callbacks/common.py", line 31, in on_epoch_end
    self.callback.on_epoch_end(epoch, logs=logs)
  File "/home/michael/keras-maskrcnn/keras_maskrcnn/bin/../../keras_maskrcnn/callbacks/eval.py", line 66, in on_epoch_end
    save_path=self.save_path
  File "/home/michael/keras-maskrcnn/keras_maskrcnn/bin/../../keras_maskrcnn/utils/eval.py", line 215, in evaluate
    mask = cv2.resize(mask, (box[2] - box[0], box[3] - box[1]))
cv2.error: OpenCV(3.4.3) /io/opencv/modules/imgproc/src/resize.cpp:4045: error: (-215:Assertion failed) !dsize.empty() || (inv_scale_x > 0 && inv_scale_y > 0) in function 'resize'

box[2] - box[0] is less than zero.

I got it again:
<class 'numpy.ndarray'> 295 277 759 614
<class 'numpy.ndarray'> 768 706 761 728
<class 'numpy.ndarray'> 802 789 752 617
<class 'numpy.ndarray'> 574 551 750 620
<class 'numpy.ndarray'> 1095 1066 768 626
<class 'numpy.ndarray'> 993 923 763 614
<class 'numpy.ndarray'> 1061 999 761 729
<class 'numpy.ndarray'> 1244 1180 762 618
<class 'numpy.ndarray'> 696 668 765 639
<class 'numpy.ndarray'> 1049 1007 721 629
<class 'numpy.ndarray'> 1049 1007 758 669
<class 'numpy.ndarray'> 131 135 723 631
Traceback (most recent call last):
  File "/home/xxxxx/keras-maskrcnn/keras_maskrcnn/bin/train.py", line 313, in <module>
    main()
  File "/home/xxxxx/keras-maskrcnn/keras_maskrcnn/bin/train.py", line 309, in main
    max_queue_size=1,
  File "/home/xxxxx/.local/lib/python3.5/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/home/xxxxx/.local/lib/python3.5/site-packages/keras/engine/training.py", line 1418, in fit_generator
    initial_epoch=initial_epoch)
  File "/home/xxxxx/.local/lib/python3.5/site-packages/keras/engine/training_generator.py", line 251, in fit_generator
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/home/xxxxx/.local/lib/python3.5/site-packages/keras/callbacks.py", line 79, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "/home/xxxxx/.local/lib/python3.5/site-packages/keras_retinanet/callbacks/common.py", line 31, in on_epoch_end
    self.callback.on_epoch_end(epoch, logs=logs)
  File "/home/xxxxx/keras-maskrcnn/keras_maskrcnn/bin/../../keras_maskrcnn/callbacks/eval.py", line 66, in on_epoch_end
    save_path=self.save_path
  File "/home/xxxxx/keras-maskrcnn/keras_maskrcnn/bin/../../keras_maskrcnn/utils/eval.py", line 215, in evaluate
    mask = cv2.resize(mask, (box[2] - box[0], box[3] - box[1]))
cv2.error: OpenCV(3.4.3) /io/opencv/modules/imgproc/src/resize.cpp:4045: error: (-215:Assertion failed) !dsize.empty() || (inv_scale_x > 0 && inv_scale_y > 0) in function 'resize'

Exception ignored in: <bound method BaseSession.__del__ of <tensorflow.python.client.session.Session object at 0x7feaf3a51c50>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 696, in __del__
TypeError: 'NoneType' object is not callable
>

I look to the val.csv and can`t find bboxes with size from this error.

Hey @MichaelMonashev did you find a fix to this error : (inv_scale_x > 0 && inv_scale_y > 0) in function 'resize'?

@RohitMidha23 , i don`t remember. :-(
I am using keras-retinanet and have not this error.

Ohh. Thanks anyway @MichaelMonashev


Allow cancellation of message processing function

The onMessageAsync is currently a function which does not take the cancellation token:
Task StartMessagePumpAsync(Func<IReadOnlyCollection<QueueMessage>, Task> onMessageAsync, ..., CancellationToken cancellationToken)

Wouldn't it be better if it would pass in the received CT? i.e. use
Func<IReadOnlyCollection<QueueMessage>, CancellationToken, Task> onMessageAsync

https://github.com/aloneguid/storage/blob/master/src/Storage.Net/Messaging/IMessageReceiver.cs#L38
This might be required to quickly shut down a process - a message processing might take multiple minutes and thus it also should be cancellable. If a message processing is cancelled then it is either not commited and/or rolled back (i.e. goes back to the queue)

Agree, makes perfect sense, CT can be populated deeper.

released in nuget


Improved Knee Testing

This pr aims to improve the knee testing functionality. When succesful we can close #28 and #29

@thht  can you check this out and cache the mne stuff. I think we can close this issue afterwards :)

can you check this out and cache the mne stuff.

Actually, I don't think this makes much sense. Caching means that the data is tarred and gzipped and then uploaded to a host and the next job just downloads it and untars/gzips it. So, basically what fetching the dataset does in the first place.
An easier route would be to use the testing dataset instead of the sample one. It contains all the files you need and it is stored on codeload.github.com which should be an internal github server. This should increase the download speed because the sample dataset is hosted on osf which is an external server.

Thanks for adding the files to lfs @thht. Tests seem to work so if you are happy lets merge and close some issues :)

Sure, punch it!


Selenium and Ubuntu 1404 troubles

I'm having troubles getting a VM provisioned with vagrant box geerlingguy/ubuntu1404 and selenium (noticed with latest acquia/blt). To confirm this was a Drupal-VM problem (or further upstream), I downloaded Drupal-VM directly, changed the vagrant_box config setting, and uncommented selenium from the installed_extras item, that's it.
I had the problem below after the above steps, and I've also had a similar problem at the "Download Selenium" step which said something along the lines of "failed to create temporary content file."
TASK [arknoll.selenium : Download Selenium] ************************************
changed: [drupalvm]

TASK [arknoll.selenium : Install FireFox (if configured)] **********************
skipping: [drupalvm]

TASK [arknoll.selenium : Install Chrome (if configured, Debian)] ***************
fatal: [drupalvm]: FAILED! => {"changed": false, "failed": true, "module_stderr": "", "module_stdout": "\r\nTraceback (most recent call last):\r\n  File \"/tmp/ansible_JFJ8DV/ansible_module_apt.py\", line 842, in <module>\r\n    main()\r\n  File \"/tmp/ansible_JFJ8DV/ansible_module_apt.py\", line 798, in main\r\n    force=force_yes, dpkg_options=p['dpkg_options'])\r\n  File \"/tmp/ansible_JFJ8DV/ansible_module_apt.py\", line 538, in install_deb\r\n    diff[\"prepared\"] += '\\n\\n' + out\r\nKeyError: 'prepared'\r\n", "msg": "MODULE FAILURE", "parsed": false}

NO MORE HOSTS LEFT *************************************************************

RUNNING HANDLER [geerlingguy.firewall : restart firewall] **********************

RUNNING HANDLER [geerlingguy.apache : restart apache] **************************

RUNNING HANDLER [geerlingguy.php : restart webserver] **************************

RUNNING HANDLER [geerlingguy.php : restart php-fpm] ****************************

RUNNING HANDLER [geerlingguy.mysql : restart mysql] ****************************
    to retry, use: --limit @/Users/dpagini/Projects/Websites/drupal-vm/provisioning/playbook.retry

PLAY RECAP *********************************************************************
drupalvm                   : ok=165  changed=64   unreachable=0    failed=1

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

Any help or thoughts are much appreciated!  Thanks.

Note that the build tests for the upstream role are currently failing... https://travis-ci.org/arknoll/ansible-role-selenium — I don't have time to look right now, but it could be something related to the PPA's/repos being used for Ubuntu.

Redirecting to upstream role: https://github.com/arknoll/ansible-role-selenium/issues/20
The above issue might also fix this problem and any others with install/running.


METAMODEL-1088 Fix aliases for MongoDB modules

This PR includes the changes to fix METAMODEL-1088 by checking whether the columns contain aliases and if so just delegating the query to the QueryPostprocessDataContext.

I know nothing about Mongodb, but code looks good :)
It does uncover one (unrelated) thing, though: It seems Mongo2 & 3 modules are very similar (a qucik glance shows no difference in your changes for 2 & 3), which makes me think more more stuff might belong in common?

Completely agree @LosD. When applying the changes I thought the same. I was about to start refactoring the code but I thought that I shouldn't do this within this PR.
I'd open a new jira to sort this out, what do you think?

Yeah, that makes sense.

Jira created: METAMODEL-1089
Thank you @LosD

OK to merge? Anyone?

+1 from me, but again my MongoDB knowledge is pretty limited. :)

Thank you! I'll wait another day for anyone who like to comment on this and then will merge it

+1 from me too.
A side-note: Have anyone checked if the same fix can be applied to e.g. CouchDB or other data context implementations? I think a similar approach of using that allSelectItemsAreColumns flag is used in other places.

Wait what ... Hmm I see that this is actually a fix that works functionally, but will mean that if you use an alias then the query will not be optimized at all. Is that what we want? I can live with it, since functionality is more important than non-functionals. But some people could take a big performance hit over this if they use an alias. Would be better to fix that too.

I agree @kaspersorensen. I've just pushed some changes to optimize the query even though it contains aliases.

Great, +1 from me!

Cool! Thank you @kaspersorensen and @LosD. I'll merge this.


⚠ ClamAV has degraded performance

In 4888f76, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 48 ms

Resolved: ClamAV performance has improved in a731640.


Refine counts for proximity

Shows 10 results to start
Show more button extends by 10 when clicked
Still ordered by proximity

Closes #160

LGTM pull request analysis was skipped for 2c94cb7dfdd1cca7628748c9bb145c48cb044db4 by mischa-s. Analysis of future commits will happen as normal.

Comment posted by LGTM.com


[FW Update][GUI] Label has not been translated

The label has not been translated into Portuguese.
Steps to reproduce the problem:

click Transfer image to device

pt-br

en

click Apply image update

pt-br

en

click Set device to 'On Hold'

pt-br

en

Version: 0.5.2-rc.1
Environment: k8s17
Operating system: Ubuntu 20.04

Analysed. It should be handled in the future.

Analysed. It should be handled in the future.

It will no longer be prioritized.


[Meta bug] GNU testsuite

The goal of this meta bug is to reference all GNU tests failing and what we need to fix them.

[ ] https://github.com/uutils/coreutils/issues/3320
[ ] https://github.com/uutils/coreutils/issues/3332
[x] https://github.com/uutils/coreutils/issues/3331
[x] https://github.com/uutils/coreutils/issues/3333
[x] https://github.com/uutils/coreutils/issues/3171
[x] https://github.com/uutils/coreutils/issues/3163
[ ] https://github.com/uutils/coreutils/issues/3147
[ ] https://github.com/uutils/coreutils/issues/2930
[x] https://github.com/uutils/coreutils/issues/3337
[x] https://github.com/uutils/coreutils/issues/3519

We moved to project mgmt for this:
https://github.com/orgs/uutils/projects/1


64-bit variant?

Since waterhash is internally 64 bits wide and is only truncated to 32 at the very end, has anyone tried running SMHasher on a 64-bit version - the same  waterhash but without the final shift and subtract with truncation to uint32_t? Would it pass the tests same as the 32-bit version does?
Is woothash what I'm looking for? It has quite a bit more code so I'm confused about how similar they are in essence. woothash is faster, but has slightly worse bias (not a meaningful difference, but still. In fact, waterhash's bias is the lowest I've seen between all the hash functions I've looked at yet).

You probably want wheathash, which is almost exactly what you said (waterhash modified to output 64 bits), other than a slightly more-involved final step when compared to waterhash. Waterhash and wheathash both process inputs in 32-bit blocks, whereas woothash processes 64-bit blocks and needs to work harder to mix them without access to 128-bit math. The speed on all of these depends a lot on your environment; I have ported these all to Java and the water/wheat/woot family is typically my go-to these days for non-cryptographic hashing in Java.

Thanks a lot! For the explanation, and for your work.
How do you choose between water / wheat / woot, and especially between the last two?

Wheat is meant for inputs in 32-bit chunks, which doesn't matter here much, but does in languages like Java that hash arrays by item rather than by byte. Woot is meant for inputs in 64-bit chunks, like the original wyhash. In my Java code (which is here), I use waterhash when hashing 32-bit-or-less items to get a 32-bit result, wheathash when hashing 32-bit-or-less items to get a 64-bit result, and woothash when hashing 64-bit items to get any result. In C or C++, where I admit I have never used a hash table (I only write hashes in C or C++ to test them with SMHasher), hashing just operates on bytes, so woothash would probably be best for speed. For collision resistance, I don't really know if any of these qualify as resistant, but woothash has a different variety of mum()-like function, and that may make it better or worse.


reformat codebase using python/black

I open this PR just to see if the test are still running after reformatting using python/black

Since black does not exist for Py2, I first decided to verify codestyle only on Py3 (after all the style check is Py version independent). But having conditional steps turned out to be rather hard.
In the end I thought:
Why not define one dedicated job for all style checks?
So now the Style checks are run using py3.7 and for all other Pythons we skip the style checks and only build and run the test suite.

@cclauss  and @StoicLoofah could this be merged?

sorry for being delinquent on merging the other branches, but there are now conflicts to resolve here. @dneise can you take a look at those?
Regarding the discussion on documenting and enforcing black, this is exactly what i was hoping for. thanks for integrating those!
... now i need to get black setup myself =)

The precommit hook is the way to go...  It blackens the code in the commit process.

 Maybe you could to a squash + merge commit? in order to have a cleaner history?
After all, the only thing I did in this PR was black . and then figure out how to verify blackness on circleci.


fix(#27): Chevron icons: multiple elements having same id should be avoided

Description

Related Issue

#27
Motivation and Context
In React-Spectrum documentation pages that include ChevronRight in the rendering of the open or closed state for <details>/<summary>, automated accessibility testing returns a warning because the page contains duplicate ids.
How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] I have signed the Adobe Open Source CLA.
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

@snowystinger @majornista I will need to let @Hinschmi know as he owned the script that generate these react icon component.

@Hinschmi please review the code change and modify your code generation script, then merge the PR when you're ready.

@jianliao @Hinschmi verified in RSP and upgraded, thank you!


Eslint

Overview of the Pull Request:
I noticed Prettier and Eslint were installed but never set up properly. That's why we had to run Prettier manually all the time. I just configured them to work together.
Now it highlights the format as you go, no need to run Prettier manually (unless you need to for all the docs at once which can still be done):

Please feel free to add more Prettier settings at .prettierrc file.
Test
Create a new .jsx file. Make a formatting error. See if the code gets highlighted.
Pull Request Checklist:
Make sure the following checkboxes[ ] are checked with x before sending your PR -- thank you!

[x] Have you included a link Trello card / Github issue and written sufficient description to help others review your work?
[x] Is your pull request up-to-date with main branch?
[x] Have you checked That code is well formatted? Did npx prettier --check . return no warnings/errors?
[x] Have you written instructions on how to test that your changes work as expected?
[x] Have you tested your work thoroughly on your local machine to ensure you are not breaking anything?

As discussed in this evening's dev huddle, we have decided to close this PR, and revisit this at a later date


[Feature request] Build all packages and then install them to make use of the pacman hooks

When building several packages or when updating, pacaur builds one package at a time and then installs it before doing the next package. This means that the hooks lose their use as they are triggered as many times as there are packages. This is very annoying with fonts for example. It would be better if the packages were all built and then installed at once, then the hooks could be used properly. Of course, this means installing the build dependencies first.

Of course, this means installing the build dependencies first.

There is no easy way to determine if a dependency of a package is also a required build dependency, so installing dependencies is always done before installing a package.
At first I thought that using a local repo (#643) would avoid that issue, but I'm not even sure that would help because of the above-mentioned requirement.
Individual packages from a split base package are however installed at the same time.

Alright. Didn't think of that... Maybe the "solution" would be to install one level of dependency at once, but that sounds like quite the hassle for not a lot gained... Sorry for the noise.

Maybe the "solution" would be to install one level of dependency at once

This doesn't account for the fact that quite some edge cases aren't covered by a simple "level of dependency" tree order. The dependency tree must be topologically sorted to always ensure a correct build order.
What would need to be done to allow installation by batch is to remember the whole dependency graph (not just the correct build order order). This requires a lot of additional complexity - probably a whole lib, as I don't think this could be done solely in bash, and eventually not worth the effort/time.
A similar issue has been discussed for the Aura helper (see https://github.com/aurapm/aura/issues/153), although not fixed either.

Clearly, a much more complex topic than I realised at first glance. It does seem tricky to implement in bash.


Plugin Version 3.3.5: Lack of brightness and color temp of entities

Plugin 3.3.4 with Hass 2023.12: no problem

Plugin 3.3.5 with Hass 2023.12 and Hass 2024.1: all brightness and temp of light entities are missing

list of my light entities:
mesh lemesh.light.wy0c15
mesh lemesh.light.wy0c08
unable to find brightness and temp attribute

same here
mesh lemesh.light.wy0c15

The same for mesh MJTS01YL/MJTS003 by Xiaomi

I have the same problem with for Aqara Smart Ceiling Light L1-350 (ZNXDD01LM, lumi.light.acn003).
Xiaomi gateway DMWG03LM, fw 1.0.7_0019. Component v3.3.5. HA 2023.12.4.

Hello, I rolled back locally changes from one of last commits, and bright and color temp starts working.

Look's like it is early to switchover the deprecated constants

Hi, created an MR with proposed changes by @avihome63

same for YLTS02YL/YLTS04YL and mesh yeelink.light.mb1grp

same issue!!!

Hello, I rolled back locally changes at 'light.py' that was introduced in one of last commits  in master, and bright and color temp starts working.

Get back wildcard import  from homeassistant.components.light import *
Rolled back line self._attr_supported_features |= SUPPORT_BRIGHTNESS | SUPPORT_TRANSITION
Rolled back line self._attr_supported_features |= SUPPORT_COLOR_TEMP

Look's like it is early to switchover from the deprecated constants to new ones

it worked for me. thanks

Same issue. With 3.3.5 all my BLE devices only had "ON/OFF" control (no more brightness or color control).
I decided to remove and readd/reconfigure the gateway, even worse, now all my lamp entities don't have any controls, ex:

Hello, I rolled back locally changes at 'light.py' that was introduced in one of last commits  in master, and bright and color temp starts working.

Get back wildcard import  from homeassistant.components.light import *
Rolled back line self._attr_supported_features |= SUPPORT_BRIGHTNESS | SUPPORT_TRANSITION
Rolled back line self._attr_supported_features |= SUPPORT_COLOR_TEMP

Look's like it is early to switchover from the deprecated constants to new ones

Do I need to restart HA after editing light.py?

https://github.com/AlexxIT/XiaomiGateway3/releases/tag/v3.3.6

Thanks @AlexxIT!
When will 3.3.6 be available within HA HACS page? I don't see any proposed update so far


chore: fixed some grammatical glitches and

Fixed some grammer mistakes.

cc @ibakshay

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


_v attribute is not changed after update

the '_v' attribute in my DB is not changed after update action, it's still 0.

Please post your code so that I can reproduce the issue.
That being said, if you're using any of Model.update(), Model.findByIdAndUpdate() or Model.findOneAndUpdate(), you're using the underlying MongoDB functions directly and bypassing the Mongoose ODM. This plugin works on document updates using Document.save(). See Caveats in the README.

Closing due to lack of response.


feat: add dynamic announcement layer to navbar with redirect link and icons

As per the Instructions, included the right, left and center portion in upper layer .

preview changes on : https://gnome-front.vercel.app/

Center all three elements with some spacing between them.
Replace the GNOME logo with an emoji and add a small square button for the redirection link, styled with the theme colors (black and white).
Update the announcement text to: “We are organizing UbuCon 2025! Volunteer registration is now open. Apply here.”
Lastly, either change the background color to something other than white to distinguish it from the navbar or add a shadow effect to the box for better separation.

After that, It is good to be merged


Meetup Zagreb about RChain

nathan is coordinating in New York.
ask about central meetup

Also @jbassiri has created a useful generic RChain presentation, see #102.

Thanks @kitblake, I will definitely need it and use it.

@tgrospic I'm going through all marketing issues, seeing if some can be closed.
Is this issue still relevant?
Has the meetup successfully happened?
If it's no longer relevant, then please close the issue.

i believe it is still on, i`ll contact @tgrospic TY

Yes, it's still relevant. @InternautNeven and @MarioPolo recently joined the team. They started translation of RChain whitepaper #449 and already done translation of Contributing document #443.
We are closer to our plan to organize monthly meetups. I also hope for useful information from the governance conference. :+1:
We will update this issue with information about our first meetup so everyone can follow.

Ok, great!!

Meetup will probably be in the begining of May, 03.05.2018. Thursday  here [ImpactHub] (https://zagreb.impacthub.net/ )
We already made a [poster] (https://postimg.org/image/pxxridyrv/ ) (local artist)  and translating several documents #544 #382 #449


Adjust handling of GDAL dependency when building RavenPy on conda-forge

The GDAL problems in RavenPy and on conda-forge have been an issue because of a handful of problems.
In a pure Python environment:

The Python-GDAL and GDAL C-libraries need to match exactly in version number.
The pip package resolver will always install the newest version of a dependency available on PyPI.
The solution: pin the dependency of GDAL to the version of the C-library of GDAL at install time.

When a Python package is built on Anaconda:

Different phases of the packaging process (build, host, run) can - and often do - have different versions, unless they are specifically pinned.
The workaround: pin the package to specific versions and offer those as separate built binaries.

The issue here is that pinning packages to specific dependency versions will cause them to be harder to resolve in a complex dependency environment. It also prevents us from being able to use patch releases, unless we specifically build them. Therefore, If we pin support for GDAL ==3.1, 3.2, or 3.3, we are only building/supporting the specific packages found at build time (e.g. 3.1.12, 3.2.8, 3.3.1). This is likely why we have so many issues with the dependency resolver in conda-forge.
This PR changes this by examining whether the package is being built by conda-forge, and assumes that the dependency resolver will know that if we ask for gdal (python-gdal), the underlying libgdal (C-library) should be installed with a compatible version.
After this is pushed, all we need to do is rebuild both RavenPy and Raven-WPS on conda-forge with the following adjustment to their recipes to make this work:
build:
  ...
  script_env:
    - CONDA_BUILD

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 88.731%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
5047

Relevant Lines:
5688

💛  - Coveralls

coverage: 88.731%. remained the same
when pulling e2224ec3efa9e853462451fb66ac3cf06f1d6986 on gdal_handling
into aec47bcdda14323bdee660d280550625c269b5e8 on master.

Wha????


mobile version

Hello,
Do you have any plan for mobile version?

Hi @alighasemzadeh ,
We are currently working on PWA version of the shop. It will be released end of this month.

Ok,Thank you.
I think flutter is good choice for this job.

@alighasemzadeh We have storefront APIs implemented in our API packages, check for the routes in the API package. Then using it you can use any mobile SDK such as Android, iOS, Windows, etc


Add shell completion value hints

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 90.11%. Comparing base (80452b9) to head (9ff544e).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #192      +/-   ##
==========================================
- Coverage   90.39%   90.11%   -0.29%
==========================================
  Files           9        9
  Lines         708      708
==========================================
- Hits          640      638       -2
- Misses         68       70       +2

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[Snyk] Upgrade quasar from 2.12.0 to 2.12.1

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade quasar from 2.12.0 to 2.12.1.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 1 version ahead of your current version.
The recommended version was released 22 days ago, on 2023-06-28.

Release notes

Package name: quasar

2.12.1 - 2023-06-28Features

feat(ui): Improve fontawesome-v6-pro.mjs (#15963)
feat(ui): Improve es.mjs language pack (#15964)

Fixes

fix(QMenu/QTooltip): position engine quirk when target is not ready yet -- resulting in jumping menus/tooltips #11247
fix(QEditor): Event names for link toolbar cause warning (#15826)
fix(QSelect): ignore most key commands when QSelect is not editable #15808 (#15819)
fix(QTree): children type safety #15991 (#15992)
fix(ui): Loosen rules param type in ValidationRule
fix(QFile): incomplete union type in QRejectedEntry['failedPropValidation'] (#15913)
fix(QUploader): exposed API is not reactive #14967
fix(BottomSheet): actions > avatar typing #15893
fix(ui): Mark QTable.pagination.sortBy as nullable (#15881)
fix(QEditor): Event names for link toolbar cause warning (#15826)

Donations
Quasar Framework is an open-source MIT-licensed project made possible due to the generous contributions by sponsors and backers. If you are interested in supporting this project, please consider the following:

Becoming a sponsor on Github
One-off donation via PayPal

2.12.0 - 2023-05-05

    from quasar GitHub release notes

Commit messages

Package name: quasar

2f1d4a1 chore(ui): Bump version
88edb52 fix(docs): vite 4 quirk on ssr manifest
bf7c70c chore(create-quasar): bump version
128de63 feat/fix(app-vite): account for special cases with vitePlugins
8cdeaa4 feat(docs): prepare docs for release
7b805a4 fix(app-vite/app-webpack): quasar.config > build > rawDefine typings
8becf45 fix(app-vite/app-webpack): quasar.config > build > env typings
e6add24 feat(app-webpack): ensure Electron runs in cjs mode (the only mode supported by Electron)
4835352 feat(app-vite): Electron support for projects with type=module
51d528c feat(app-webpack): improve quasar.config computeConfig management
597a9f7 feat(app-vite): improve quasar.config computeConfig management
f939a3e feat(app-vite): improve quasar.config file debugging
a8d0a57 chore(app-vite): small tweaks to quasar.config file
daf3fdb feat(app-webpack): ability to update appPkg
4796125 fix(app-webpack): update appPkg after changing it in an AE
c96ff25 fix(app-vite): update appPkg after changing it in an AE
c7e2b8d feat(app-webpack): normalize app-paths to app-vite standard
c580519 feat(app-vite): improve app-paths typing (make it more precise)
150e83a perf(app-vite): require() quasar.config file when output is in cjs format -- better perf as we can clean cache
5c6b1d4 feat(app-vite): re-read root package.json on each quasar.config file change
1e18d46 fix(app-vite): SSR support for both when project package.json has type=module and type=commonjs
91bce5a fix(app-vite): app-paths TS definition
7c33851 fix(app-vite): compilation of quasar.config in TS format
7ab57e7 chore(app-*): bump alpha versions

Compare

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

Build failed


passing 'seed=sequential' in LCA object not working

The following did not yield expected results:
excs = [*act.technosphere()]
sample_array = np.eye(len(excs))
indices = [(exc['input'], exc['output'], 'technosphere') for exc in excs]
_, fp = create_presample_matrix(sample_array, indices, 'technosphere')
lca = LCA({act:1}, presamples=[fp], seed='sequential')
lca.lci() #expect first exc to be==1, others 0: not the case
lca.lci() #expect second exc to be==1, others 0: not the case
...
Samples are in fact returned in random order.

Problem definition not correct: 'sequential' should have been passed in presample package, not LCA object.
In fact, a seed at the level of the LCA object is not relevant.
That said, there is still an issue with use of 'sequential' in LCA, but this is another issue.


featnext]: use context vars instead of global state in embedded iterator execution

Changed:

In the iterator embedded execution, both column_range and offset_provider global variables have been converted to context variables and fencil closures are now executed in isolated contexts with properly initialized values, to avoid using global state.

Opening as a reminder, instead of #896

I can clean up the code and provide a final version of this prototype without too much effort. @tehrengruber  do you also agree with using this implementation strategy to fix the problem?

Yes, given the model itself this is much better than the global variable.

I've requested a formal review from @tehrengruber because apparently @havogt can't approve it since he's the creator of the PR, but I think both of you should take a look.


[Accessibility]A11y_VSCode_ShowRunningTasks_ScreenReader:JAWS is not reading the placeholder text

Environment Details:
VSCode Version : 1.24.1
OS Version : Win10
Additional Details:
ScreenReader,,[JAWS18.0.5038],MAS1.3.1
Repro Steps:
Launch VS Code.
Navigate   to Tasks(Alt+T) menu option and select "Show Running Tasks" from   dropdown.
Observe what jaws reads.
Actual:
JAWS is not reading the placeholder text
Expected:
JAWS should read the placeholder text
Recommendations:
Refer below link which is repository of bug fixes code snippets:
https://microsoft.sharepoint.com/teams/msenable/mas/pages/browse-fixes.aspx
User Impact:
The user will not be able to know the placeholder text.
MAS Reference:
https://microsoft.sharepoint.com/:w:/r/teams/msenable/_layouts/15/WopiFrame.aspx?sourcedoc={54f28d1f-a2d1-4dcd-84e1-5c9b87e8aba4}
Does this issue occur when all extensions are disabled?: Yes
Attachment for Reference
show.pptx

Closing this issue as it's only observed in JAWS, and this isn't strictly needed to meet MAS C compliance.


Crash: Thread 1: EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)

It's the Example. How to fix it?

I get the same issue trying to run the example:
  Referenced from: /var/containers/Bundle/Application/33F19B9F-2E02-4D7E-991B-124C81699D53/CameraEngineExample.app/CameraEngineExample
  Reason: image not found```

+1

the second answer in this post solved it for me: https://stackoverflow.com/questions/24333981/ios-app-with-framework-crashed-on-device-dyld-library-not-loaded-xcode-6-beta

I had the same issue as of May 2018. Need to add CameraEngine.framework as an Embedded Binary into the example target.


npx and pnpx are not npm style

fixes #164
don't use the -- convention when running things with a *px command. -- is used with npm run scripts to pass arguments to the underlying command. *px runs a command directly, so the -- is interpreted by the interface, which leads to unexpected results.

@all-contributors please add @noslouch for code, bug

you're welcome thank you for the quick turn around!


Fixed GEMM conversion

I wonder if this logic should also be applied to the beta attribute.

I wonder if this logic should also be applied to the beta attribute.

In the future this should be handled by an optimization pass. This is a workaround for a specific model while the optimization passes are not yet in place.


Problem loading (loadAll) all entities of a graph

Hello,
we are trying to retrieve all entities stored in neo4j as specified in the Actor-Movie-Role example from the Neo4J OGM documentation:
here the three java classes:
Actor
@NodeEntity
public class Actor {

   private Long id;

   private Role playedIn;

   public Long getId() {
       return id;
   }

   public void setId(Long id) {
       this.id = id;
   }

   public Role getPlayedIn() {
       return playedIn;
   }

   public void setPlayedIn(Role playedIn) {
       this.playedIn = playedIn;
   }

}

Movie
@NodeEntity
public class Movie {

    private Long id;

    private String title;

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getTitle() {
        return title;
    }

    public void setTitle(String title) {
        this.title = title;
    }

}

Role
@RelationshipEntity
public class Role {

    @GraphId
    private Long relationshipId;

    @Property
    private String title;

    @StartNode
    private Actor actor;

    @EndNode
    private Movie movie;

    public Long getRelationshipId() {
        return relationshipId;
    }

    public void setRelationshipId(Long relationshipId) {
        this.relationshipId = relationshipId;
    }

    public String getTitle() {
        return title;
    }

    public void setTitle(String title) {
        this.title = title;
    }

    public Actor getActor() {
        return actor;
    }

    public void setActor(Actor actor) {
        this.actor = actor;
    }

    public Movie getMovie() {
        return movie;
    }

    public void setMovie(Movie movie) {
        this.movie = movie;
    }

}

Here the code used to insert nodes and relations in the db:
private void insertIntoDB() {

        Session session = Neo4jSessionFactory.getNeo4jSession();
        session.beginTransaction();

        Actor actor = new Actor();

        Movie movie = new Movie();
        movie.setTitle("Title");

        Role role = new Role();
        role.setActor(actor);
        role.setMovie(movie);
        role.setTitle("Role");

        actor.setPlayedIn(role);
        session.save(actor);
        session.getTransaction().commit();
}

And here the code used to retrieving all the nodes
private void loadAllEntities() {

        Session session = Neo4jSessionFactory.getNeo4jSession();
        session.beginTransaction();

        Collection<Actor> actors = session.loadAll(Actor.class, -1); /* Note -1 for deeply search*/

        for (Actor actor : actors) {
            System.out.println(actor.getPlayedIn());/* actor.getPlayedIn() return null */
        }

        session.getTransaction().commit();
 }

The println output for actor.getPlayedIn() is null instead of printing the pointed role object (with title "Role").
Note that querying the Neo4J DB using its web interface (localhost:7474) gives us the expected results

We are using the 2.2.4 version of Neo4J and the 1.1.1 version (from maven repository) of Neo4J OGM
Can anyone help us to understand where we are going wrong?
Thank you in advance,
Ciao Simone
ps: sorry for my very bad english!

Please specify the relationship type on the @RelationshipEntity like this:
@RelationshipEntity(type = "PLAYED_IN")
There's a bug open for this (https://jira.spring.io/browse/DATAGRAPH-615) - even so, the docs need updating (https://github.com/neo4j/neo4j-ogm/issues/33)


Add a Gitter chat badge to README.md

EV-LUG/www.evlug.com now has a Chat Room on Gitter
@hicksca has just created a chat room. You can visit it here: https://gitter.im/EV-LUG/www.evlug.com.
This pull-request adds this badge to your README.md:

If my aim is a little off, please let me know.
Happy chatting.
PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

Duplicate see #3