Support

I am trying to load data into Power BI using Python to_dataframe. I have been working with an enverus analytics support person and have multiple emails of discussion if needed, they referred me to create an issue here.
Simply, I can connect and load all [Values] EXCEPT 'EntryDate' for the rigs-through-time V3 dataset. I believe the to-dataframe code creates a header, then pulls data starting at [1] therefore skipping EntryData at [0]. Please respond back with a solution as soon as you can. I have beent trying to clear this up for over a month.
To be clear I am using 'EntryDate' as a filter so I know it is there, that is not the issue.
I cannot use the BI connection due to firewall issues, so our IT group wants me to pull data using the API.
Thank you for you help with this matter,
Jennifer
Screenshots.docx

I do not use Postman, so it would take a bit for me to confirm that. 99% sure it is not BI.  It is about as simple as it can get.
from enverus_developer_api import DeveloperAPIv3
v3 = DeveloperAPIv3(secret_key='#######')
rigs_time = v3.to_dataframe("RigsThroughTime", EntryDate='gt(2020-01-01)', deleteddate="null")
From: ewestling @.>
Sent: Wednesday, March 6, 2024 2:35 PM
To: enverus-ea/enverus-developer-api @.>
Cc: Jennifer Darby @.>; Author @.>
Subject: [EXTERNAL] Re: [enverus-ea/enverus-developer-api] Support (Issue #31)
External Sender: Use caution with links/attachments.
I am curious, do you see EntryDate in the response if you test with Postman?
https://app.enverus.com/direct/#/api/explorer/v3/rigsThroughTime says that EntryDate should be there but my login doesn't have access to query against the "rigs-through-time" dataset and I access the API directly instead of using something like PowerBI. If EntryDate is in the Postman response, then you'd want to start troubleshooting PowerBI directly.

Reply to this email directly, view it on GitHubhttps://github.com/enverus-ea/enverus-developer-api/issues/31#issuecomment-1981736307, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BGXZNWLC63PFCG7FM5S6B53YW54WNAVCNFSM6AAAAABEJXHQIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBRG4ZTMMZQG4.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.

Also, I can and have confirmed that query function works, to_dataframe function is the problem which is straight from Enverus Git Hub README.
Enverus README
@.***
Thank you,
Jennifer
From: ewestling @.>
Sent: Wednesday, March 6, 2024 2:35 PM
To: enverus-ea/enverus-developer-api @.>
Cc: Jennifer Darby @.>; Author @.>
Subject: [EXTERNAL] Re: [enverus-ea/enverus-developer-api] Support (Issue #31)
External Sender: Use caution with links/attachments.
I am curious, do you see EntryDate in the response if you test with Postman?
https://app.enverus.com/direct/#/api/explorer/v3/rigsThroughTime says that EntryDate should be there but my login doesn't have access to query against the "rigs-through-time" dataset and I access the API directly instead of using something like PowerBI. If EntryDate is in the Postman response, then you'd want to start troubleshooting PowerBI directly.

Reply to this email directly, view it on GitHubhttps://github.com/enverus-ea/enverus-developer-api/issues/31#issuecomment-1981736307, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BGXZNWLC63PFCG7FM5S6B53YW54WNAVCNFSM6AAAAABEJXHQIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBRG4ZTMMZQG4.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.

I understand and that does look like a simple script.  Troubleshooting-wise, I'd start by making sure that the data you're looking for comes across in an API response. The simplest way to do that is through Postman.
But to do it with the package that you're using in PowerBI, you can use Jupyter or Google Colab to run
!pip install enverus-developer-api
from enverus_developer_api import DeveloperAPIv3
v3 = DeveloperAPIv3(secret_key='#######')
rigs_time = v3.to_dataframe("RigsThroughTime", EntryDate='gt(2020-01-01)', deleteddate="null")
See if EntryDate comes back in the row that gets printed.  If it does, then you'll want to contact PowerBI/Microsoft for support on why all columns aren't visible since you know for sure it is in the returned data set.
Maybe there's an issue displaying the date value for some reason: https://stackoverflow.com/questions/51929420/python-script-in-power-bi-returns-date-as-microsoft-oledb-date

I will try Jupyter.
Get Outlook for iOShttps://aka.ms/o0ukef

From: ewestling @.>
Sent: Wednesday, March 6, 2024 3:29:29 PM
To: enverus-ea/enverus-developer-api @.>
Cc: Jennifer Darby @.>; Author @.>
Subject: [EXTERNAL] Re: [enverus-ea/enverus-developer-api] Support (Issue #31)
External Sender: Use caution with links/attachments.
I understand and that does look like a simple script. Troubleshooting-wise, I'd start by making sure that the data you're looking for comes across in an API response. The simplest way to do that is through Postman.
But to do it with the package that you're using in PowerBI, you can use Jupyter or Google Colab to run
!pip install enverus-developer-api
from enverus_developer_api import DeveloperAPIv3
v3 = DeveloperAPIv3(secret_key='#######')
rigs_time = v3.to_dataframe("RigsThroughTime", EntryDate='gt(2020-01-01)', deleteddate="null")
See if EntryDate comes back in the row that gets printed. If it does, then you'll want to contact PowerBI/Microsoft for support on why all columns aren't visible since you know for sure it is in the returned data set.
Maybe there's an issue displaying the date value for some reason: https://stackoverflow.com/questions/51929420/python-script-in-power-bi-returns-date-as-microsoft-oledb-date
—
Reply to this email directly, view it on GitHubhttps://github.com/enverus-ea/enverus-developer-api/issues/31#issuecomment-1981818938, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BGXZNWJAEGTA6RXYL6MF353YW6DDTAVCNFSM6AAAAABEJXHQIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBRHAYTQOJTHA.
You are receiving this because you authored the thread.Message ID: @.***>
This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.

Yes apologies on the delayed response. Pleas make the call like this. `rigs_time = v3.to_dataframe("rigs-through-time", EntryDate='gt(2020-01-01)', deleteddate="null")```

It works, but calls in every column but EntryDate still.  Can you please check the code for to_dataframe in GitHub? I truly think the code is started on the second [1] row (headers removed already) and skipping entrydate which I believe is in [0]. I know it is in the database, but that python code is not pulling it in. I did try to manually add EntryDate to the language below and get the error not found.
@.***

From: jra8927 @.>
Sent: Thursday, March 21, 2024 9:28 AM
To: enverus-ea/enverus-developer-api @.>
Cc: Jennifer Darby @.>; Author @.>
Subject: [EXTERNAL] Re: [enverus-ea/enverus-developer-api] Support (Issue #31)
External Sender: Use caution with links/attachments.
Yes apologies on the delayed response. Pleas make the call like this. `rigs_time = v3.to_dataframe("rigs-through-time", EntryDate='gt(2020-01-01)', deleteddate="null")```

Reply to this email directly, view it on GitHubhttps://github.com/enverus-ea/enverus-developer-api/issues/31#issuecomment-2012444488, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BGXZNWOGAAE5FE7KKFYEXT3YZLU6PAVCNFSM6AAAAABEJXHQIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJSGQ2DINBYHA.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.

@jddarb  please try this.
df = v3.to_dataframe(dataset, pagesize=10000, ENVBasin="SACRAMENTO")
df.reset_index(inplace=True)
df.head(10)

Jose,
Here are two results. I would have liked to use the Enverus GitHub Library to_dataframe function because it 'writes query results to a pandas Dataframe with properly set dtypes and index columns" (code line 266 https://github.com/enverus-ea/enverus-developer-api/blob/master/enverus_developer_api/init.py#L179)
To_dataframe code on GitHub should be corrected to allow for all fields to come across or be removed from the GitHub instructions for which the code you sent me came directly from and was tried. The only field with an issue is EntryDate.
However, I need to move in another direction to get what I need because I am really out of time. Since query function works, I am using Pandas DataFrame to get what I need - see shots below.
What you sent below with to_dataframe function from GitHub does not bring in EntryDate back to original issue.
from enverus_developer_api import DeveloperAPIv3
v3 = DeveloperAPIv3(secret_key='secret key')
df = v3.to_dataframe('rigs-through-time', pagesize=10000, ENVBasin="SACRAMENTO")
@.***
The query function from GitHub does.
from enverus_developer_api import DeveloperAPIv3
import pandas as pd
import csv
import json
v3 = DeveloperAPIv3(secret_key='secret key')
rtt_rows = v3.query('rigs-through-time', pagesize=10000, ENVBasin="SACRAMENTO")
rigs_time = pd.DataFrame(rtt_rows)
@.***
From: Jose Ramos @.>
Sent: Tuesday, March 26, 2024 10:30 AM
To: enverus-ea/enverus-developer-api @.>
Cc: Jennifer Darby @.>; Mention @.>
Subject: [EXTERNAL] Re: [enverus-ea/enverus-developer-api] Support (Issue #31)
External Sender: Use caution with links/attachments.
@jddarbhttps://github.com/jddarb please try this.
df = v3.to_dataframe(dataset, pagesize=10000, ENVBasin="SACRAMENTO")
df.reset_index(inplace=True)
df.head(10)

Reply to this email directly, view it on GitHubhttps://github.com/enverus-ea/enverus-developer-api/issues/31#issuecomment-2020750605, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BGXZNWO5ICTXMBP74PINBWLY2GH73AVCNFSM6AAAAABEJXHQIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRQG42TANRQGU.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.

Okay I am getting a limit error on multiple filter options. Please see if the code for to_dataframe can be corrected. It pulls for the right timeframe for me.
Thank you,
Jennifer
From: Jennifer Darby
Sent: Tuesday, March 26, 2024 12:51 PM
To: enverus-ea/enverus-developer-api @.***>
Subject: RE: [EXTERNAL] Re: [enverus-ea/enverus-developer-api] Support (Issue #31)
Jose,
Here are two results. I would have liked to use the Enverus GitHub Library to_dataframe function because it 'writes query results to a pandas Dataframe with properly set dtypes and index columns" (code line 266 https://github.com/enverus-ea/enverus-developer-api/blob/master/enverus_developer_api/init.py#L179)
To_dataframe code on GitHub should be corrected to allow for all fields to come across or be removed from the GitHub instructions for which the code you sent me came directly from and was tried. The only field with an issue is EntryDate.
However, I need to move in another direction to get what I need because I am really out of time. Since query function works, I am using Pandas DataFrame to get what I need - see shots below.
What you sent below with to_dataframe function from GitHub does not bring in EntryDate back to original issue.
from enverus_developer_api import DeveloperAPIv3
v3 = DeveloperAPIv3(secret_key='secret key')
df = v3.to_dataframe('rigs-through-time', pagesize=10000, ENVBasin="SACRAMENTO")
@.***
The query function from GitHub does.
from enverus_developer_api import DeveloperAPIv3
import pandas as pd
import csv
import json
v3 = DeveloperAPIv3(secret_key='secret key')
rtt_rows = v3.query('rigs-through-time', pagesize=10000, ENVBasin="SACRAMENTO")
rigs_time = pd.DataFrame(rtt_rows)
@.***
From: Jose Ramos<EMAIL_ADDRESS>Sent: Tuesday, March 26, 2024 10:30 AM
To: enverus-ea/enverus-developer-api<EMAIL_ADDRESS>Cc: Jennifer Darby<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: [EXTERNAL] Re: [enverus-ea/enverus-developer-api] Support (Issue #31)
External Sender: Use caution with links/attachments.
@jddarbhttps://github.com/jddarb please try this.
df = v3.to_dataframe(dataset, pagesize=10000, ENVBasin="SACRAMENTO")
df.reset_index(inplace=True)
df.head(10)

Reply to this email directly, view it on GitHubhttps://github.com/enverus-ea/enverus-developer-api/issues/31#issuecomment-2020750605, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BGXZNWO5ICTXMBP74PINBWLY2GH73AVCNFSM6AAAAABEJXHQIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRQG42TANRQGU.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.

@jddarb  I'm sorry can you clarify what you would like to do. I'm sorry I was a little confused by the message.

I am not sure which message you are referring to because when you reply, it doesn’t show history for me. There is definitely a problem with the to_dataframe code that needs to be fixed.
It does not pull in the entrydate, but I know it’s in the database and it works on the query code.
Get Outlook for iOShttps://aka.ms/o0ukef

From: Jose Ramos @.>
Sent: Thursday, March 28, 2024 8:53:24 AM
To: enverus-ea/enverus-developer-api @.>
Cc: Jennifer Darby @.>; Mention @.>
Subject: [EXTERNAL] Re: [enverus-ea/enverus-developer-api] Support (Issue #31)
External Sender: Use caution with links/attachments.
@jddarbhttps://github.com/jddarb I'm sorry can you clarify what you would like to do. I'm sorry I was a little confused by the message.
—
Reply to this email directly, view it on GitHubhttps://github.com/enverus-ea/enverus-developer-api/issues/31#issuecomment-2025240671, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BGXZNWLX75QAEPVSYHC7GZ3Y2QOFJAVCNFSM6AAAAABEJXHQIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRVGI2DANRXGE.
You are receiving this because you were mentioned.Message ID: @.***>
This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.

My apolologies, when I run the command, I see the entrydate

I am sure when you run the to_dataframe function you do not have EnteyDate. You do when run the query function.
Get Outlook for iOShttps://aka.ms/o0ukef

From: Jose Ramos @.>
Sent: Thursday, March 28, 2024 9:48:18 AM
To: enverus-ea/enverus-developer-api @.>
Cc: Jennifer Darby @.>; Mention @.>
Subject: [EXTERNAL] Re: [enverus-ea/enverus-developer-api] Support (Issue #31)
External Sender: Use caution with links/attachments.
MicrosoftTeams-image.3.png (view on web)https://github.com/enverus-ea/enverus-developer-api/assets/144828250/3a1c802f-dd01-42ed-bcc7-6a2a8a0a5fb7 My apolologies, when I run the command, I see the entrydate
—
Reply to this email directly, view it on GitHubhttps://github.com/enverus-ea/enverus-developer-api/issues/31#issuecomment-2025409953, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BGXZNWLDMWZCKBN646LX5YDY2QUTFAVCNFSM6AAAAABEJXHQIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRVGQYDSOJVGM.
You are receiving this because you were mentioned.Message ID: @.***>
This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.

@jddarb  my apologies, but if you look at the screenshot , on line 11, we are doing the to_dataframe call. can you please post your exact call youre making, obviously we dont need the secret key.

Thank you!!! It worked.  I am very sorry, I did not see the screenshot as those weren't coming in my emails, just your comments.
Jennifer
From: Jose Ramos @.>
Sent: Thursday, March 28, 2024 1:16 PM
To: enverus-ea/enverus-developer-api @.>
Cc: Jennifer Darby @.>; Mention @.>
Subject: [EXTERNAL] Re: [enverus-ea/enverus-developer-api] Support (Issue #31)
External Sender: Use caution with links/attachments.
@jddarbhttps://github.com/jddarb my apologies, but if you look at the screenshot , on line 11, we are doing the to_dataframe call. can you please post your exact call youre making, obviously we dont need the secret key.

Reply to this email directly, view it on GitHubhttps://github.com/enverus-ea/enverus-developer-api/issues/31#issuecomment-2025833773, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BGXZNWMHQN4CO4XHVMGFGPLY2RM7NAVCNFSM6AAAAABEJXHQIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRVHAZTGNZXGM.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.


No response from lita whois github.com

I've installed lira-whois but I'm not getting a response when I do lita whois github.com for example. Lita does not come with a response with FQDN. Other queries work fine though, I have lita deployed on Heroku, do you think there is an issue with the environment or is it something else?

You should check if whois gem works from there https://github.com/weppos/whois


Automated cherry pick of #5236: huawei & ctyun security group sync fix

Cherry pick of #5236 on release/3.1.
#5236: huawei & ctyun security group sync fix

/lgtm

/lgtm

/approve


Imporve setup_env.sh

Please provide a description of this PR:
Imporve setup_env.sh

The test result:
$ ./b.sh  x86_64
amd64
$  vi b.sh
$  ./b.sh  x86_64
amd64
$ ./b.sh  armv8
arm64
$ ./b.sh  arm6444
arm64
$  ./b.sh  aarch64x
arm64
$  ./b.sh  armv
arm
$  ./b.sh  armvdf
arm
$  ./b.sh  s390x
s390x
$  ./b.sh  ppc64le
ppc64le
$ ./b.sh  ppc64ledf
This system's architecture, ppc64ledf, isn't supported

b.sh:

Note that changes in the common directory are actually done in the istio/common-files repository and they are copied to the various repos.
See file header

OK, Thanks


Use require instead of assert in postgres_integration_common_test.go

Codecov Report

Merging #538 (a2f3088) into develop (be39e29) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #538   +/-   ##
========================================
  Coverage    33.18%   33.18%
========================================
  Files           24       24
  Lines         4842     4842
========================================
  Hits          1607     1607
  Misses        2998     2998
  Partials       237      237

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update be39e29...a2f3088. Read the comment docs.


Prevent attempt to create migrations table again if it already exists

There is an issue where if you are creating migrations table inside a schema, knex tries to create the table every time even though the table is already created. The real issue is that knex migrations do not handle custom schemas but this is a quick fix to handle those until knex rolls out better support for migrations in multiple schemas.

Seems legit +1


Namespace is always kyverno in exported metrics + adding rule to grafana dashboard filtering

I've started to use the policy reporter metrics for Kyverno and I faced some issues I would like to raise:

As I see, for each pass/fail row in the Grafana dashboard table, the namespace is always Kyverno, I would like it to present the namespace of the kind of report
Also, suggestion is to add another variable in default for the Grafana dashboard which is rule, Clusters will probably have more than 1 rule so filtering on rules can be very useful
Another thing regarding rule, I've noticed I see in the metrics 2 rules, one is "rule" and the second is "autogen-rule", I see that autogen rule is coming from ClusterPolicy status. Still, I'm not sure why it is being exported in the metrics as it is causing duplication.

You have to check the „exported_namespace“. Just namespace I’d set by Prometheus and relates to the namespaces of the policy reporter pod.
policy reporter does not process clusterpolicies. Autogen rules are created and applied by kyverno and produces policy report results. If you don’t want autogen rules or results it’s up to your policy configuration.

@fjogeleit Thank you very much i will try those


add category and tags pages

Pages and posts can have tags already and they are displayed with badges. They should be links to overview pages, that filter by tags.
bind routes:

/tag/:tag_name --> all page types
/category/:category_name
/blog/tag/:tag_name --> all posts with tag_name

Advanced tags search/selection should be possible, e. g. filter by tag_name1 and tag_name2, but omit posts with tag_name3...

tags pages use the improved search engine since 0.2.0


Error when using Python version 2 instead of Python3

Confuguration:
MacOS using default Python version (2.7.10) you will get the following error when using the program
$ python eliza.py
How do you do.  Please tell me your problem.
> Hello
Traceback (most recent call last):
  File "eliza.py", line 232, in <module>
    main()
  File "eliza.py", line 228, in main
    eliza.run()
  File "eliza.py", line 214, in run
    sent = input('> ')
  File "<string>", line 1, in <module>
NameError: name 'Hello' is not defined

If you use python3 from the command line the program will work as expected.
I'm not sure if you can make it work under Python2. Either way, a note in the readme file could be helpful for people getting started with your script.
Thanks!

Thanks for the report, looks like input meant something else in Python 2. I've pushed a fix, and added the usage readme as you suggested.


Obsolete code

Some obsolete code in app/modules/frontend/controllers/SetupController.php (line 51) :
         $action = new \Entity\Action;

        $action->name = 'administer all';

        $action->is_global = true;

        $this->em->persist($action);

(line 62) :
        $rha = new \Entity\RoleHasAction;

        $rha->fromArray($this->em, [

            'role' => $role,

            'action' => $action,

        ]);

        $this->em->persist($rha);

@andbou Good catch. Thank you for letting me know. :)


ftest is not defined

🐛 Bug Report
ftest is not defined, but it's documented here https://jestjs.io/docs/en/api#testonlyname-fn-timeout.
To Reproduce
ftest('example', () => {

});

 FAIL  src/__tests__/example.test.js
  ● Test suite failed to run

    ReferenceError: ftest is not defined

    > 1 | ftest('example', () => {});
        | ^
      2 |

      at Object.<anonymous> (src/__tests__/example.test.js:1:1)

Expected behavior
ftest should works exactly like fit.
envinfo
 "ts-jest": "^26.0.0"

  System:
    OS: macOS 10.15.4
    CPU: (16) x64 Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
  Binaries:
    Node: 10.18.1 - ~/.nvm/versions/node/v10.18.1/bin/node
    Yarn: 1.21.1 - ~/.nvm/versions/node/v10.18.1/bin/yarn
    npm: 6.13.4 - ~/.nvm/versions/node/v10.18.1/bin/npm

Thanks for the report. Looks like the docs change came from here https://github.com/facebook/jest/pull/9802
I don't think we should be adding these globals when we have the much clearer test.only / it.only, rather we should undo the change that introduced things that do not exist into the docs :sweat_smile:
Wanna send a PR for it?
(It is a bit annoying that there is xtest but no ftest, but I think the better solution would be deprecating xtest)


Training InSPyReNet_SwinB.yaml on large dataset

Hi @plemeri
I have a custom dataset of 200K+ images containing real world images of cars , products , humans, animals etc , I wanted to know how many epoch should I train for ? I am using InSPyReNet_SwinB.yaml
I would like to inform that my model still in training and it has reached currently 80 epochs , but I see that the train loss was decreasing gradually for the first 4-5 epochs , then from 5 to 80 epoch the train loss is fluctuating between 0.2 to 0.5 on each iteration .
Also I notice that few images are getting better after a few epochs and then again a few ecochs those images are getting worse. I am unable to see any consistent improvement in training .
Can you please guide me ?
Looking forward to your reply.

Hello,
I had a few questions about training our model with custom dataset, and they are mostly larger than our method. I'm not sure about the reason why there are some problems there. Maybe I think you need to check if there's any problem with your dataset. Are the annotations are consistent? Are those images' size similar to each other? If so, you need to check the dataset and make sure the size and annotation criteria is consistent. Large dataset often worsen the result if it is not constructed carefully.
I'm currently working on using composite dataset with different image sizes. If you are interested in large dataset, please stay tuned.
Thank you.

Hi @plemeri
Thank you for your quick reply. I would like to inform you that our dataset is having same size as the original image and their corresponding masks, Also in your training code I notice that you are already doing a static resize which I think is converting the images to 380x380 so I don't know if the problem is occurring with our dataset ?
Also can you please help me understand what do you mean by  checking if annotation criteria is consistent ? and if you can kindly recommend any method to follow while constructing the large dataset ?
Looking forward to hearing back from you.

Sorry for the late reply. I was trying on different conditions to reproduce such problem, but I couldn't find any trouble with training with large dataset. If you don't mind, could you share the dataset that you've in trouble with? I would like to reproduce the issue and fix it as soon as possible.

Hi @plemeri
Thank you for your reply.
Can you kindly share me your email address so I can share the dataset with you.
Thanks

<EMAIL_ADDRESS>Thanks

Hi @plemeri
I would also like too draw your attention to one more problem which I recently came across , When I inference the inspyrenet_swinb model which is trained on my custom dataset I see that in many images the edges are having background image in it. Can you please tell me why this is occuring?
Thanks and looking forward to hearing back from you.
P.S see attached reference output and input image for a better understanding of the issue.
Problem_compressed.pdf

This can be easily removed by applying morphological operation called erosion.


3D DenseNet

Hello and sorry to bother you beforehand,
I am currently conducting my master thesis project and I am trying to implement a 3D DenseNet-121 with knee MRIs as input data. While I was searching on how to implement a 3D version of the DenseNet I came across your repository and tried to change it for my application.
I have some issues regarding my try and I didn't know where else to ask about it and again I am sorry if I am completely of topic asking them here.
Firstly, my input shapes are (250,320,18,1) and when I give them as input to the 3D DenseNet I developed with stride_size=1 for my Conv_block and pooling_size=(2,2,2) and strides=(2,2,1) for my AveragePooling3D layer in the transition block, the model is constructed properly with the specific input_size, while when I am trying to load a DenseNet121 from classification_models_3d.tfkeras classifiers I am unable to construct it with input_shape(250,320,18,1), stride_size=1 and kernel_size=2. It gives as an error "Negative dimension size... for node pool4_pool/AvgPool3D". Is there a way to specifically define the strides for AvgPool3D layer in the transition block?
And secondly, I was thinking to load the 3D weights to my 3D DenseNet 121, is there a folder in your repository where I can find your pre-trained weights on imagenet??
Again thank you for having this repository publicly available and sorry if I am completely of topic asking such things here.
I look forward for you answer,
Kind regards,
Anastasis

Hello. Try this:
        modelPoint, preprocess_input = Classifiers.get(type)
        model = modelPoint(
            input_shape=(250, 320, 18, 1),
            include_top=include_top,
            stride_size=(
                (2, 2, 2),
                (2, 2, 2),
                (2, 2, 2),
                (2, 2, 2),
                (2, 2, 1),
            ),
            weights=use_weights
        )
        print(model.summary())
        print(get_model_memory_usage(1, model), 'GB')

Your error was caused by last dimension 18. By default there are 5 poolings of size 2. You need at least 32 for this dimension.
In my example I didn't do poling on last stage.

Thank you for your answer. I will try and implement the changes you suggested.
Unfortunately my MRI input data do not have more than 18 slices in depth that is why I tried to change the pooling stride size.
Thanks again for your reply!!!

Hello, I have a related question about using imagenet weights on a 1 channel 3D input images.
Can I know what method can be used to adapt the weights to the 1 channel input MRI image (110,110,110,1)?

Hola, tengo una pregunta relacionada con el uso de pesos de imagenet en imágenes de entrada 3D de 1 canal. ¿Puedo saber qué método se puede usar para adaptar los pesos a la imagen de resonancia magnética de entrada de 1 canal (110,110,110,1)?

@theresahiu Has conseguido resolverlo? tengo el mismo problema


location of type-names

The type-names  ("Ban", "Warn", "Mute ...") are not listed in the lang/en_US.utf8.php and I can find not in the web-files. Where I can find it? :)

https://github.com/ruany/litebans-php/blob/master/lang/en_US.utf8.php#L19
        $array["generic.ban"] = "Ban";
        $array["generic.mute"] = "Mute";
        $array["generic.warn"] = "Warning";
        $array["generic.kick"] = "Kick";

I mean this:

        $array["generic.ban"] = "Bann";

Fixed https://github.com/ruany/litebans-php/commit/9218cdc430240ae9049736ffe982f6344331c0f7

Thanks. :)


TiKV Fails to Compile with Newer Toolchains (Clang 17+, GCC 13+)

Bug Report

What version of TiKV are you using?

master
What operating system and CPU are you using?

Steps to reproduce

Compile TiKV with Clang 17, Clang18, GCC 14. Didn't try GCC13, but it seems the breaking change was made since GCC13.
What did you expect?
What did happened?
TiKV -> grpc-rs (0.10.x) -> gRPC(1.4x) -> absl  which uses uint8_t without #include <stdint.h>
We need to upgrade grpcio version, and, as a prerequisite, we need to upgrade grpcio in kvproto.

FYI, if upgrading the grpc-rs to the latest version or versions >= 0.13.x, it will bring performance regression issues, referring to https://github.com/tikv/grpc-rs/issues/643.


Add examples of set_index and provide more clear error message

[x] Closes #3176 by adding examples of set_index and raising an error with a more clear error message.
[x] Added unit test for error message.
[x] Passes black .  and flake8.
[x] Documented in whats-new.rst.

This is great! Thanks @gwgundersen !


'disabled' applied only when true, 'required' applied only when true and not disabled

I am not sure for the required flag, as I was not able to test.
I had to upgrade esbuild because of https://github.com/evanw/esbuild/issues/1711#issuecomment-969655949

@ymarcon thanks so much. First look looks all good. Will merge and release later this week.
Is it ok only on next ?

Could you make a pre-release instead of (or in addition of) the @next snapshot ? What's your plan with releasing 1.0.0?

@ymarcon closed in favour of cherry pick:
https://github.com/CyCraft/blitzar/commits/v1.0.0
will release just as the latest version on next for now.
I plan to release 1.0.0 ideally in 2021 : D
Only couple things left to do:

[ ] finalise rewriting BlitzTable
[ ] clean up the repo
[ ] port over docs to VitePress

I'm working on BlitzTable during December.


support for invalid line endings

From @durin42 on May 4, 2012 21:26
I've seen bad servers in the wild that respond with \n or \r (yes, \r) line endings instead of \r\n. It'd be great to be able to test that.
Copied from original issue: mitmproxy/pathod#3

Closing, as pathod and pathoc command line tools and their libraries inside mitmproxy are deprecated. See #4273.


Wrong default value behavior in disaggregated mode

Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
tiup playground:nightly nightly --mode=tidb-cse

mysql> create table c(a int);
Query OK, 0 rows affected (0.06 sec)

mysql> insert into c values (1);
Query OK, 1 row affected (0.00 sec)

mysql> alter table c set tiflash replica 1;
Query OK, 0 rows affected (0.04 sec)

mysql> select * from information_schema.tiflash_replica;
+--------------+------------+----------+---------------+-----------------+-----------+----------+
| TABLE_SCHEMA | TABLE_NAME | TABLE_ID | REPLICA_COUNT | LOCATION_LABELS | AVAILABLE | PROGRESS |
+--------------+------------+----------+---------------+-----------------+-----------+----------+
| test         | c          |      104 |             1 |                 |         1 |        1 |
+--------------+------------+----------+---------------+-----------------+-----------+----------+
1 row in set (0.00 sec)

mysql> alter table c add column test_c1 int default 0;
Query OK, 0 rows affected (0.10 sec)

mysql> set @@tidb_isolation_read_engines="tidb,tiflash";
Query OK, 0 rows affected (0.00 sec)

mysql> select count(*) from c where test_c1 = 0;
+----------+
| count(*) |
+----------+
|        0 |
+----------+
1 row in set (0.04 sec)

mysql> select count(*) from c where test_c1 is null;
+----------+
| count(*) |
+----------+
|        1 |
+----------+
1 row in set (0.01 sec)

mysql> set @@tidb_isolation_read_engines="tidb,tikv";
Query OK, 0 rows affected (0.00 sec)

mysql> select count(*) from c where test_c1 = 0;
+----------+
| count(*) |
+----------+
|        1 |
+----------+
1 row in set (0.01 sec)

mysql> select count(*) from c where test_c1 is null;
+----------+
| count(*) |
+----------+
|        0 |
+----------+
1 row in set (0.01 sec)

2. What did you expect to see? (Required)
TiFlash and TiKV should produce same result.
3. What did you see instead (Required)
4. What is your TiFlash version? (Required)
nightly

use test;
CREATE TABLE `relationships` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `id2` int(11) NOT NULL,
  PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci AUTO_INCREMENT=90001;
insert into relationships (id2) values (1), (2), (3);
insert into relationships (id2) select id2 from relationships;
insert into relationships (id2) select id2 from relationships;
insert into relationships (id2) select id2 from relationships;
insert into relationships (id2) select id2 from relationships;
insert into relationships (id2) select id2 from relationships;
insert into relationships (id2) select id2 from relationships;
alter table relationships set tiflash replica 1;
alter table relationships add column weight int default '0';
select * from information_schema.tiflash_replica where TABLE_NAME = 'relationships';
set @@tidb_isolation_read_engines='tiflash'; select count(*) from relationships where weight = 0;


Make compile with pioarduino 3.0.5 (ESP-IDF 5.1.4)

Problem Description
Hi PSchatzmann,
due to lack of supporting Arduino 3 on PlatformIO i switched to pioarduino. This works perfectly fine!
Framework versions: Arduino 3.0.5 , ESP-IDF 5.1.4
Compiling ESP32-A2DP i get some errors:
.pio/libdeps/lolin_d32_pro_sdmmc_pe/ESP32-A2DP/src/BluetoothA2DPSink.cpp:285:41: error: 'portTickType' was not declared in this scope   285 |                                        (portTickType)portMAX_DELAY)) {       |                                         ^~~~~~~~~~~~
and
C:/Users/dcars/.platformio/packages/framework-espidf@src-29cf07cc82e49157db19869ff2d1be3f/components/freertos/FreeRTOS-Kernel/portable/xtensa/include/freertos/portmacro.h:125:36: error: expected primary-expression before ')' token   125 | #define portMAX_DELAY ( TickType_t ) 0xffffffffUL       |
I changed this line
https://github.com/pschatzmann/ESP32-A2DP/blob/3624fe3f23c8a4d181f891910f5b62dc87a51105/src/BluetoothA2DPCommon.h#L94
to
#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 1 , 4)
and everything compiles. Can you fix it?
Thanks & best
Dirk
Device Description
Lolin D32 pro
Sketch
demo sketch

Other Steps to Reproduce
No response
Provide your Version of the EP32 Arduino Core (or the IDF Version)
Arduino 3.0.5, ESP-IDF 5.1.4
I have checked existing issues, discussions and online documentation

[X] I confirm I have checked existing issues, discussions and online documentation

can you share some minimal code to reproduce the issue ?


Not able to find product through image.

Bug report
Title
Not able to find product through image.
Issue Description
Not able to find product through image.
Preconditions
1. framework Version.-v1.2.0-BETA1

Expected result
If product image is uploaded then product  should display on search it it is available.
Actual result
Product is available but doesn't display

@jyoti-webkul
The image search feature will detect the object from the given image and provide you all the analyzed keywords for that object.
In your example; you have uploaded the Headset then TensorFlow will detect all the objects in this image "mike" "radio" and giving you all the keyword, which will helps customer to easily search the product by the related image.


GPT-2 doesn't always have an lm_head

The GPT2 ggml models don't always have an lm_head, but llm assumes it does:
https://github.com/rustformers/llm/blob/9a222690cd0a9e3322bb6a926d54e90d08fb2c0f/crates/models/gpt2/src/lib.rs#L62
See ggml example repo where they mention lm_head is optional: https://github.com/ggerganov/ggml/blob/ee1b3727e60403012dd2b57d35b60558f4db66d8/examples/gpt-2/main.cpp#L364

I tried replacing it with
let lm_head = tl.load("model/lm_head").or_else(|_| tl.load("model/wte"))?;

But then I ran into a more ggml errors and had to move on
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 252199424, available 250733056)

Yeah i also noticed that, we probably have to share the wte tensor as the context size is calculated via the combined size of all tensors.


itemsetoption.bmd

Hello, thank you for your help. I found another problem. After decryption of itemsetoption.bmd, the  attribute has a negative value， +-1

Hi,
It's not a bug. -1 = (int)0xFF = (int)0xFFFF = (int)0xFFFFFFFF
-1 means no value assigned to the attribute...


Unable to place images in collaboration mode

When an image is selected in the official excalidraw whiteboard with the live collaboration mode enabled, then the preview of the image is shown but as soon as the user clicks the mouse button to place the image, it disappears completely or it is shown an empty image square like this:

The issue can be reproduced with different browsers (Firefox, Chrome) and OS (Linux, Mac, Windows). The issue seems not to appear when the live collaboration mode is not enabled. It does not matter if someone has joined the live collaboration. Just enabling it is enough to see the issue.
In the console tab of the browser no new issue or log entry appears when placing an image which fails:

@SymFlorian do you mind sharing a recording so its more clear and easy for us to reproduce?

@ad1992 I mean the live collaboration feature to see the drawing of other users:

As soon as you start the session, the placing of images

is not working (most times)

Yep, I got that, but what I meant is if you could record a video with the same exact steps, that would be great (of course unless it's confidential) as it works for me when I try to add images in collaboration mode as well.

Sorry, totally misunderstood you ;)
Here a video where you can see the issue:
https://user-images.githubusercontent.com/58416661/166419231-2927ab2b-f2ba-4804-b36f-83df9540ab75.mov

Can you post the image here if not private? I can't reproduce with any image of mine (huge or small) on Windows or MacOS.
Also, can you check the excalidraw version? (right-click on canvas → stats for nerds → click on version to copy)

It seems as if the issue is not 100% consistent, sometimes its better, sometimes worse. On some systems I can see that the image can be placed in the first attempt, but when placing the same image a second time (regardless of deleting it before), then it does not work.
It does not seem to have any correlation to the image used, but nevertheless I can put here two images where the issue appears.
So the repro steps could be:

Start a collaboration session
Place an image through the image button (9)
Place the same image again through the image button (9) --> Cannot be placed

ExampleImages.zip

Thanks for the info.
Does drag & drop (instead of using the file picker) work fine?
The strangest thing is you can repro only when collaborating. Are you sure you can't repro in solo?

With drag & drop it works fine, no issues at all. In the solo session it also works fine. Its only the file picker in the collaboration session wich causes the issue.

After looking through the code I think I found the issue!
It seems to be a that pendingImageElement that is used when inserting images and the corresponding element in the scene are two different instances of the same element. It appears this is caused by src/excalidraw-app/collab/Portal.tsx#L120 (It's even explained in the comment above!)
When I replace the line with mutateElement(element, { status: "saved" }) I can't reproduce the issue anymore.

Fix here https://github.com/excalidraw/excalidraw/pull/5322 (preview: https://excalidraw-git-fixpendingimageelementstalereference-excalidraw.vercel.app/). Lemme know if the fix works for you.


Try to train CenterPoint on kitti

I tried to train CenterPoint on kitti， when finished training, the val result is far from expected.
Use PillarFeatureNet as feature extracter, second as backbone and second_fpn as neck, train on 3-class kitti, cyc lr and 80 epochs, eval by strict 3D iou.
Got mAP of Car is 69.17, Ped is 30.68, Cyc is 53.53).
Are there problems with my config file？

base = [
'../base/datasets/kitti-3class.py',
'../base/models/centerpoint_02pillar_second_secfpn_kitti.py',
'../base/schedules/cyclic_40e.py',
'../base/default_runtime.py'
]
point_cloud_range = [0, -39.68, -3, 69.12, 39.68, 1]
dataset settings
data_root = 'data/kitti/'
class_names = ['Pedestrian', 'Cyclist', 'Car']
PointPillars adopted a different sampling strategies among classes
db_sampler = dict(
data_root=data_root,
info_path=data_root + 'kitti_dbinfos_train.pkl',
rate=1.0,
prepare=dict(
filter_by_difficulty=[-1],
filter_by_min_points=dict(Car=5, Pedestrian=10, Cyclist=10)),
classes=class_names,
sample_groups=dict(Car=15, Pedestrian=10, Cyclist=10))
model = dict(
pts_voxel_layer=dict(point_cloud_range=point_cloud_range),
pts_voxel_encoder=dict(point_cloud_range=point_cloud_range),
pts_bbox_head=dict(bbox_coder=dict(pc_range=point_cloud_range[:2])),
# model training and testing settings
train_cfg=dict(pts=dict(point_cloud_range=point_cloud_range)),
test_cfg=dict(pts=dict(pc_range=point_cloud_range[:2])))
train_pipeline = [
dict(type='LoadPointsFromFile', coord_type='LIDAR', load_dim=4, use_dim=4),
dict(type='LoadAnnotations3D', with_bbox_3d=True, with_label_3d=True),
dict(type='ObjectSample', db_sampler=db_sampler),
dict(
type='ObjectNoise',
num_try=100,
translation_std=[0.25, 0.25, 0.25],
global_rot_range=[0.0, 0.0],
rot_range=[-0.15707963267, 0.15707963267]),
dict(type='RandomFlip3D', flip_ratio_bev_horizontal=0.5),
dict(
type='GlobalRotScaleTrans',
rot_range=[-0.78539816, 0.78539816],
scale_ratio_range=[0.95, 1.05]),
dict(type='PointsRangeFilter', point_cloud_range=point_cloud_range),
dict(type='ObjectRangeFilter', point_cloud_range=point_cloud_range),
dict(type='PointShuffle'),
dict(type='DefaultFormatBundle3D', class_names=class_names),
dict(type='Collect3D', keys=['points', 'gt_bboxes_3d', 'gt_labels_3d'])
]
test_pipeline = [
dict(type='LoadPointsFromFile', coord_type='LIDAR', load_dim=4, use_dim=4),
dict(
type='MultiScaleFlipAug3D',
img_scale=(1333, 800),
pts_scale_ratio=1,
flip=False,
transforms=[
dict(
type='GlobalRotScaleTrans',
rot_range=[0, 0],
scale_ratio_range=[1., 1.],
translation_std=[0, 0, 0]),
dict(type='RandomFlip3D'),
dict(
type='PointsRangeFilter', point_cloud_range=point_cloud_range),
dict(
type='DefaultFormatBundle3D',
class_names=class_names,
with_label=False),
dict(type='Collect3D', keys=['points'])
])
]
data = dict(
train=dict(dataset=dict(pipeline=train_pipeline, classes=class_names)),
val=dict(pipeline=test_pipeline, classes=class_names),
test=dict(pipeline=test_pipeline, classes=class_names))
lr = 0.001
optimizer = dict(lr=lr)
optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2))
evaluation = dict(interval=5)
total_epochs = 80

'centerpoint_02pillar_second_secfpn_kitti.py' in my config file is modified from centerpoint_02pillar_second_secfpn_nus.py，only changed some params (such as class names) to adapt KITTI， and I didn't change the architecture.

Hi, we are also working on training CenterPoint on KITTI.
May I ask what the voxel_size that you set? Since we met the error when training the centerpoint. The error is shown as below. Did you meet the same problem before?
Thanks in advance.
_2021-04-27 13:54:02,527 - mmdet - INFO - Start running, host: root@b181bde9e289, work_dir: /content/gdrive/My Drive/Colab_Notebooks/mmdet3d-colab/mmdetection3d/work_dirs/centerpoint_nusc2kitti_02pillar
2021-04-27 13:54:02,527 - mmdet - INFO - workflow: [('train', 1)], max: 20 epochs
2021-04-27 13:54:02.709169: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
max_voxels:  30000
max_voxels:  30000
voxel_size:  [0.2, 0.2, 8]
coors_range:  [0, -39.68, -3, 69.12, 39.68, 1]
voxels_size:  torch.Size([30000, 20, 4])
coors:  torch.Size([30000, 3])
num_points_per_voxel:  torch.Size([30000])
max_points:  20
Traceback (most recent call last):
File "tools/train.py", line 212, in
main()
File "tools/train.py", line 208, in main
meta=meta)
File "/content/gdrive/My Drive/Colab_Notebooks/mmdet3d-colab/mmdetection/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/usr/local/lib/python3.7/dist-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
epoch_runner(data_loaders[i], **kwargs)
File "/usr/local/lib/python3.7/dist-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
self.run_iter(data_batch, train_mode=True, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/mmcv/runner/epoch_based_runner.py", line 30, in run_iter
**kwargs)
File "/usr/local/lib/python3.7/dist-packages/mmcv/parallel/data_parallel.py", line 67, in train_step
return self.module.train_step(*inputs[0], **kwargs[0])
File "/content/gdrive/My Drive/Colab_Notebooks/mmdet3d-colab/mmdetection/mmdet/models/detectors/base.py", line 247, in train_step
losses = self(**data)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/mmcv/runner/fp16_utils.py", line 84, in new_func
return old_func(*args, **kwargs)
File "/content/gdrive/My Drive/Colab_Notebooks/mmdet3d-colab/mmdetection3d/mmdet3d/models/detectors/base.py", line 58, in forward
return self.forward_train(**kwargs)
File "/content/gdrive/My Drive/Colab_Notebooks/mmdet3d-colab/mmdetection3d/mmdet3d/models/detectors/mvx_two_stage.py", line 273, in forward_train
points, img=img, img_metas=img_metas)
File "/content/gdrive/My Drive/Colab_Notebooks/mmdet3d-colab/mmdetection3d/mmdet3d/models/detectors/mvx_two_stage.py", line 207, in extract_feat
pts_feats = self.extract_pts_feat(points, img_feats, img_metas)
File "/content/gdrive/My Drive/Colab_Notebooks/mmdet3d-colab/mmdetection3d/mmdet3d/models/detectors/centerpoint.py", line 38, in extract_pts_feat
voxels, num_points, coors = self.voxelize(pts)
File "/usr/local/lib/python3.7/dist-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/mmcv/runner/fp16_utils.py", line 164, in new_func
return old_func(*args, **kwargs)
File "/content/gdrive/My Drive/Colab_Notebooks/mmdet3d-colab/mmdetection3d/mmdet3d/models/detectors/mvx_two_stage.py", line 224, in voxelize
res_voxels, res_coors, res_num_points = self.pts_voxel_layer(res)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 727, in call_impl
result = self.forward(*input, **kwargs)
File "/content/gdrive/My Drive/Colab_Notebooks/mmdet3d-colab/mmdetection3d/mmdet3d/ops/voxel/voxelize.py", line 123, in forward
self.max_num_points, max_voxels)
File "/content/gdrive/My Drive/Colab_Notebooks/mmdet3d-colab/mmdetection3d/mmdet3d/ops/voxel/voxelize.py", line 63, in forward
voxel_num = hard_voxelize(points, voxels, coors, num_points_per_voxel, voxel_size,coors_range, max_points, max_voxels, 3)
RuntimeError: CUDA error: no kernel image is available for execution on the device

The voxel size in my config file is [0.16, 0.16, 4]， point cloud range is [0, -40.96, -1, 81.92, 40.96, 3],  Pillar feature net output shape and grid size is [512, 512]. I'm sorry that I didn't meet this error, may be caused by pytorch version according to https://blog.csdn.net/yyhaohaoxuexi/article/details/107460836  @ZhangYu1ing

One implementation here https://github.com/tianweiy/CenterPoint-KITTI
I played with mmdet3d in the past but I don't get a clean codebase to release at the moment. Additionally, PointPillars just doesn't work too well on KITTI so you may consider playing with the VoxelNet backbone.

RuntimeError: CUDA error: no kernel image is available for execution on the device

That means that the hard_voxelize code is not compiled correctly for your GPU.

Thanks very much for reply and sharing the code ! @tianweiy  Another question is what is the mAP difference roughly between pillars and voxelnet on KITTI according to your experiment result?

I only played with PointPillars in the Det3D repo. I am not able to get upper than 75 mAP. I will add model zoo to the repo soon.

@guojingming @Tai-Wang Thanks for giving the advice. We noticed that the problem only be raised up in Google Colab. But it could work on PC.
Thanks a lot for your sharing.

Hi, I am trying to implement your provided code. I created the KITTI data following the instruction in OpenPCDet. However, there is a bug when I run the tools/train.py, which is shown below. Do you have any ideas? Thanks a lot.
2021-04-28_ 15:44:55,801   INFO  Start training tools/cfgs/kitti_models/centerpoint(default)
epochs:   0%|                                                                           | 0/80 [00:00<?, ?it/s]
Traceback (most recent call last):                                                     | 0/464 [00:00<?, ?it/s]
File "tools/train.py", line 198, in
main()
File "tools/train.py", line 170, in main
merge_all_iters_to_one_epoch=args.merge_all_iters_to_one_epoch
File "/home/xymbiotec/CenterPoint-KITTI-main/tools/train_utils/train_utils.py", line 93, in train_model
dataloader_iter=dataloader_iter
File "/home/xymbiotec/CenterPoint-KITTI-main/tools/train_utils/train_utils.py", line 19, in train_one_epoch
batch = next(dataloader_iter)
File "/home/xymbiotec/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 435, in next
data = self._next_data()
File "/home/xymbiotec/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1085, in _next_data
return self._process_data(data)
File "/home/xymbiotec/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data
data.reraise()
File "/home/xymbiotec/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/_utils.py", line 428, in reraise
raise self.exc_type(msg)
KeyError: Caught KeyError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/xymbiotec/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop
data = fetcher.fetch(index)
File "/home/xymbiotec/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/xymbiotec/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/xymbiotec/CenterPoint-KITTI-main/pcdet/datasets/kitti/kitti_dataset.py", line 348, in getitem
sample_idx = info['point_cloud']['lidar_idx']
KeyError: 'lidar_idx'

Maybe, there are some version comparability issue. I will check today

Maybe, there are some version compatibility issue. I will check today

No worries, the problem has been solved. The problem is related to the path of Kitti Dataset. I modified the DATA_PATH in tools/cfgs/dataset_configs/kitti_dataset.yaml
I changed the DATA_PATH='/data/kitti', instead of DATA_PATH='../data/kitti'
Then, it could successfully work.
This is my attempt to solve this problem. I am not sure if there are another issues also lead to this problem?
Thanks for your help.


Easy remote debug

http://www.godotengine.org/forum/viewtopic.php?f=12&t=1523
is this issue resolved?

Yup
On Aug 28, 2015 5:57 AM, "volzhs"<EMAIL_ADDRESS>wrote:

http://www.godotengine.org/forum/viewtopic.php?f=12&t=1523
is this issue resolved?
—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/2401.

I tried to debug on android devices.
So, I checked on "Deploy Remote Debug" option and run on android.
and adb rogcat says.
08-28 23:16:40.565: I/godot(8274): ERROR: _resolve_hostname: gethostbyname failed!
08-28 23:16:40.565: I/godot(8274):    At: drivers/unix/ip_unix.cpp:69.
08-28 23:16:40.566: I/godot(8274): ERROR: connect: Condition ' p_host.host == 0 ' is true. returned: ERR_INVALID_PARAMETER
08-28 23:16:40.566: I/godot(8274):    At: drivers/unix/stream_peer_tcp_posix.cpp:138.
08-28 23:16:40.887: W/ActivityManager(511): Activity pause timeout for ActivityRecord{de61d2 u0 com.macaron.twinpuzzle/com.android.godot.Godot t3378}
08-28 23:16:40.921: E/cpu_temp(6690): MSG_ON_WATCH_CURRENT
08-28 23:16:41.200: D/WifiService(511): New client listening to asynchronous messages
08-28 23:16:41.566: I/godot(8274): Remote Debugger: Connection failed with status: 0'', retrying in 1 sec.
08-28 23:16:42.567: I/godot(8274): Remote Debugger: Connection failed with status: 0'', retrying in 1 sec.
08-28 23:16:43.568: I/godot(8274): Remote Debugger: Connection failed with status: 0'', retrying in 1 sec.
08-28 23:16:43.568: I/godot(8274): Remote Debugger: Unable to connect
08-28 23:16:43.568: I/godot(8274): pre-removing print handler...
08-28 23:16:43.568: I/godot(8274): removing print handler...
08-28 23:16:43.568: I/godot(8274): found
08-28 23:16:43.568: I/godot(8274): print hanlder list is 0x0
08-28 23:16:43.577: I/godot(8274): has res dir:
08-28 23:16:43.579: I/godot(8274): exists engine cfg? 0
08-28 23:16:43.583: I/godot(8274): loaded engine.cfg
08-28 23:16:43.588: I/godot(8274): *****SETUP OK
08-28 23:16:43.654: I/godot(8274): **START

What should I do to debug on android?
I use c2e2f2e0aebf6342e6f18ae5d67b6a825590675a commit (Aug 8, 2015) for now.

check a few things, mainly that

your export templates are as recent as your editor
both your device and your PC are in the same intranet and can see
eachother (remote debug goes via wifi, not via usb cable)

if both are met, it should work
On Fri, Aug 28, 2015 at 11:21 AM, volzhs<EMAIL_ADDRESS>wrote:

I tried to debug on android devices.
So, I checked on "Deploy Remote Debug" option and run on android.
and adb rogcat says.
08-28 23:16:40.565: I/godot(8274): �[1;31;40mERROR: _resolve_hostname: �[1;37;40mgethostbyname failed!
08-28 23:16:40.565: I/godot(8274): �[0;31;40m   At: drivers/unix/ip_unix.cpp:69.�[0;0;37m
08-28 23:16:40.566: I/godot(8274): �[1;31;40mERROR: connect: �[1;37;40mCondition ' p_host.host == 0 ' is true. returned: ERR_INVALID_PARAMETER
08-28 23:16:40.566: I/godot(8274): �[0;31;40m   At: drivers/unix/stream_peer_tcp_posix.cpp:138.�[0;0;37m
08-28 23:16:40.887: W/ActivityManager(511): Activity pause timeout for ActivityRecord{de61d2 u0 com.macaron.twinpuzzle/com.android.godot.Godot t3378}
08-28 23:16:40.921: E/cpu_temp(6690): MSG_ON_WATCH_CURRENT
08-28 23:16:41.200: D/WifiService(511): New client listening to asynchronous messages
08-28 23:16:41.566: I/godot(8274): Remote Debugger: Connection failed with status: 0'', retrying in 1 sec.
08-28 23:16:42.567: I/godot(8274): Remote Debugger: Connection failed with status: 0'', retrying in 1 sec.
08-28 23:16:43.568: I/godot(8274): Remote Debugger: Connection failed with status: 0'', retrying in 1 sec.
08-28 23:16:43.568: I/godot(8274): Remote Debugger: Unable to connect
08-28 23:16:43.568: I/godot(8274): pre-removing print handler...
08-28 23:16:43.568: I/godot(8274): removing print handler...
08-28 23:16:43.568: I/godot(8274): found
08-28 23:16:43.568: I/godot(8274): print hanlder list is 0x0
08-28 23:16:43.577: I/godot(8274): has res dir:
08-28 23:16:43.579: I/godot(8274): exists engine cfg? 0
08-28 23:16:43.583: I/godot(8274): loaded engine.cfg
08-28 23:16:43.588: I/godot(8274): *****SETUP OK
08-28 23:16:43.654: I/godot(8274): **START
What should I do to debug on android?
I use c2e2f2e
https://github.com/okamstudio/godot/commit/c2e2f2e0aebf6342e6f18ae5d67b6a825590675a
commit (Aug 8, 2015) for now.
—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/2401#issuecomment-135786709.

--
OkamStudio

I also just noticed that Android added adb reverse, this should eventually
allow to add a more efficient remotefs
On Fri, Aug 28, 2015 at 11:32 AM, Okam Studio<EMAIL_ADDRESS>wrote:

check a few things, mainly that

your export templates are as recent as your editor
both your device and your PC are in the same intranet and can see
eachother (remote debug goes via wifi, not via usb cable)

if both are met, it should work
On Fri, Aug 28, 2015 at 11:21 AM, volzhs<EMAIL_ADDRESS>wrote:

I tried to debug on android devices.
So, I checked on "Deploy Remote Debug" option and run on android.
and adb rogcat says.
08-28 23:16:40.565: I/godot(8274): �[1;31;40mERROR: _resolve_hostname:
�[1;37;40mgethostbyname failed!
08-28 23:16:40.565: I/godot(8274): �[0;31;40m At:
drivers/unix/ip_unix.cpp:69.�[0;0;37m
08-28 23:16:40.566: I/godot(8274): �[1;31;40mERROR: connect:
�[1;37;40mCondition ' p_host.host == 0 ' is true. returned:
ERR_INVALID_PARAMETER
08-28 23:16:40.566: I/godot(8274): �[0;31;40m At:
drivers/unix/stream_peer_tcp_posix.cpp:138.�[0;0;37m
08-28 23:16:40.887: W/ActivityManager(511): Activity pause timeout for
ActivityRecord{de61d2 u0 com.macaron.twinpuzzle/com.android.godot.Godot
t3378}
08-28 23:16:40.921: E/cpu_temp(6690): MSG_ON_WATCH_CURRENT
08-28 23:16:41.200: D/WifiService(511): New client listening to
asynchronous messages
08-28 23:16:41.566: I/godot(8274): Remote Debugger: Connection failed
with status: 0'', retrying in 1 sec.
08-28 23:16:42.567: I/godot(8274): Remote Debugger: Connection failed
with status: 0'', retrying in 1 sec.
08-28 23:16:43.568: I/godot(8274): Remote Debugger: Connection failed
with status: 0'', retrying in 1 sec.
08-28 23:16:43.568: I/godot(8274): Remote Debugger: Unable to connect
08-28 23:16:43.568: I/godot(8274): pre-removing print handler...
08-28 23:16:43.568: I/godot(8274): removing print handler...
08-28 23:16:43.568: I/godot(8274): found
08-28 23:16:43.568: I/godot(8274): print hanlder list is 0x0
08-28 23:16:43.577: I/godot(8274): has res dir:
08-28 23:16:43.579: I/godot(8274): exists engine cfg? 0
08-28 23:16:43.583: I/godot(8274): loaded engine.cfg
08-28 23:16:43.588: I/godot(8274): *****SETUP OK
08-28 23:16:43.654: I/godot(8274): **START
What should I do to debug on android?
I use c2e2f2e
<
https://github.com/okamstudio/godot/commit/c2e2f2e0aebf6342e6f18ae5d67b6a825590675a
commit (Aug 8, 2015) for now.
—
Reply to this email directly or view it on GitHub
<https://github.com/okamstudio/godot/issues/2401#issuecomment-135786709
.

--
OkamStudio
—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/2401#issuecomment-135789071.

@okamstudio
I recompiled tools and templates. (still I'm using c2e2f2e0aebf6342e6f18ae5d67b6a825590675a)
And connect device with adb tcpip 5555
C:\Users\Administrator>adb devices
List of devices attached
<IP_ADDRESS>:5555       device

I updated tools and template, and connect over wifi in same intranet.
But logcat says same messages.
Should I use latest commit?

I finally found what causes this problem.
I found empty string for network/debug_host in editor_setting.xml.
<string name="network/debug_host"> "" </string>

but Settings > Editor Settings > Network > Debug host shows proper IP.
I never change this property before.
So, I changed it something another and change it back to default IP.
Then, editor_setting.xml is saved properly.
And debug remote & live edit seems to work now. :)

Marked as solved by the OP.


Dev: Allow user login on local dev sites

Local development sites do not currently support user login. This would be useful for developing various features.

Nothing to qa here


Ionic2 - I can’t retrieve the localStorage value

I am trying to retrieve the localStorage value; however, I can't get any value and my log looks like the following:

log is: {}

This is my code so far:
import {Storage, LocalStorage} from 'ionic-angular';
//I am storing here
this.local = new Storage(LocalStorage);
this.local.set('options', 'op2');
//In the chrome developer tools, I can see that the value was stored successfully in localStorage

//I can't see anything, when trying to retrieve the stored value
console.log('log is : '+ JSON.stringify(this.local.get('options')));

For your information:
I used the default functions localStorage.getItem() and localStorage.setItem(), they work fine. I believe it's and Ionic2 bug! So please check it out!
Your system information:
Cordova CLI: Not installed
Ionic Framework Version: 2.0.0-beta.4
Ionic CLI Version: 2.0.0-beta.24
Ionic App Lib Version: 2.0.0-beta.14
ios-deploy version: Not installed
ios-sim version: Not installed
OS: Mac OS X El Capitan
Node Version: v5.9.0
Xcode version: Xcode 7.2 Build version 7C68

Hi @HamzaLJ the .get() method returns a promise because it's an async call, if I'm not mistaken. so in order to retrieve the value and console log it, you'd have to do something like -
this.local.get('options').then((valueFromStorage) => console.log(valueFromStorage));

Please read the very well written docs regarding localstorage here - http://ionicframework.com/docs/v2/api/platform/storage/LocalStorage/#get

Hello! Thanks for opening an issue with us! As this seems like more of a support question i will urge that you ask this question on our forum (forum.ionicframework.com). Thanks for using Ionic!


Error when trying to bind Error Message Resource

I get the following error when trying to bind an error message resource on an input element:
Uncaught Error: Template parse errors:
Can't bind to 'errorMessageResource' since it isn't a known property of 'input'. ("     <input mz-input mz-validation required [label]="'First Name'" [validate]="true" id="firstName" [ERROR ->]
<input mz-input mz-validation required [label]="'First Name'" [validate]="true" id="firstName" [errorMessageResource]="errorMessages.firstName" formControlName="firstName" type="text" placeholder="First Name"/>
  errorMessages = {
    firstName: {
      required: `A first name is required.`,
      minlength: `First name must be at least 2 characters.`,
      maxlength: `First name may be no more than 50 characters.`
    }
  }

I was able to get this to work by Importing the MzInputDirective, MzValidationComponent components into the component.. There is no mention of this in the documentation or the examples

Hi @detchison, you should only need to import MaterializeModule in your Angular module as it already include both MzInputDirective and MzValidationComponent.
import { NgModule } from '@angular/core';
import { FormsModule, ReactiveFormsModule } from '@angular/forms';
import { MaterializeModule } from 'ng2-materialize';

@NgModule({
  imports: [
    FormsModule,
    ReactiveFormsModule,
    MaterializeModule,
  ],
...
})
export class FormExampleModule { }

I wasnt able to get it to work that way. I am starting to think with this and other issues, i.e. setting the selected value of an mz-select not working, that there is some versioning issues perhaps...

Can you add an example in your detchison/ngmaterializeTest repository? I'll could take a look at it ...

I was able to rectify the issue, not sure what it was, but just went and made sure everything was imported that way it should have been... sorry for the time waster.


Set files field in package.json

Publish only files we needed via files field in package.json. Now in npm published dev stuff too, what causes build error in my project (.babelrc).

Thanks! I also removed source and styles since only compiled files are used in the package.

Published 2.2.2 with your change: https://github.com/ilyabirman/Likely/releases/tag/v2.2.2


Implement support for macOS Apple Silicon => x86_64 cross compilation

I hope to use this in CI in the near future to compile x86 release binaries on Apple Silicon to make this process faster and have access to the latest compiler version. That can be discussed in detail on another issue.

[ ] The PR currently adds a test job that cross-compiles to x86 on the arm64-based and then runs the tests via Rosetta. This is solely to demonstrate that this change works, remove it before merging.

For macOS-based reviewers who want to try this out locally, bazel test --config=macos-cross-x86_64 //src/workerd/util:sqlite-test && file bazel-bin/src/workerd/util/sqlite-test is a good way to compile a test binary and confirm that it is compiled for x86, without having to recompile everything.

Love it!
Any chance you want to take the same opportunity to add a build step to release.yml so we can remove the internal runner for macos arm64?

Love it! Any chance you want to take the same opportunity to add a build step to release.yml so we can remove the internal runner for macos arm64?

That is something I have in mind (#2884 was a step in that direction), but it merits some discussion internally first – based on https://docs.github.com/en/actions/administering-github-actions/usage-limits-billing-and-administration#usage-limits we are limited to 5 concurrent macOS build jobs across the cloudflare org. We have encountered macOS runner exhaustion for workerd before. Adding a second release job on GitHub will not add much overhead (Apple Silicon macOS is the fastest free tier runner) but might be enough to make us hit runner exhaustion again occasionally and have macOS builds stall.

As you’ve mentioned, I don’t think that the release build runs often enough. I think we should go ahead with it.
As Steve mentioned in an internal conversation, GitHub limits are something he’s working on, I don’t think we should work too hard to engineer around them.

As you’ve mentioned, I don’t think that the release build runs often enough. I think we should go ahead with it. As Steve mentioned in an internal conversation, GitHub limits are something he’s working on, I don’t think we should work too hard to engineer around them.

👍 Let's discuss at the team meeting


Debugging "JSON value '443' of type NSNumber cannot be converted to NSString" type errors

I get a bunch of these errors in the app, and I understand that it's some javascript value that should have been a string is sent over the bridge, however, some of these values are generated dynamically, how can I track down where this issue is originating in javascript?

Can you add a breakpoint to the place where this warning is being called on the native side? There is probably more context available there to help debug the issue.

Thanks, that's a good point, but I'm only a js developer. I don't know Objective C, so it's all Spanish to me :(
So there isn't any way other than debugging the Objective C side to figure out what caused the error on the js side?

Closing this issue because it has been inactive for a while. If you think it should still be opened let us know why.


[DA] Import and compile libraries

Import required device attestation libraries from @hawk248 and integrate into CHIP

I believe this ticket is outdated and can be closed.

Closing as this is obsolete.


Implementation for "The DiscreteFormatStyle Protocol" proposal

This PR lists all changes proposed in the "The DiscreteFormatStyle Protocol, with few exceptions:

Conformance for Date.ComponentsFormatStyle is not included as the type is not part of FoundationPreview yet
Conformance for Date.ISO8601FormatStyle is not included to the type's recent rewrite.

@swift-ci please test

@swift-ci please test

@swift-ci please test

@swift-ci please test

@swift-ci please test


test case: canister sandbox panic

upgrades drun, disables crashing amounts in basic-cycles.mo and exhibits the panic in canster-sandbox-panic.mo until upstream has fixed that. Reported bug at https://forum.dfinity.org/t/10729?u=nomeata

This triggers
ingress Completed: Reply: 0x4449444c016c01b3c4b1f204680100010a00000000000000000101
ingress Completed: Reply: 0x4449444c0000
debug.print: topping up
thread '<unnamed>' panicked at 'Failed to serialize message: ErrorImpl { code: Message("The number can't be stored in CBOR"), offset: 0 }', /build/rs/canister_sandbox/common/src/transport.rs:285:55
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'MR Batch Processor' panicked at 'Thread pool worker panicked', /build/drun-vendor.tar.gz/scoped_threadpool/src/lib.rs:236:13
thread 'main' panicked at 'Failed to send batch 24: background worker is dead', messaging/src/message_routing.rs:949:51

This seems to come from https://rustdoc.etonomy.org/src/serde_cbor/ser.rs.html#267-279 or https://rustdoc.etonomy.org/src/serde_cbor/ser.rs.html#302-308, i.e serialize_u128 or serialize_u64. Looks like there is u128 or i128 used somewhere in the system API, and the sandbox uses CBOR, but serde doesn’t support that (despite the fact that CBOR does, via bignums).

Related: https://github.com/pyfisch/cbor/issues/77 and https://github.com/pyfisch/cbor/pull/145

BTW, looks like cbor_serde is no longer maintained by upstream: https://github.com/pyfisch/cbor. It recommends two other libraries. And https://crates.io/crates/ciborium at least get CBOR numbers right (always encodes at smallest size)

Maybe we can actually merge this, so that we notice other breakage in drun in the meantime.

Do we need an issue for the commented-out tests?

Not really, we'll here on https://forum.dfinity.org/t/panicked-at-failed-to-serialize-message-errorimpl-code-message-the-number-cant-be-stored-in-cbor-offset-0-build-rs-canister-sandbox-common-src-transport-rs55/10729/4 once it's fixed, I hope

Hmm, looks like drun on MacOS isn’t using the canister sandbox, so there we get different output. Maybe we can just wait for the bug to be fixed.

@ulan says that this is fixed. Let’s see…

I see
       >   The package requires the Cargo feature called `named-profiles`, but that feature is not stabilized in this version of Cargo (1.56.0).
       >   Consider adding `cargo-features = ["named-profiles"]` to the top of Cargo.toml (above the [package] table) to tell Cargo you are opting in to use this unstable feature.
       >   See https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#custom-named-profiles for more information about the status of this feature.

Do we need to update rust and/or cargo maybe? @ggreif didn’t you recently try to bump rust?

@ggreif, could you help us updating drun maybe?

With rust-1.57 I am making progress. I still need to apply this patch to ic, because otherwise cargo vendor complains about multiple versions of wabt:
diff --git a/rs/boundary_node/prober/Cargo.toml b/rs/boundary_node/prober/Cargo.toml
index b28edf48..6beb9bd1 100644
--- a/rs/boundary_node/prober/Cargo.toml
+++ b/rs/boundary_node/prober/Cargo.toml
@@ -35,4 +35,4 @@ slog-scope = "4.1.2"
 slog-term = "2.6.0"
 tokio = { version = "1.15.0", features = ["full"] }
 tokio-openssl = "0.6.0"
-wabt = "0.10.0"
+wabt = { git = "https://github.com/dfinity-lab/wabt-rs", tag = "0.10.0-dfinity" }

Maybe someone can upstream this?

Next error would be
   Compiling ic-registry-provisional-whitelist v0.8.0 (/build/rs/registry/provisional_whitelist)
The following warnings were emitted during compilation:

warning: ../src/backtrace.c:11:10: fatal error: libunwind.h: No such file or directory
warning:    11 | #include <libunwind.h>
warning:       |          ^~~~~~~~~~~~~
warning: compilation terminated.

error: failed to run custom build command for `ic-canister-sandbox-launcher v0.8.0 (/build/rs/canister_sandbox/sandbox_launcher)`

Caused by:
  process didn't exit successfully: `/build/rs/target/release/build/ic-canister-sandbox-launcher-4d61a222ccdb986f/build-script-build` (exit status: 1)
  --- stdout
  TARGET = Some("x86_64-unknown-linux-gnu")
  OPT_LEVEL = Some("3")
  HOST = Some("x86_64-unknown-linux-gnu")
  CC_x86_64-unknown-linux-gnu = Some("/nix/store/ajisnbqj2qyirds1hnwf3pn919c2c8yc-gcc-wrapper-10.3.0/bin/cc")
  CFLAGS_x86_64-unknown-linux-gnu = None
  CFLAGS_x86_64_unknown_linux_gnu = None
  HOST_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  running: "/nix/store/ajisnbqj2qyirds1hnwf3pn919c2c8yc-gcc-wrapper-10.3.0/bin/cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-Wall" "-Wextra" "-o" "/build/rs/target/x86_64-unknown-linux-gnu/release/build/ic-canister-sandbox-launcher-fee8adc3b2e6dbd4/out/../src/backtrace.o" "-c" "../src/backtrace.c"
  cargo:warning=../src/backtrace.c:11:10: fatal error: libunwind.h: No such file or directory
  cargo:warning=   11 | #include <libunwind.h>
  cargo:warning=      |          ^~~~~~~~~~~~~
  cargo:warning=compilation terminated.
  exit status: 1

  --- stderr

  error occurred: Command "/nix/store/ajisnbqj2qyirds1hnwf3pn919c2c8yc-gcc-wrapper-10.3.0/bin/cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-Wall" "-Wextra" "-o" "/build/rs/target/x86_64-unknown-linux-gnu/release/build/ic-canister-sandbox-launcher-fee8adc3b2e6dbd4/out/../src/backtrace.o" "-c" "../src/backtrace.c" with args "cc" did not execute successfully (status code exit status: 1).

warning: build failed, waiting for other jobs to finish...

@hcbdfinity ^ any ideas why libunwind.h is failing for @nomeata ?

Maybe someone can upstream this?
Thanks! I'll upstream the fix.

I think you would simply need to get libunwind into your nix env, AFAICT it is supported already: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/libraries/libunwind/default.nix
there may be a need to pass pkg-config flags, TBOMK libunwind upstream has been pkg-config'ed

As @hcbdfinity said, making libunwind accessible in the nix env is enough. So once the Cargo files are in better order in the ic repo, we can start testing the latest replica code here again (I hope)

Upstreamed fix: https://github.com/dfinity/ic/commit/bedc41753c38a3635e22be3dcb4c81c2bbddff2d

Bumping drun separately in https://github.com/dfinity/motoko/pull/3181, afterwards this PR can add the new test case

Probably not needed anymore, as there is a fix in the replica already (https://github.com/dfinity/motoko/pull/3181)


is support RSCoordinator about rediscluster client mode?

Cause sentinel memory capacity is limited, I need more memory, I want to use redisearch cluster mode (RSCoordinator), do Jredisearch support  the rediscluster client?

thanks for you reply,I will have a try to use jedis 4.11 client!


Dynamic select results "No property or field 'MyProperty' exists in type 'List`1'"

1. Description
I'm selecting fields dynamic on a class that has as list property mapped as One to Many.
The client sends to the server an array on fields and the server convert the list to the lib's format.
This is a simple class to show my use-case.
public class Parent
{
	public int Id { get; set; }
	public List<Child> Children { get; set; }
}

public class Child
{
	public int Id { get; set; }
	public string MyProperty { get; set; }
}

Every thing works fine except when I select an field from the Child object.
This is the select string that result and throws an error:
new { Id, new { Children.MyProperty } as Children }
2. Exception
"No property or field 'MyProperty' exists in type 'List`1'"
4. Any further technical details
.NET 6
DynamicCore version 1.2.20

Can you show the non dynamic linq select version?

Sure. It will be something like this.
 return (from p in DbSet.Include(p => p.Children)
      select new
      {
          p.Id,
          Children = from c in p.Children
                     select new
                     {
                         c.Id,
                         c.MyProperty
                     }
      }).ToList();

@LuigiMaestrelli
This will be something like:
var resultDynamic = DbSet.Include(p => p.Children).Select("new (it.Id as ParentId, Children.Select(new (Id, MyProperty)) as Children)");


Margin top on side bar

Added a top margin to sidebar
refs #115

Thanks for this, I'll be adding this margin back into the sidebar, but I decided to refactor the standard and overlay options for scrollbars as well, so I won't be needing to merge this PR, but I did add the change. Thanks for the help


Help with pilot to submit for review app to TestFlight external testers

What parameters should I send to pilot in Fastfile to request submit the app for review for external testers after the app has been delivered to iTunes?
Any info would be appreciated!

You can set the --distribute_external flag. Also, you can run pilot help to get a list of all possible arguments.

I tried the distribute to external command, however I need to set some required fields, like changes and what to test before submitting.

Hi @kikolobo
Thanks for following up. Can you send over the output with the error message you get?
It'll help if you can send over the information in this template. https://github.com/fastlane/fastlane/blob/master/ISSUE_TEMPLATE.md
Thanks!

@kikolobo did you find a solution?

@bolom  Yes I did!
I just added some text to the what to test parameter to the pilot command and set the flag to distribute to externals to true.
Here's the fastFile :
pilot(distribute_external: true,
        changelog: 'Misc. bug fixes and improvements throughout the app.'
    )

@kikolobo Awesome! Glad to hear you are up and running. Feel free to ping this issue if you have any questions. Closing for housekeeping :+1:


Enable flow in react-native-implementation

I noticed I didn't get type defs anymore for react-native. Looks like it is broken since we removed the .flow file in https://github.com/facebook/react-native/commit/3e153b2a5bee943f69621d05e271782e1a3b3edd. To fix it we can now enable flow in react-native-implementation since it now supports properties.
Test plan
Tested that I get type hints when using imports from react-native in a project.

cc @satya164

@facebook-github-bot shipit

I just noticed this caused some flow errors to appear in UIExplorer now that the types are exported. This probably won't land because of those errors, working on a fix.

@facebook-github-bot shipit

@ericvicenti Could you have a look at this? If you are using this internally it will probably cause some new flow errors that were previously ignored because all of RN was any.

Fixed conflicts and new flow errors, would be nice to merge this asap to avoid new ones. cc @ericvicenti @hramos

@facebook-github-bot shipit


Get IaC production-ready: documentation

If we decide to proceed with Pulumi as a framework for managing hubverse cloud infrastructure, we need to ensure that the process is well-documented.
Definition of done:

[x] Documentation exists in the repo (e.g., README or similar)
[x] Documentation has high-level instructions for the use case of onboarding a hub to Hubverse-owned AWS resources (basically, update the YAML config)
[ ] Documentation has more detailed instructions for those who need to update the Pulumi code itself (for example, to fix a bug or add a new AWS resource to support additional features)

There is currently an open PR that address two of the three items on this ticket: https://github.com/Infectious-Disease-Modeling-Hubs/hubverse-infrastructure/pull/44
This PR from last week addresses the third item: https://github.com/Infectious-Disease-Modeling-Hubs/hubverse-infrastructure/pull/42

Cleaning up--never closed this after the PRs were merged!


SSL Options are not visible with a long Domain-List

I have one Certificate, that contains nearly all of my Domains, which was issued with the Cloudflare DNS-Challenge.
The upper 2 Options (Force SSL & HTTP/2 Support) in the SSL Menu are not visible when using a Certificate with a long domain list/name (see screenshot)
Version 2.8.0
Expected behavior
All Options visible or even an Option to give Certificates custom Names

Operating System
Docker-Compose / Ubuntu 20 (KVM VM)

PR has been merged. Will be part of the next release.


Use Rectangle::to()

Use new Rectangle::to<NewBaseType>() method to simplify code.
Related:
PR https://github.com/lairworks/nas2d-core/pull/450 (introduced Rectangle::to<NewBaseType())
Issue #206 Replace isPointInRect with rect.contains(point)

Hmm, seems MSVC is complaining.

Collision with Windows.h header.
2>E:\Git\OPHD\src\Common.cpp(243,18): error C2872: 'Rectangle': ambiguous symbol
2>C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\wingdi.h(4515,24): message : could be 'BOOL Rectangle(HDC,int,int,int,int)' (compiling source file ..\..\src\Common.cpp)
2>E:\Git\OPHD\nas2d-core\include\NAS2D\Renderer\Rectangle.h(22,1): message : or       'NAS2D::Rectangle' (compiling source file ..\..\src\Common.cpp)
2>E:\Git\OPHD\src\Common.cpp(243,9): error C3321: an initializer list is unexpected in this context
2>E:\Git\OPHD\src\Common.cpp(243,53): error C2062: type 'int' unexpected
2>E:\Git\OPHD\src\Common.cpp(243,71): error C2143: syntax error: missing ';' before '{'
2>E:\Git\OPHD\src\Common.cpp(243,76): error C2143: syntax error: missing ';' before '}'
2>E:\Git\OPHD\src\Common.cpp(243,77): error C2059: syntax error: ')'

Qualifying it with a namespace fixes the issue.

bool pointInRect_f(int x, int y, float rectX, float rectY, float rectW, float rectH)
{
	return NAS2D::Rectangle{rectX, rectY, rectW, rectH}.to<int>().contains(Point{x, y});
}

Yeah, I figure adding a NAS2D:: prefix will fix that. We should probably be adding those prefixes anyway, and getting rid of using namespace statements.
I'm got 3 different test runs going to verify some different possible solutions. I'll update once they finish.

Even better. The windows header isn't needed in Common.cpp. Take it out.

Not sure about that. I see code such as:
#if defined(WINDOWS) || defined(WIN32)
	MessageBoxA(WIN32_getWindowHandle(), msg.c_str(), title.c_str(), MB_OK | MB_ICONERROR | MB_TASKMODAL);
#else
	SDL_ShowSimpleMessageBox(SDL_MESSAGEBOX_ERROR, title.c_str(), msg.c_str(), NULL);
#endif

We probably need to clean that up first before removing the Windows header. That would be a separate PR.

Huh. I missed that. It must be a transitive include then.


[GLBC] Front-end not created

I recently ran following Shell script with given Kubernetes definitions and after some time, GLBC was created but it did not get any front-end created.
set -ex

read -p "Press any key to start..." -sn1

gcloud config set project echo-test-166405
gcloud container clusters create echo-cluster --machine-type="g1-small" --num-nodes=1

echo "Cluster created"

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /tmp/tls.key -out /tmp/tls.crt -subj "/C=LT"
kubectl create secret tls echoserver-tls --key /tmp/tls.key --cert /tmp/tls.crt

# Note: At this point I just waited a minute to make sure it's all OK.
echo "Self-generated certificate created"
read -p "Press any key to continue... " -sn1

kubectl apply -f echoserver/00-namespace.yaml
kubectl apply -f echoserver/deployment.yaml
kubectl apply -f echoserver/service.yaml
kubectl apply -f echoserver/ingress-tls.yaml

Here are all YAML files merged into one:
apiVersion: v1
kind: Namespace
metadata:
  name: echoserver
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: echoserver
  namespace: echoserver
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: echoserver
    spec:
      containers:
      - image: gcr.io/google_containers/echoserver:1.0
        imagePullPolicy: Always
        name: echoserver
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: echoserver
  namespace: echoserver
spec:
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
  type: NodePort
  selector:
    app: echoserver
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: echoserver
  namespace: echoserver
  annotations:
    kubernetes.io/tls-acme: "true"
    kubernetes.io/ingress.class: "gce"
    kubernetes.io/ingress.allow-http: "false"
spec:
  tls:
  - hosts:
    - echo.pijusn.eu
    secretName: echoserver-tls
  rules:
  - host: echo.pijusn.eu
    http:
      paths:
      - backend:
          serviceName: echoserver
          servicePort: 80

I expected HTTPS front-end to be created. It was a fresh project, fresh cluster so no quotes were kicking in.
After I removed kubernetes.io/ingress.allow-http: "false" it did create an HTTP front-end but still did not create HTTPS one.
This seems like an issue. Also, if you have ideas where to look for an error message or something (why it failed to create it) - please share.

Ok, so there are a few things here that result in you seeing what you see.
Firstly, kube-lego needs to be install in your cluster for the tls-acme annotation to work (for getting a cert from letsencrypt).
Secondly, because there is no cert in the cluster (from kube-lego) AND http is disabled, then no front-end will be created.
Once you allow http by setting ingress.allow-http to true (default setting), then it will create the http front end, but again, kube-lego is not setup to create the cert for the https front end.

@nicksardo if you are comfortable, I would like to contribute to this repository by labeling issues nginx or gce.

@tonglil Correct me if I'm wrong, but I believe for labeling power you must be an org member and get write permission for the repo. With the start of the governance, I think all org changes are frozen for the time being.

Gotcha, no worries. Happy to apply again another time.

@tonglil I actually used a self-signed certificate. See (you can also find it in the Shell script I pasted earlier):
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /tmp/tls.key -out /tmp/tls.crt -subj "/C=LT"
kubectl create secret tls echoserver-tls --key /tmp/tls.key --cert /tmp/tls.crt

I would expect Kubernetes not to depend on kube-lego. To be specific, I would expect it to create an HTTPS frontend with the provided certificate even if it's not trusted, is issued for a different domain or whatever.
I now know that kube-lego does not support it but (correct me if I'm wrong) ACME server ignores certificate when using HTTPS. That's a nice way to kick-start LB setup. To achieve it, ingress controller should support such scenarios.

@pijusn oops, sorry I missed that part.
What happens if you do this:
spec:
  tls:
-  - hosts:
-    - echo.pijusn.eu
-    secretName: echoserver-tls
+  - secretName: echoserver-tls

I'm betting you're creating the secret in the default namespace but the ingress exists in echoserver.  It really should be surfaced as an event.

@nicksardo I think you are right. That would certainly make sense. I will verify it some time this week. Thank you for pointing it out.

Let's use this issue to track surfacing an event when cert lookup fails.
In the future, stack overflow would be a better medium for this question.

+1 I also hit this issue and was at a loss to find out what the issue was.  The target proxy and global forwarding rule would just not be created.  Users should have visibility into any errors in ingress => GCP object creation pipeline.

This issue was moved to kubernetes/ingress-gce#41


Fix small doc errors

Signed-off-by: Arnaud J Le Hors<EMAIL_ADDRESS>Type of change

Documentation update

Description
Fixes a couple of small errors (typo like)

@mergifyio backport release-2.3


Added empty build.gradle files for arm64 and s390x because the gradle assemble task fails without them

Description
This change fixes the gradle assemble task which is failing due to missing build.gradle files in the docker subprojects for arm64 and s390x. These files were removed in PR #4723. I created issue #8903 to discuss this particular bug and created this PR for a quick fix. In addition, issue #8112 was created to further investigate why removing those files causes the build to fail.
Related Issues
Resolves #8903
Related #8112
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed per the DCO using --signoff
[ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@evankielley could you please rebase against latest main (to fix build issues)? Also, thanks to you finding, I will suggest to add distribution checks for pull requests (https://github.com/opensearch-project/OpenSearch/pull/8924), that would have caught the issue  ... thank you

@evankielley could you please rebase against latest main (to fix build issues)? Also, thanks to you finding, I will suggest to add distribution checks for pull requests (#8924), that would have caught the issue ... thank you

@reta I just rebased but as for adding a distro check to gradle assemble, it looks like you have that covered now in PR #8924 ?

@reta I just rebased but as for adding a distro check to gradle assemble, it looks like you have that covered now in PR #8924 ?

Thanks @evankielley , I temporarily applied your fix (to prove that all checks pass) but it is removed. I will rebase the my change after we get this one in, thank you.

@reta I just rebased but as for adding a distro check to gradle assemble, it looks like you have that covered now in PR #8924 ?

Thanks @evankielley , I temporarily applied your fix (to prove that all checks pass) but it is removed. I will rebase the my change after we get this one in, thank you.

Perfect, thank you


Make iOS client crash when Cronet issues 2 finishing callbacks to gRPC

Make global counter for the number of finishing callbacks (on_succeeded, on_failed, on_canceled) issued by Cronet for each Cronet stream. Crashes if Cronet issues two finishing callbacks for the same stream. This would help confirm Cronet is behaving correctly when received crash reports with only stack trace log.

[trickle] No significant performance differences

[microbenchmarks] No significant performance differences


Add fork support

This PR implements the fork support for the provider (issue #2 ). It consists of:

static pool for pkcs11 modules, object- and session handles
atfork handler, invalidates the elements in the pools after a fork in the child
additional testcase with forking between openssl crypto operations

Closes #2

I have tested this code with Apache httpd (in non-debug mode) and it works fine.

The force-push contains the reworked lazy-init of the PKCS#11 library. If you run tfork testcase with PKCS11SIGN_DEBUG_LEVEL=4 you can verify this behavior. The PKCS#11 version information is logged on each re-initialize.


Policy assignment definitions deleted in platform/alz/2024.10.0

It looks like there were a number of policy assignment definitions that were deleted as part of https://github.com/Azure/Azure-Landing-Zones-Library/pull/76, for example platform/alz/policy_assignments/deny_resource_locations.alz_policy_assignment.json
I presume this is correct as I don't see them upstream, but just wanted to confirm that this was the intent, and that if we want to carry on using these assignments that we need to create / copy them into our own library?
Also curious how they got there to begin with as they don't ever seem to have been in the upstream definitions. Looks like it might have come across from CAF? If so is there an argument they should be included to help people migrating?

#RR


New sdk for Azure Blueprint

Description
Add new SDK for Azure Blueprint RP.
Swagger PR: https://github.com/Azure/azure-rest-api-specs/pull/3855

This checklist is used to make sure that common guidelines for a pull request are followed.

[x] Please add REST spec PR link to the SDK PR
[x] I have read the contribution guidelines.
[x] The pull request does not introduce breaking changes.

General Guidelines

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

SDK Generation Guidelines

[x] If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
[x] The generate.cmd file for the SDK has been updated with the version of AutoRest, as well as the commitid of your swagger spec or link to the swagger spec, used to generate the code.
[x] The *.csproj and AssemblyInfo.cs files have been updated with the new version of the SDK.

Please squash commits into a single commit and update the PR

Please squash commits into a single commit and update the PR

we can do a squash merge, right?

@haitch not on this PR. You will have to recreate the branch with a single commit and force push

@haitch not on this PR. You will have to recreate the branch with a single commit and force push

✅


Abstract model inheritence

To revisit for 2.0.
import uuid
from bloop import BaseModel, Column, UUID, String, Engine

class HasId(BaseModel):
    class Meta:
        abstract = True
    id = Column(UUID, hash_key=True)

class HasManager(BaseModel):
    class Meta:
        abstract = True
    manager_id = Column(UUID, hash_key=True)

class HasReports(BaseModel):
    class Meta:
        abstract = True
    reports = Column(Set(UUID), hash_key=True)

# -----------

class Employee(HasId, HasManager):
    employer = Column(String)

# order shouldn't matter here.  For this specific example,
# (HasReports, Employee, HasManager) should be equivalent.

class Manager(Employee, HasReports, HasManager):
    class Meta:
        table_name = "Employee"
    pass

# -----------

engine = Engine()
engine.bind(Employee)

# -----------

employee = Employee(
    id=uuid.uuid4(),
    employer="WidgetCo Temp"
)

manager = Manager(
    id=uuid.uuid4(),
    manager_id=None,
    reports={employee.id},
    employer=employee.employer
)
employee.manager_id=manager.id

engine.save(employee, manager)

Might be easier with a different base - ModelFragment or such.  Or maybe this can be accomplished with a clever wrapper over class.mro() and __getattr__. Hmm..


Update type in Cluster Overview docs

The command line arguments for clustering incorrectly had the 2nd and 3rd servers using the same checkpoint directory.

Not sure what happened here, but the CIs did not automatically start and aren't listed in GH Actions either ... almost like those CIs were either never started or were deleted.

There was a recent ci.yml file changed that ignores all .md files and won't run CI when only .md files are changed. Therefore, the CIs didn't run (by design) but the PR seems to be looking for that CI to be ran.

A good test for .md PRs then!
It's a pretty trivial change, just noticed it and thought it would be easy to fix.

Yeah, early stages of a release hiccups. @Vazois can still merge it after reviewing it.


Relax "Windows RT" checks for devices that are "Windows RT 8.1"

Windows tablets are not detected as being "touch capable" or "tablet", which obviously is wrong.
Maybe this could be fixed at a lower level in ua_parser package, but the regexp with its os replacement is explicitly stating the version, so I guess it was the intention.

Thanks for the PR!