[BUG] Azure-Identity: UsernamePasswordCredential.GetToken was unable to retrieve an access token (Value cannot be null. (Parameter 'tenantId'))

Library name and version
Azure.Identity 1.13.1
Describe the bug
Starting at 1.11.0 the code below does not work. If I downgrade to one version lower, 1.10.4 this code works without issue.
using Azure.Core.Diagnostics;
using Azure.Identity;
using Azure.ResourceManager;
using Azure.ResourceManager.Compute;
using Azure.ResourceManager.Network;
using Azure.ResourceManager.Resources;
using Azure.ResourceManager.Storage;

const string AZ_CLIENT_ID = "";
const string AZ_LOGIN_TENANT = "asdf";
const string AZ_PASS = "";
const string AZ_USER = "";
const string DEFAULT_TCP_RESOURCE_GROUP = "";
const string NAME = "";
const string SERIAL = "";

string LOGIN_ENDPOINT = $"https://login.{SERIAL}.microsoftdatabox.com";
string MANAGEMENT_ENDPOINT = $"https://management.{SERIAL}.microsoftdatabox.com";

using var listener = AzureEventSourceListener.CreateConsoleLogger();

var tokenOptions = new UsernamePasswordCredentialOptions
{
    AuthorityHost = new Uri(LOGIN_ENDPOINT),
    Diagnostics =
                {
                    IsLoggingContentEnabled = true,
                    LoggedHeaderNames = { "x-ms-request-id", "Authorization" },
                    LoggedQueryParameters = { "api-version" },
                }
};

var ArmClientOptions = new ArmClientOptions
{
    Environment = new ArmEnvironment(new Uri(MANAGEMENT_ENDPOINT), MANAGEMENT_ENDPOINT)
};

ArmClientOptions.SetApiVersion(ResourceGroupResource.ResourceType, "2019-10-01");
ArmClientOptions.SetApiVersion(VirtualMachineResource.ResourceType, "2020-06-01");
ArmClientOptions.SetApiVersion(DiskImageResource.ResourceType, "2020-06-01");
ArmClientOptions.SetApiVersion(VirtualNetworkResource.ResourceType, "2018-11-01");
ArmClientOptions.SetApiVersion(NetworkInterfaceResource.ResourceType, "2018-11-01");
ArmClientOptions.SetApiVersion(StorageAccountResource.ResourceType, "2019-06-01");
ArmClientOptions.SetApiVersion(BlobContainerResource.ResourceType, "2019-06-01");

var creds = new UsernamePasswordCredential(AZ_USER, AZ_PASS, AZ_LOGIN_TENANT, AZ_CLIENT_ID, tokenOptions);

ArmClient armClient = new(creds, "", ArmClientOptions);

var rgClient = armClient
    .GetDefaultSubscription()
    .GetResourceGroup(DEFAULT_TCP_RESOURCE_GROUP);

Expected behavior
No erros
Actual behavior
Errors with this:
[Informational] Azure-Identity: UsernamePasswordCredential.GetToken invoked. Scopes: [ https://management.redacted.microsoftdatabox.com/.default ] ParentRequestId: 606ed31c-a999-48e3-bb8d-4b3d08c8da5b
[Informational] Azure-Identity: UsernamePasswordCredential.Authenticate invoked. Scopes: [ https://management.redacted.microsoftdatabox.com/.default ] ParentRequestId: 606ed31c-a999-48e3-bb8d-4b3d08c8da5b
[Informational] Azure-Identity: UsernamePasswordCredential.Authenticate was unable to retrieve an access token. Scopes: [ https://management.redacted.microsoftdatabox.com/.default ] ParentRequestId: 606ed31c-a999-48e3-bb8d-4b3d08c8da5b Exception: Azure.Identity.AuthenticationFailedException (0x80131500): UsernamePasswordCredential authentication failed: Value cannot be null. (Parameter 'tenantId')
See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/usernamepasswordcredential/troubleshoot
---> System.ArgumentNullException (0x80004003): Value cannot be null. (Parameter 'tenantId')
[Informational] Azure-Identity: UsernamePasswordCredential.GetToken was unable to retrieve an access token. Scopes: [ https://management.redacted.microsoftdatabox.com/.default ] ParentRequestId: 606ed31c-a999-48e3-bb8d-4b3d08c8da5b Exception: Azure.Identity.AuthenticationFailedException (0x80131500): UsernamePasswordCredential authentication failed: Value cannot be null. (Parameter 'tenantId')
See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/usernamepasswordcredential/troubleshoot
---> System.ArgumentNullException (0x80004003): Value cannot be null. (Parameter 'tenantId')

Reproduction Steps
Change to version 1.11.0 or later to fail
Change to version 1.10.4 or earlier to work
Environment
Windows, VS2022, .net 6 or 8
ASE is disconnected

Hi @johnypony3 -
Could you please provide the full stack trace from the exception you are seeing? Also, it appears that you already have logging enabled. Can you share the logs as well?
I tried to reproduce the error based on your configuration above and I don't see the problem.

I will provide the info, currently afk. I will update my post with the ASE being disconnected. Could you try that way please?

@christothes I believe this issue is what I am having still: https://github.com/AzureAD/microsoft-authentication-library-for-dotnet/issues/4860

It appears different to me, but the full stack trace and the full logging output should provide more context.

Got it, I am new to this space and I am looking for clues :)
I added loggers with LogAlways with the results below.
I'd like to point out that with fiddler enabled, I do not see any calls being made when using version 1.13.1 of the Azure.Identity package. This error is happening locally.
Console logs:
[Informational] Azure-Identity: UsernamePasswordCredential.GetToken invoked. Scopes: [ https://management.dvm1559-00123.microsoftdatabox.com/.default ] ParentRequestId:
[Informational] Azure-Identity: UsernamePasswordCredential.Authenticate invoked. Scopes: [ https://management.dvm1559-00123.microsoftdatabox.com/.default ] ParentRequestId:
[Informational] Azure-Identity: UsernamePasswordCredential.Authenticate was unable to retrieve an access token. Scopes: [ https://management.dvm1559-00123.microsoftdatabox.com/.default ] ParentRequestId:  Exception: Azure.Identity.AuthenticationFailedException (0x80131500): UsernamePasswordCredential authentication failed: Value cannot be null. (Parameter 'tenantId')
See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/usernamepasswordcredential/troubleshoot
 ---> System.ArgumentNullException (0x80004003): Value cannot be null. (Parameter 'tenantId')
[Informational] Azure-Identity: UsernamePasswordCredential.GetToken was unable to retrieve an access token. Scopes: [ https://management.dvm1559-00123.microsoftdatabox.com/.default ] ParentRequestId:  Exception: Azure.Identity.AuthenticationFailedException (0x80131500): UsernamePasswordCredential authentication failed: Value cannot be null. (Parameter 'tenantId')
See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/usernamepasswordcredential/troubleshoot
 ---> System.ArgumentNullException (0x80004003): Value cannot be null. (Parameter 'tenantId')
Error: UsernamePasswordCredential authentication failed: Value cannot be null. (Parameter 'tenantId')
See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/usernamepasswordcredential/troubleshoot

Stack trace:
Timestamp: 12/17/2024 7:33:38 PM
Exception: AuthenticationFailedException
Message: UsernamePasswordCredential authentication failed: Value cannot be null. (Parameter 'tenantId')
See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/usernamepasswordcredential/troubleshoot
Formatted Stack Trace:
  at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow in Unknown File:line 0
  at Azure.Identity.UsernamePasswordCredential+<AuthenticateImplAsync>d__31.MoveNext in Unknown File:line 0
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw in Unknown File:line 0
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess in Unknown File:line 0
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification in Unknown File:line 0
  at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter.GetResult in Unknown File:line 0
  at Azure.Identity.UsernamePasswordCredential+<GetTokenImplAsync>d__32.MoveNext in Unknown File:line 0
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw in Unknown File:line 0
  at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow in Unknown File:line 0
  at Azure.Identity.UsernamePasswordCredential+<GetTokenImplAsync>d__32.MoveNext in Unknown File:line 0
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw in Unknown File:line 0
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess in Unknown File:line 0
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification in Unknown File:line 0
  at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult in Unknown File:line 0
  at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted in Unknown File:line 0
  at Azure.Identity.UsernamePasswordCredential.GetToken in Unknown File:line 0
  at Program.Main in C:\Users\v-edmeshuris\source\TCP-ArmClient\TCP-ArmClient\Program.cs:line 56

Thanks @johnypony3 - in the stack trace above, is there possibly an inner exception that is missing? There is nowhere in Azure.Identity directly that throws that ArgumentNullException for tenantId during the GetToken code path, but it's possible that it is coming from one of its dependencies. If not, can you run your repro under the debugger and enable all exceptions to see from where this is being thrown?

{
    "Timestamp": "12/18/2024 6:22:35 PM",
    "ExceptionDetails": {
        "Type": "Azure.Identity.AuthenticationFailedException",
        "Message": "UsernamePasswordCredential authentication failed: Value cannot be null. (Parameter 'tenantId')\nSee the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/usernamepasswordcredential/troubleshoot",
        "StackTrace": "   at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)\r\n
            at Azure.Identity.UsernamePasswordCredential.AuthenticateImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)\r\n
            at Azure.Identity.UsernamePasswordCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)\r\n
            at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)\r\n
            at Azure.Identity.UsernamePasswordCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)\r\n
            at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](Task`1 task)\r\n
            at Azure.Identity.UsernamePasswordCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)\r\n
            at Program.Main(String[] args) in C:\\Users\\v-redacted\\source\\TCP-ArmClient\\TCP-ArmClient\\Program.cs:line 64",
        "Source": "Azure.Identity",
        "HResult": -2146233088,
        "Data": {},
        "InnerException": {
            "Type": "System.ArgumentNullException",
            "Message": "Value cannot be null. (Parameter 'tenantId')",
            "StackTrace": "   at Microsoft.Identity.Client.AbstractAcquireTokenParameterBuilder`1.WithTenantId(String tenantId)\r\n
            at Microsoft.Identity.Client.AbstractAcquireTokenParameterBuilder`1.WithTenantIdFromAuthority(Uri authorityUri)\r\n
            at Azure.Identity.MsalPublicClient.AcquireTokenByUsernamePasswordCoreAsync(String[] scopes, String claims, String username, String password, String tenantId, Boolean enableCae, Boolean async, CancellationToken cancellationToken)\r\n
            at Azure.Identity.MsalPublicClient.AcquireTokenByUsernamePasswordAsync(String[] scopes, String claims, String username, String password, String tenantId, Boolean enableCae, Boolean async, CancellationToken cancellationToken)\r\n
            at Azure.Identity.UsernamePasswordCredential.AuthenticateImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)",
            "Source": "Microsoft.Identity.Client",
            "HResult": -2147467261,
            "Data": {},
            "InnerException": null
        }
    }
}

InnerException is null.
Are you certain that this code is not the culprit?

It would be helpful to know the specific value you are using for AZ_LOGIN_TENANT in the repro. If you don't want to disclose your actual tenant Id guid, please substitute it with a  placeholder.

const string AZ_LOGIN_TENANT = "asdf";

I updated the code in this issue yesterday am and called it out :)

Did you mean to type "adfs" by chance? 😄 If so, yes, I can reproduce this now and I agree it seems to be an issue with the MSAL library.
Please provide your repro details to the issue over there so that they can investigate further - https://github.com/AzureAD/microsoft-authentication-library-for-dotnet/issues/4860

Ha! Yes, I meant adfs :D
Ok, I will post my code there. Is your opinion to close this issue?

we'll keep it open until things resolve in the other issue.


Nominatim Help

Hello, Mr.Tananaev.
I run Debian 8.7, Nginx 1.6, Traccar 2.12 with traccar-web. Every thing works perfectly with the provided url address of Nominatim server in the configuration file. I get the addresses right and displayed in Bulgarian.
Recently I installed Nominatim 2.5 and Traccar 3.9. They are on the same machine.
When I point my browser to <IP_ADDRESS>/nominatim/ I am able to connect and view the page. In the upper right corner is "reverse search". When I press it I get http://<IP_ADDRESS>/nominatim/reverse.php?format=html. I assume Nominatim works.
Traccar 2.12 does not display address with my instance of Nominatim. With url of nominatim server in original file provided by you it works.
Traccar 3.9 does not display address with my instance of Nominatim.
With google it works. With my server - no. I tried with url of nominatim server which works with 2.12  but it does not work either. Settings in default.xml
    <entry key='geocoder.enable'>true</entry>
    <entry key='geocoder.type'>nominatim</entry>
    <entry key='geocoder.url'>http://localhost/nominatim/reverse.php</entry>

I do not know what other settings to check for reverse geocoding.

If you try it on 3.10 and it doesn't work, it should print some more details about the error in the logs.

Traccar 3.10 log:
2017-02-21 11:13:37  WARN: Geocoding failed - Unexpected char 69 at (line no=1, column no=1, offset=0) - JsonParsingExc$

It means that geocoder returned an invalid response.

Any suggestions what I should do?

I'm not sure, but you can probably find out by sending a request yourself from browser.

http://localhost/nominatim/reverse.php?format=html&lat=43&lon=28&zoom=18 returns: Варна, Varna Region, 9000, България (City)
43.2166104,27.9017131
http://localhost/nominatim/reverse.php?format=json&lat=43&lon=28&zoom=18 returns:
EOF
{"place_id":"100175","licence":"Data © OpenStreetMap contributors, ODbL 1.0. http://www.openstreetmap.org/copyright","osm_type":"node","osm_id":"31058194","lat":"43.2166104","lon":"27.9017131","display_name":"Варна, Varna Region, 9000, България","address":{"city":"Варна","county":"Varna Region","postcode":"9000","country":"България","country_code":"bg"}}
http://localhost/nominatim/reverse.php?format=xml&lat=43&lon=28&zoom=18 returns:
This page contains the following errors:
error on line 1 at column 1: Document is empty
Below is a rendering of the page up to the first error.

What does the EOF mean in JSON output?

I do not know. I just copied and pasted what the page dispalays.

Maybe that's your problem. It's not a valid JSON.


use std::function<void()> instead of global callbacks

Description
[Describe what this change achieves]
Issues Resolved
[List any existing issues this PR resolves; include Fixes #xxx or Closes #xxx (where xxx is issue number)]
Check List
General

[ ] Code follows coding style defined in STYLE.md
[ ] Doxygen comments are included inline with code
[ ] No unnecessary whitespace; check with git diff --check before committing.

The following have been modified to reflect new features, if warranted

[ ] README.md
[ ] keywords.txt (use tabs as whitespace separators)
[ ] library.properties
[ ] examples/ - update or create new ones, as warranted

The following have NOT been modified

[ ] doc/ - will be updated upon versioned release
[ ] .ruby-gemset
[ ] .ruby-version
[ ] CHANGELOG.md - will be updated upon versioned release (HISTORY.md is deprecated)
[ ] Gemfile
[ ] LICENSE
[ ] VERSION - will be updated upon versioned release

NOTE: use ?w=1 to see the changes and ignore whitespaces
https://github.com/4-20ma/ModbusMaster/pull/102/commits/eed84d3f9862b81cde2b28a80f73a05d01a9b7fa?w=1

can you merge this pool reqst?


Change the cookie serializer to :json

To avoid issues associated with the :marshal approach to cookies. This
application has already been using the :hybrid approach, so existing
cookies should already be in the :json format.

Merging this in as Chris is on annual leave.


EmbeddingHandler gets stuck trying to create embeddings for images

Hi Team,
When I try to create embeddigns for some pdf docuements, the program got stuck after some time. After some investigations, it appears that all teh blocks which were pending ( and where application got stuck), had content_type : "image" in the mongodb block document. Perhaps not trying to embed non text blocks should work , trying to do this in code.

Sorry you are running into this issue.   A couple of questions/ideas -

Which combination of embedding db and embedding model are you using?
Is there "text" in the image block?

Wondering if there is a problem combination of embedding db + embedding model if there is no text found in the block?

Hi Darren,
thanks for the response. I am using Milvus along with embedding model industry-bert-sec.
Also, there is no "text" in the image block, pasting the screenshot of one document which I think is stuck

It seems the code is stuck at the following place method in status.py, apparently to keep listening for embedding status:
def tail_embedding_status(self, library_name, model_name, poll_seconds=0.2):
Thread(target = self._tail_embedding_status, args = (library_name, model_name, poll_seconds)).start()
However, it gives appearance as if the code is stuck. This can be remedied by making thread as a daemon thread as below  so that the thread exits after the main thread has exited ( which shoudl be desired behaviour IMHO) :
def tail_embedding_status(self, library_name, model_name, poll_seconds=0.2):
   thread=Thread(target = self._tail_embedding_status, args = (library_name, model_name, poll_seconds))
   thread.daemon = True
   thread.start()

If that makes sense then I can raise a PR for the same.

Thanks for finding this issue and resolving it - your PR has been merged into the main branch.   Appreciate your contribution - and hopefully, the first of many!   :)

Thanks for the encouraging comments, I would certainly like to keep contributing. I have not personally found anything better than llmware framework on RAG yet, so I have decided to use it in my own project which is based on RAG.


Not found FastBarcodeScanner()

     FastBarcodeScanner(),           HoneywellBarcodeScanner(),           UnitechBarcodeScanner(),           BlueBirdBarcodeScanner(),           ZebraBarcodeScanner('my_profile'),
version combined_barcode_scanner: ^0.0.1-dev.1

There is an issue downstream that blocks us from fully releasing this package. You can use it using git tags, for example:
dependencies:
  combined_barcode_scanner: 0.0.1-dev.1
  combined_barcode_scanner_fast:
    git:
      url: git://github.com/icapps/combined_barcode_scanner.git
      path: pkgs/fast_barcode_scanner
  combined_barcode_scanner_unitech:
    git:
      url: git://github.com/icapps/combined_barcode_scanner.git
      path: pkgs/unitech
  combined_barcode_scanner_honeywell:
    git:
      url: git://github.com/icapps/combined_barcode_scanner.git
      path: pkgs/honeywell
  combined_barcode_scanner_zebra:
    git:
      url: git://github.com/icapps/combined_barcode_scanner.git
      path: pkgs/zebra_datawedge
  combined_barcode_scanner_blue_bird:
    git:
      url: git://github.com/icapps/combined_barcode_scanner.git
      path: pkgs/blue_bird_scanner
...

dependency_overrides:
  combined_barcode_scanner:
    git:
      url: git://github.com/icapps/combined_barcode_scanner.git
      ref: feature/6-extra-controller-methods
  fast_barcode_scanner:
    git:
      url: git://github.com/ikbendewilliam/fast_barcode_scanner.git
      path: fast_barcode_scanner
  fast_barcode_scanner_platform_interface:
    git:
      url: git://github.com/ikbendewilliam/fast_barcode_scanner.git
      path: fast_barcode_scanner_platform_interface

Waiting on https://github.com/jhoogstraat/fast_barcode_scanner/pull/51

We hoped it would be merged upstream by now 😅 this is a "temporary" workaround.

We've rereleased the fast_barcode_scanner so we can finally release this package fully. You can now update to version 1 and use this package without the overrides!

Thanks @ikbendewilliam


non-raw urls should support passwords in the url too

raw urls support having the password in the url, for example https://demo.modernpaste.com/paste/ejo5DpW974MJp~Bg4S0wzRQDRB9BZ42MRHb4Tt8RGHY=/raw?password=VDiL7sbcVr4vBMEpNWpF

but non-raw urls, for example, https://demo.modernpaste.com/paste/ejo5DpW974MJp~Bg4S0wzRQDRB9BZ42MRHb4Tt8RGHY=?password=VDiL7sbcVr4vBMEpNWpF

does not. that feels like a weird limitation, and it could come in handy to support it.
is there any specific reason for not supporting it? and if not, could support be added?

Passing a plain-text password as part of a URL is inherently insecure.
The reason that this is explicitly not supported in the non-raw view, or more accurately, the reason this is only supported in the raw view, is that the raw view does not provide any facilities for the user to securely enter a password. Passing the password in the URL is a bit of a hack for the raw view.
One possible additional feature is supporting passing the password as an HTTP header, rather than as a URL parameter. This can be supported in both the raw and non-raw views.

I don't think it should be an default or encouraged scheme, but I do think that silently supporting it would useful (I know it would be useful for me), but nevermind. As for the "password-in-header" idea, if someone wants to pursue it, I think it deserves it's own issue


Error HRESULT E_FAIL has been returned from a call to a COM component.

On Calling below mentioned method for getting list of available serial ports throws exception
"Error HRESULT E_FAIL has been returned from a call to a COM component."
Called Method
Windows.Devices.Enumeration.DeviceInformation.FindAllAsync(String aqsFilter, IEnumerable`1 additionalProperties)
Stack Strace
Windows.Devices.Enumeration.DeviceInformation.FindAllAsync(String aqsFilter, IEnumerable`1 additionalProperties)
This exception occurs 2-3 times on calling FindAllAsync method again and again, after few trials I get list of ports available.
How to solve this to get list of port on first time executing FindAllAsync method?
Code
public async Task GetAllDevices()
{
string aqs = Windows.Devices.SerialCommunication.SerialDevice.GetDeviceSelector();
// Find all COM ports
var dis = await Windows.Devices.Enumeration.DeviceInformation.FindAllAsync(aqs, null);
}
Please help if anyone knows about this error. Thank you.

Hi @urjapatel -- thank you for your comment. Can you tell me which page in our docs this refers to? (This will help me route your question to the correct author.) Thank you!

This refers to below mentioned link.
https://github.com/MicrosoftDocs/winrt-api/blob/docs/windows.devices.enumeration/deviceinformation_findallasync_1257462890.md
I hope this helps. Thank you.

I'm sorry your encountering this problem with this API. This repo is specifically for feedback on the docs, and it is not intended as a resource for technical support. I recommend that you ask this question on our forums or StackOverflow.


add unit tests

We need some unit tests. I think the file structure for test/unit/specs should mirror that of src/components. So the test for src/components/[component-group]/mdc-[component].vue would be test/unit/specs/[component-group]/mdc-[component].spec.js

agreed on the structure. I have not started Unit Test yet, mostly because  I don't know how karma/mocha works yet. but it'll be a must have before RC candidate so that we can update dependencies to MDC safely.
I wanted to finish a first draft of each components and go through one full review/refactoring first and then I guess unit doc and UT will be the step...

plan is to deploy tests with jest

need UT to validate foundation adapter's api
need UT to validate component APIs

npm run test merged in v0.0.49


PDF copy /paste lossign format (No space between paragraphs)

Attach (recommended) or Link to PDF file here:
Configuration:

Web browser and its version: Chrome
Operating system and its version: Windows 10
PDF.js version: Latest
Is a browser extension: No

Steps to reproduce the problem:

Open the sample document  https://mozilla.github.io/pdf.js/web/viewer.html
Copy the text and paste it to Notepad++ or MS word see no space between paragraphs

What is the expected behavior? (add screenshot)
Copy/Paste should retain formatting
What went wrong? (add screenshot)

Link to a viewer (if hosted on a site other than mozilla.github.io/pdf.js or as Firefox/Chrome extension):
https://mozilla.github.io/pdf.js/web/viewer.html

I tried in Edge and in Acrobat and there are no space between paragraphs with them.
In Acrobat, the -\n are striped out and with Edge they're replaced by \u0002 (STX).

Please note that in general there's no concept of e.g. paragraphs in PDF documents[1], and that glyphs are absolutely positioned in PDF documents.
Hence I'm not sure that this, again generally, can actually work like you want and it thus seem (to me at least) that this works as intended here and that the issue thus could/should be closed.

[1] Possibly with the exception of tagged PDF documents, which doesn't apply to the tracemonkey.pdf file.


Add SQL format with auto add

In HIVE, many SQL functions cannot be highlighted and formatted. Currently,"," is used for line breaking. It is not flexible enough.Thank you very much!

Seems to be duplicate of #6439


REP/REPIN table

Ich habe gerade RAREFAN laufen lassen.  Für jeden Strain eine Spalte und REP/REPINs differenziert ausgeben: #REPINs(#REPs)

Do you want to report #REPs always or only when "Analyze REPINs" is switched off? And do you want to report #REP(IN)s for each group or only the sum? would be helpful to see an example of the table you have in mind.

yes, I think we can report REPs always. we can report REPINs for each group. We can talk about it again if you like

"Analyze Repins" unticked:

"Analyze Repins" ticked:

@fredeBio, please comment

I think we need to write REP singlets and REPINs when the REPIN option is ticked and otherwise we should not display the REPIN part. Is that possible? (just to make sure you have chosen the allREPs and allREPINs columns right?)

looks good!

Fixed by 194c6f6


Switch runtime.project.json to depend on MSBuild packages built during the build, and restore runtime/dependencies to separate deployment folder

#336 gets us part of the way to this

I think this'll go away post-SDK conversion.

I don't think this is particularly meaningful any more; marking as complete.


Support other sites on the StackExchange network (not just StackOverflow)

Currently the code seems to be written to only work for StackOverflow, rather than any site in the StackExchange network:
https://github.com/bramses/obsidian-stack-overflow/blob/4f2a556dcb7d4a33fc062ecb2234001ccdf15581/main.ts#L113-L116
https://github.com/bramses/obsidian-stack-overflow/blob/4f2a556dcb7d4a33fc062ecb2234001ccdf15581/main.ts#L85
It would be cool if it worked for any/all of the StackExchange sites:

https://stackexchange.com/sites

A number of these seem to be in the format: https://FOO.stackexchange.com/, though it seems that there are also a few that use their own custom domains
This answer seems to talk a bit more about it:

https://meta.stackexchange.com/questions/81379/can-we-have-a-list-of-all-the-stack-exchange-domains-somewhere-for-firewall-pur/81383#81383

As well as suggesting that the API can also provide this information:

You can get a complete (and always up to date) list of Stack Exchange site URLs from the API.

By tweaking the default filter used we can reduce the results to just returning the site name, site_url, api_site_parameter, etc that would be required to handle each in an easy way:

https://api.stackexchange.com/docs/sites#filter=!3yGYnsx2qxoV*e)DY&run=true

A live example of the output from this can be seen at:

https://api.stackexchange.com/2.3/sites?filter=!b.9PNkK9eppDdb

{
items: [
{
aliases: [
"https://www.stackoverflow.com/",
"https://facebook.stackoverflow.com/"
],
site_url: "https://stackoverflow.com/",
api_site_parameter: "stackoverflow",
name: "Stack Overflow"
},
{
site_url: "https://serverfault.com/",
api_site_parameter: "serverfault",
name: "Server Fault"
},
{
site_url: "https://superuser.com/",
api_site_parameter: "superuser",
name: "Super User"
},
{
site_url: "https://meta.stackexchange.com/",
api_site_parameter: "meta",
name: "Meta Stack Exchange"
},
// ...etc...

This is a great idea! It seems they all follow a similar structure as well, so it might be straightforward in doing similar HTML parsing techniques

Yeah, that was my thoughts too. I hadn't actually looked deeper into the parsing side of things, but I assumed they're all deployed on basically the same core 'product', just with different URLs/styling/etc.
As an MVP hack it might be enough to just swap out the couple of hardcoded references for one of the other sites and see if still works.
As for supporting all the sites, not sure the best approach for it, but I would probably hit the API to get the data, and then cache that somewhere (I assume obsidian plugins can support something like that?) so that it wouldn't need to be looked up each time. Alternately it could be 'baked in' to the code, but then it would need to be updated if/when any of the sites change, which might be more maintenance effort than it's worth.


Wrong extension of the windows executable file

From<EMAIL_ADDRESS>on 2010-12-15T13:15:17Z
I've just downloaded and installed this file https://code.google.com/p/spyderlib/downloads/detail?name=spyder-2.0.4_py26.exe The installation process removed the existing C:\Python26\Scripts\spyder.pyw file and replaced it with C:\Python26\Scripts\spyder (note no extension). Launching spyder.bat opens both the Spyder IDE and a cmd (Windows console) window.  To fix this, I renamed the "spyder" file to "spyder.pyw" and now I'm able to launch Spyder without the cmd window.
Original issue: http://code.google.com/p/spyderlib/issues/detail?id=505

From pierre.raybaut on 2010-12-15T10:24:19Z
See: http://groups.google.com/group/spyderlib/t/7a9e2f212b722e54
Status: Done


Using multiple optimizers for the same architecture

Is there a way to use two different optimizers for different parts of the architecture without repeating the training step twice?

https://pytorch-lightning.readthedocs.io/en/latest/lightning-module.html#configure-optimizers
no, but we typically need to do different things for each optimizer, therefore we need to call training_step multiple times with a different optimizer_idx as argument.
If you can provide some pseudo code, I may better understand what you want to do and I can try help convert it to Lightning.

@CuauLuzbel did I answer your question? Let me know if anything is unclear

Hi @awaelchli,
My requirements are similar to this issue and also this one. The linked issue already has the pseudo-code. It would help a lot of us if you could share how to solve this in the lightning framework.

Hi @awaelchli,
My requirements are similar to this issue and also this one. The linked issue already has the pseudo-code. It would help a lot of us if you could share how to solve this in the lightning framework.

First, check out the manual optimization docs, there you will already find a GAN example.
There you have two parts, a discriminator and a generator, just as in the other post you mention.
You can call step and manual backward as you wish. It's almost identical to what you would do in pure pytorch.

First, check out the manual optimization docs, there you will already find a GAN example.
There you have two parts, a discriminator and a generator, just as in the other post you mention.
You can call step and manual backward as you wish. It's almost identical to what you would do in pure pytorch.

My interpretation of what the other poster wanted to do:
@property
def automatic_optimization(self):
    return False

def training_step(self, batch, batch_idx):
    out = self.first_module(batch)
    out = self.second_module(out)

    opt1, opt2 = self.optimizes()

    loss = criterion(out, batch['target'])

    self.manual_backward(loss)
    opt1.step()
    opt2.step()

def configure_optimizers(self):
    optimizer_first_module = torch.optim.Adam(self.first_module.parameters())
    optimizer_second_module = torch.optim.Adam(self.second_module.parameters())
    return [optimizer_first_module, optimizer_second_module]

I didn't run this code, there may be a typo or two. I hope it makes a bit more sense?

My interpretation of what the other poster wanted to do:
@property
def automatic_optimization(self):
    return False

def training_step(self, batch, batch_idx):
    out = self.first_module(batch)
    out = self.second_module(out)

    opt1, opt2 = self.optimizes()

    loss = criterion(out, batch['target'])

    self.manual_backward(loss)
    opt1.step()
    opt2.step()

def configure_optimizers(self):
    optimizer_first_module = torch.optim.Adam(self.first_module.parameters())
    optimizer_second_module = torch.optim.Adam(self.second_module.parameters())
    return [optimizer_first_module, optimizer_second_module]

I didn't run this code, there may be a typo or two. I hope it makes a bit more sense?

Thanks @awaelchli, this looks very promising. There is one small issue though. Currently, calling just self.manual_backward(loss) is not allowed within lightning as we need to pass an optimizer as well (required arg), i.e., call self.manual_backward(loss, opt1) instead. I checked the code for manual_backward - it uses the optimizer param to clip the gradients and synchronize the optimizer (during DDP I think).
From the docs, one way to override the backward call is to redefine the LightningModule.backward() function to just perform loss.backward() and ignore any optimizers passed. That'll override the optimizer specific code being run within manual_backward. But I'd love to hear your thoughts as I'm quite new to the framework. Is this the cleanest solution to the problem?

Thanks @awaelchli, this looks very promising. There is one small issue though. Currently, calling just self.manual_backward(loss) is not allowed within lightning as we need to pass an optimizer as well (required arg), i.e., call self.manual_backward(loss, opt1) instead. I checked the code for manual_backward - it uses the optimizer param to clip the gradients and synchronize the optimizer (during DDP I think).
From the docs, one way to override the backward call is to redefine the LightningModule.backward() function to just perform loss.backward() and ignore any optimizers passed. That'll override the optimizer specific code being run within manual_backward. But I'd love to hear your thoughts as I'm quite new to the framework. Is this the cleanest solution to the problem?

same question asked on slack: https://pytorch-lightning.slack.com/archives/CRBLFHY79/p1610233885334100
Colab demo with a possible solution:
https://colab.research.google.com/drive/1jVPI6as9gBCRxdu7r1Q6RvYu2Jh08OKJ?usp=sharing

same question asked on slack: https://pytorch-lightning.slack.com/archives/CRBLFHY79/p1610233885334100
Colab demo with a possible solution:
https://colab.research.google.com/drive/1jVPI6as9gBCRxdu7r1Q6RvYu2Jh08OKJ?usp=sharing

my issue is slightly different, I simply have two parts of the same network trained with two different learning rates, e.g., first K layers with one LR and last N layers with another one. However the input and the loss I would use for the backward pass would be the exact same, so it is a waste of computation to pass the same input through the network twice. Can this scenario be implemented in lightning or I need to resort to the manual training method? Thanks :)

@pasinit In PyTorch, you can do one optimizer with learning rates per parameter groups: https://pytorch.org/docs/stable/optim.html#per-parameter-options
I would go this way unless there are more specific reasons for needing multiple optimizers in this use case.

same question asked on slack: https://pytorch-lightning.slack.com/archives/CRBLFHY79/p1610233885334100 Colab demo with a possible solution: https://colab.research.google.com/drive/1jVPI6as9gBCRxdu7r1Q6RvYu2Jh08OKJ?usp=sharing

While this solution is awesome, and clearly worked at the time, this isn't the case anymore. Pytorch-lightning has changed the optimizer-step to now work with closures and putting the call to training_step inside the closure. Failing to call said closure incurs an error: MisconfigurationException: The closure hasn't been executed.
There are now still 2 workarounds :

Write your own custom training loop inside training_step by setting self.automatic_optimization to False.
Wrap the 2 optimizers inside one new 'hybrid' optimizer (this also necessitates a wrapper around lr_scheduler if you want to apply a different lr_scheduler to each optimizer)

I prefer solution 2 since it avoids supplementary coding to support all the pytorch-lightning parameters like gradient accumulation. It however asks to add 2 classes to our code like so (this code was tested and works even when resuming training from a checkpoint):
import torch
from collections import defaultdict
class HybridOptim(torch.optim.Optimizer):
    # Wrapper around multiple optimizers that should be executed at the same time
    def __init__(self, optimizers):
        self.optimizers = optimizers

    @property
    def state(self):
        state = defaultdict(dict)
        for optimizer in self.optimizers:
            state = {**state, **optimizer.state}
        return state

    @property
    def param_groups(self):
        param_groups = []
        for optimizer in self.optimizers:
            param_groups = param_groups + optimizer.param_groups
        return param_groups

    def __getstate__(self):
        return [optimizer.__getstate__() for optimizer in self.optimizers]

    def __setstate__(self, state):
        for opt_state, optimizer in zip(self.optimizers, state):
            optimizer.__setstate__(opt_state)

    def __repr__(self):
        format_string = self.__class__.__name__ + ' ('
        for optimizer in self.optimizers:
            format_string += '\n'
            format_string += optimizer.__repr__()
        format_string += ')'
        return format_string

    def _hook_for_profile(self):
        for optimizer in self.optimizers:
            optimizer._hook_for_profile()

    def state_dict(self):
        return [optimizer.state_dict() for optimizer in self.optimizers]

    def load_state_dict(self, state_dict):
        for state, optimizer in zip(state_dict, self.optimizers):
            optimizer.load_state_dict(state)

    def zero_grad(self, set_to_none: bool = False):
        for optimizer in self.optimizers:
            optimizer.zero_grad(set_to_none=set_to_none)

    def add_param_group(self, param_group):
        raise NotImplementedError()

    def step(self, closure=None):
        loss = None
        if closure is not None:
            with torch.enable_grad():
                loss = closure()

        for optimizer in self.optimizers:
            optimizer.step()

        return loss

class HybridLRS():
    """ Wrapper Class around lr_scheduler to return a 'dummy' optimizer to pass
        pytorch lightning checks
    """
    def __init__(self, hybrid_optimizer, idx, lr_scheduler) -> None:
        self.optimizer = hybrid_optimizer
        self.idx = idx
        self.lr_scheduler = lr_scheduler

    def __getattribute__(self, __name: str):
        if __name in {"optimizer", "idx", "lr_scheduler"}:
            return super().__getattribute__(__name)
        else:
            return self.lr_scheduler.__getattribute__(__name)

class PLModule():
    def configure_optimizers(self):
        # Classic initialization
        optimizer1 = ...
        optimizer2 = ...
        lrs1 = LRS(optimizer1, ...)
        lrs2 = LRS(optimizer2, ...)
        hoptim = HybridOptim([opt1, opt2])
        return [hoptim], [HybridLRS(hoptim, 0, lrs1), HybridLRS(hoptim, 1, lrs2)]

Hope this can help people who also struggled with this issue :) (this code should not break even during future pytorch-lightning updates since it works independently of any specifics of the framework)

@vanderschuea very nice solution, I appreciate you sharing it here.  Sorry to bump an old thread like this, but it seems version 1.9.0 kills your approach.  I get the error:

lightning_fabric.utilities.exceptions.MisconfigurationException: Unknown configuration for model optimizers.

Rolling back to 1.8.6 gets my code to run as expected.
Wanted to comment an update here because AFAIK this is still the most useful thread online about how to use different optimizers in one architecture with Lightning.

Updating the code to include a defaults property seems to fix the issue in PyTorch Lightning 2.0. I'll post my working code below:
class HybridOptim(torch.optim.Optimizer):
    """
    Wrapper around multiple optimizers that should be stepped together at a single time. This is
    a hack to avoid PyTorch Lightning calling ``training_step`` once for each optimizer, which
    increases training time and is not always necessary.

    Modified from the reply in a GitHub Issue thread here:
    https://github.com/Lightning-AI/lightning/issues/3346#issuecomment-1036063687

    Parameters
    ----------
    optimizers: list of optimizers

    """
    def __init__(self, optimizers: Iterable[torch.optim.Optimizer]) -> None:
        self.optimizers = optimizers

    @property
    def state(self) -> Dict[str, torch.Tensor]:
        """Return the combined state for each optimizer in ``self.optimizers``."""
        return {
            key: value
            for optimizer in self.optimizers
            for key, value in optimizer.state.items()
        }

    @property
    def param_groups(self) -> List[Dict[str, Union[torch.Tensor, float, bool, Any]]]:
        """Return the combined parameter groups for each optimizer in ``self.optimizers``."""
        return [
            element
            for optimizer in self.optimizers
            for element in optimizer.param_groups
        ]

    # !!!!!! HERE IS THE NEW CODE !!!!!!!!!
    @property
    def defaults(self) -> Dict[str, torch.Tensor]:
        """Return the combined defaults for each optimizer in ``self.optimizers``."""
        return {
            key: value
            for optimizer in self.optimizers
            for key, value in optimizer.defaults.items()
        }
    # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

    def __getstate__(self) -> List[torch.optim.Optimizer]:
        """Return ``self.optimizers`` for pickling purposes."""
        return self.optimizers

    def __setstate__(self, optimizers: List[torch.optim.Optimizer]) -> None:
        """
        Load ``optimizers`` into ``self.optimizers`` for pickling purposes and call
        ``__setstate__``.

        """
        self.optimizers = optimizers

        # call remaining lines of the ``torch.optim.Optimizer.__setstate__`` method just to be safe.
        # copied from: https://pytorch.org/docs/stable/_modules/torch/optim/optimizer.html#Optimizer
        for optimizer in self.optimizers:
            optimizer._hook_for_profile()  # To support multiprocessing pickle/unpickle.
            optimizer.defaults.setdefault('differentiable', False)

    def __repr__(self) -> str:
        """Call and concatenate ``__repr__`` for each optimizer in ``self.optimizers``."""
        repr_str = f'``{self.__class__.__name__}`` containing {len(self.optimizers)} optimizers:\n'

        for optimizer in self.optimizers:
            repr_str += '\n' + optimizer.__repr__()

        return repr_str

    def _hook_for_profile(self) -> None:
        """Call ``_hook_for_profile`` for each optimizer in ``self.optimizers``."""
        for optimizer in self.optimizers:
            optimizer._hook_for_profile()

    def state_dict(self) -> List[
        Dict[str, Union[torch.Tensor, List[Dict[str, Union[torch.Tensor, float, bool, Any]]]]]
    ]:
        """
        Returns the state of the optimizer as a dictionary.

        It contains two entries:

            * ``state`` - a dict holding current optimization state. Its content differs between
              optimizer classes.
            * ``param_groups`` - a list containing all parameter groups where each parameter group
              is a dict

        """
        return [optimizer.state_dict() for optimizer in self.optimizers]

    def load_state_dict(
        self,
        state_dict: List[
            Dict[str, Union[torch.Tensor, List[Dict[str, Union[torch.Tensor, float, bool, Any]]]]]
        ],
    ) -> None:
        """
        Loads the optimizer state.

        Parameters
        ----------
        state_dict: dict
            Optimizer state. Should be an object returned from a call to ``state_dict()``

        """
        for state, optimizer in zip(state_dict, self.optimizers):
            optimizer.load_state_dict(state)

    def zero_grad(self, set_to_none: bool = False) -> None:
        """
        Sets the gradients of all optimized ``torch.Tensor``s to zero.

        Parameters
        ----------
        set_to_none: bool
            Instead of setting to zero, set the grads to ``None``. This will in general have lower
            memory footprint, and can modestly improve performance. However, it changes certain
            behaviors. For example:

                1. When the user tries to access a gradient and perform manual ops on it, a ``None``
                   attribute or a ``torch.Tensor`` full of ``0``s will behave differently.

                2. If the user requests ``zero_grad(set_to_none=True)`` followed by a backward pass,
                   ``.grad``s are guaranteed to be ``None`` for params that did not receive a
                   gradient.

                3. ``torch.optim`` optimizers have a different behavior if the gradient is ``0`` or
                   ``None`` (in one case it does the step with a gradient of ``0`` and in the other
                   it skips the step altogether).

        """
        for optimizer in self.optimizers:
            optimizer.zero_grad(set_to_none=set_to_none)

    def step(self, closure: Callable[[], torch.Tensor] = None) -> torch.Tensor:
        """
        Performs a single optimization step (parameter update).

        Parameters
        ----------
        closure: function
            A closure that reevaluates the model and returns the loss. Optional for most optimizers.

        Notes
        -----
        Unless otherwise specified, this function should not modify the ``.grad`` field of the
        parameters.

        """
        loss = None

        if closure is not None:
            with torch.enable_grad():
                loss = closure()

        for optimizer in self.optimizers:
            optimizer.step()

        return loss

class HybridLRS(torch.optim.lr_scheduler._LRScheduler):
    """
    Wrapper class around ``lr_scheduler``s to return a dummy optimizer to pass PyTorch Lightning
    checks.

    Modified from the reply in a GitHub Issue thread here:
    https://github.com/Lightning-AI/lightning/issues/3346#issuecomment-1036063687

    Parameters
    ----------
    hybrid_optimizer: HybridOptim
    lr_scheduler: torch.optim.lr_scheduler._LRScheduler
    idx: int
        Index of the optimizer in ``hybrid_optimizer`` the learning rate scheduler ``lr_scheduler``
        is assigned to

    """
    def __init__(
        self,
        hybrid_optimizer: HybridOptim,
        lr_scheduler: torch.optim.lr_scheduler._LRScheduler,
        optimizer_idx: int,
    ) -> None:
        self.optimizer = hybrid_optimizer
        self.lr_scheduler = lr_scheduler
        self.idx = optimizer_idx

    def __getattribute__(self, __name: str) -> Any:
        """
        If the attribute name is one of ``optimizer``, ``idx``, or ``lr_scheduler``, return this
        class's attribute with the same name, else return the ``lr_scheduler``'s attribute with
        that name.

        """
        if __name in {'optimizer', 'lr_scheduler', 'idx'}:
            return super().__getattribute__(__name)
        else:
            return self.lr_scheduler.__getattribute__(__name)


[CT-3254] Remove hard-coded adapter logic from tests for date_spine macros

Housekeeping

[X] I am a maintainer of dbt-core

Short description
@jwills asked within https://github.com/dbt-labs/dbt-core/discussions/8307#discussioncomment-7373642 if we can make it easier for adapter maintainers to specify their preferred date format within the tests for date_spine and supporting macros.
Currently, there is hardcoding of target.type == 'postgres' etc. whose logic that we want in adapter implementations rather than in dbt-core.
Acceptance criteria

there is no hard-coded adapter names (like target.type == 'postgres')
there is a clear and easy way for adapter maintainers to customize the test cases if their data platform doesn't support the syntax in the base adapter

Impact to Other Teams
This change directly affects dbt-bigquery, dbt-postgres, and dbt-redshift that may need to accommodate any change. The appropriate Impact: Adapters label has been added.
Will backports be required?
Backport to 1.7.latest. The backport 1.7.latest label has been added.
Context
See dbt-labs/dbt-core#8172, dbt-labs/dbt-core#8616, and https://github.com/dbt-labs/dbt-core/discussions/8307#discussioncomment-7373642 for background context.
See dbt-labs/dbt-adapters#192 for proposed cross-database to_date macro that is intended to make test customization on a per-adapter basis either completely unnecessary or exceedingly rare.

Moving to dbt-adapters since the relevant tests have also moved there:

fixture_date_spine


WIP: Style Block decorations

I started playing around with the block decorations. The next few days I will work a lot with hydrogen so I can test this PR and probably I will find some more styling problems. What do you think about the styling so far?
New:

Master:

I like the shadow in the multiline bubbles. Could we have them as well in the in the single line bubbles too?

Think this is fit to ship now?

How does it work? Is it the float:left what makes the block decoration to be on the same line?

@rgbkrk There are just some minor inconsistencies but nothing major. If you like you can merge and I fix these in a future PR.
@n-riesco It's the negative margin-top in combination with float: left. I just played around with the values until it worked ;).

Neat!

We fit to take this in now @n-riesco?

I just noticed a little problem with the watches panel. It'll be ready in a few minutes ;)

Ah, sorry. I noticed the [WIP] tag coming off and assumed that was you stating intent. :smile:

I'm  good with merging it!
You are right. At the time I removed the [WIP] I thought it was ready. :laughing:

@lgeiger Check the SVG example you gave me.
I gotta go offline for a while.

@n-riesco The SVG output looks good to me.

@lgeiger I've tested this branch and works well in my machine.
PS: The problem I saw before was actually caused by the beta version of the IJavascript kernel.

@n-riesco no problem. Thanks for the testing.

Anything else we want to do before we make a release?
Were people willing to try out zmq-prebuilt? It has prebuilds for Linux, OS X, and 32-bit Windows. 64-bit Windows users still have to build (as they would in the past too).

@rgbkrk This is now just a question of replacing the jmp dependency with jmp-prebuilt.


Help wanted: per-project code coverage

The repo generates code coverage reports both for the individual libraries and for the repo as a whole. At the moment it only seems possible to present a badge/shield for the repo as a whole, but I'd like to show code coverage for the individual libraries.
This suggests it's not yet feasible: https://github.com/lemurheavy/coveralls-public/issues/1174
No other open-source projects I've found have per-project code coverage numbers, but if anybody knows of one that does then I'd love to see it.

Hi, I came across your question when searching for a solution to this issue and I have found a workaround, so I thought I'd leave it here as I'm probably not the only one who finds their way here.
I'm using the fact that coveralls.io allows to track the coverage per branch to generate badges for individual libraries/packages using GitHub Actions:
name: Run tests with pytest
env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    COVERALLS_FLAG_NAME: package
    REPORT_PACKAGE: ${{ github.ref == 'refs/heads/master' }}
run: |
    set -e
    cd ./to/the/package-root

    pip install -r requirements-test.txt
    pytest -v --cov=package tests/

    # when on master, pretend to be on branch x-cov-package
    $REPORT_PACKAGE && export GITHUB_REF="refs/heads/x-cov-package"
    $REPORT_PACKAGE && export GITHUB_HEAD_REF="x-cov-package"
    coveralls --service=github

You need to permanently keep a branch x-cov-package for coveralls.io to keep the data and provide a badge at https://coveralls.io/github/your-org/your-repo?branch=x-cov-package (I named them x-cov-* to be able to identify and protect them from deletion.)
It's not an ideal solution, as it depends on the internals of the particular coveralls Python client, but it should be fairly stable and easy to port to other coveralls clients.
You can see the full setup in action at https://github.com/XAI-Demonstrator/xai-demonstrator.

Hi, I came across your question when searching for a solution to this issue and I have found a workaround, so I thought I'd leave it here as I'm probably not the only one who finds their way here.
I'm using the fact that coveralls.io allows to track the coverage per branch to generate badges for individual libraries/packages using GitHub Actions:
name: Run tests with pytest
env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    COVERALLS_FLAG_NAME: package
    REPORT_PACKAGE: ${{ github.ref == 'refs/heads/master' }}
run: |
    set -e
    cd ./to/the/package-root

    pip install -r requirements-test.txt
    pytest -v --cov=package tests/

    # when on master, pretend to be on branch x-cov-package
    $REPORT_PACKAGE && export GITHUB_REF="refs/heads/x-cov-package"
    $REPORT_PACKAGE && export GITHUB_HEAD_REF="x-cov-package"
    coveralls --service=github

You need to permanently keep a branch x-cov-package for coveralls.io to keep the data and provide a badge at https://coveralls.io/github/your-org/your-repo?branch=x-cov-package (I named them x-cov-* to be able to identify and protect them from deletion.)
It's not an ideal solution, as it depends on the internals of the particular coveralls Python client, but it should be fairly stable and easy to port to other coveralls clients.
You can see the full setup in action at https://github.com/XAI-Demonstrator/xai-demonstrator.

That's clever! Thank you for the info -- I'll give it a try

That's clever! Thank you for the info -- I'll give it a try


where to input offset?

What do you mean?

It would just be sigs I believe. I can't seem to fix the injection crash yet so I can't really assist.
I know someone who has the full fixed source but says that source is private. Why? I have no clue why a pasted source would be private.

if the CS2 updated today or yesterday? i inject it yesterday.

I believe it updated yesterday. Not 100% sure.
I get too the part where it says '256' schema and then it crashes.
When debugging it shows the crash here
HashFixedDataInternal_t<tData>* GetNext() {     return *reinterpret_cast<HashFixedDataInternal_t<tData>**>(reinterpret_cast<std::uintptr_t>(this) + (0x20)); }

soon

https://kip.lanzouq.com/b00b3k10jg
密码（pwd）:fw9i

I believe it updated yesterday. Not 100% sure.
I get too the part where it says '256' schema and then it crashes.
When debugging it shows the crash here HashFixedDataInternal_t<tData>* GetNext() { return *reinterpret_cast<HashFixedDataInternal_t<tData>**>(reinterpret_cast<std::uintptr_t>(this) + (0x20)); }

Did you magage to fix it?

still work.just inject,not need update.17/04/2024 01:30 GMT+8

It doesn't after I build it from the source

extreme and xenos with multiple settings (no scrambling)

@AMXBus sadly no I haven't fixed yet

maybe the question is your injector

我是基址获取有问题，不知道怎么回事

run as admin and add -allow_third_partsoftware

run as admin and add -allow_third_partsoftware

do as follows,but still cannot get the offsets
debug control output:
[19-04-2024 17:43:45] [memory.cpp:370] [error] pattern not found: "48 89 5C 24 00 48 89 6C 24 00 56 57 41 56 48 81 EC 00 00 00 00 48 8D 0D"

It's a very interesting situation.
If you compile a new source code, the size of the dll will be: 3,631 KB. And if you download the new DLL, the size is already 7,492 KB. The source breaks the game and doesn't start. But the released dll works). It's very interesting that you added something that you didn't insert into the source code?))

Mb virus?)

It's a very interesting situation.
If you compile a new source code, the size of the dll will be: 3,631 KB. And if you download the new DLL, the size is already 7,492 KB. The source breaks the game and doesn't start. But the released dll works). It's very interesting that you added something that you didn't insert into the source code?))

Mb virus?)

XD, did you successfully compiled the full sour

It's a very interesting situation.
If you compile a new source code, the size of the dll will be: 3,631 KB. And if you download the new DLL, the size is already 7,492 KB. The source breaks the game and doesn't start. But the released dll works). It's very interesting that you added something that you didn't insert into the source code?))

Mb virus?)

 For the cheat to work, you need to update all the signatures and schemas of the cheat. If you didn't even build the code completly, do not comment about it being a virus.  Being only 3600KB is the reason it doesn't start for you and keep crashing.

so the source code is lagging behind the latest version of the dll?'

It's a very interesting situation.
If you compile a new source code, the size of the dll will be: 3,631 KB. And if you download the new DLL, the size is already 7,492 KB. The source breaks the game and doesn't start. But the released dll works). It's very interesting that you added something that you didn't insert into the source code?))

Mb virus?)

 For the cheat to work, you need to update all the signatures and schemas of the cheat. If you didn't even build the code completly, do not comment about it being a virus. Being only 3600KB is the reason it doesn't start for you and keep crashing.

so the source code is lagging behind the latest version of the dll?'

It's a very interesting situation.
If you compile a new source code, the size of the dll will be: 3,631 KB. And if you download the new DLL, the size is already 7,492 KB. The source breaks the game and doesn't start. But the released dll works). It's very interesting that you added something that you didn't insert into the source code?))

Mb virus?)

 For the cheat to work, you need to update all the signatures and schemas of the cheat. If you didn't even build the code completly, do not comment about it being a virus. Being only 3600KB is the reason it doesn't start for you and keep crashing.

Not really, but I released the source code for coders to build their own cheats and get hooks and other requirements from this repository. (Basically as inspiration source)
I won't release the updated source code because I don't want to see 1:1 clones of this cheat everywhere on the internet.

Xd

so the source code is lagging behind the latest version of the dll?'

It's a very interesting situation.
If you compile a new source code, the size of the dll will be: 3,631 KB. And if you download the new DLL, the size is already 7,492 KB. The source breaks the game and doesn't start. But the released dll works). It's very interesting that you added something that you didn't insert into the source code?))

Mb virus?)

 For the cheat to work, you need to update all the signatures and schemas of the cheat. If you didn't even build the code completly, do not comment about it being a virus. Being only 3600KB is the reason it doesn't start for you and keep crashing.

Not really, but I released the source code for coders to build their own cheats and get hooks and other requirements from this repository. (Basically as inspiration source) I won't release the updated source code because I don't want to see 1:1 clones of this cheat everywhere on the internet.

Xd


Unify output tensor allocation across all callers and devices

This PR unifies the output tensor frame allocation within a single util: allocateEmptyHWCTensorForStream.
All high-level decoding entry-points are now responsible for allocating that output tensor.
All lower-level decoding entry-points now accept and expect this pre-allocated tensor to be passed. These low-level entry-points do not allocate anything that gets returned.
TODO, as a follow-up: try to fold the preAllocatedOutputTensor parameter within the rawOutput struct.

I've discussed this PR with both of you @ahmadsharif1 @scotts and the conclusion is that bubbling-up the tensor allocation might not be the best move at this time, because it would change the source of where we retrieve height and width from.
I'll follow-up soon with some alternative improvements.


Adding segregation of resultType for CloudError

Which issue this PR addresses:

Fixes ARO-5112
What this PR does / why we need it:

As of now, all CloudErrors are termed as UserError.  This PR will check upon the statusCode of the error and provide resultType accordingly.
Test plan for issue:

Done : Manually Tested mapping of CloudError StatusCode with ResultType.
Is there any documentation that needs to be updated for this PR?

No

Looks good but I have a small suggestion. Would we consider putting this into the CloudError itself?
We could have a:
func (err *CloudError) GetResultType() ResultType {...}

This way the logic of mapping stausCode to ResultType lives in one place i.e. the thing that holds the statusCode.

/azp run e2e

Looks good but I have a small suggestion. Would we consider putting this into the CloudError itself?
We could have a:
func (err *CloudError) GetResultType() ResultType {...}

This way the logic of mapping stausCode to ResultType lives in one place i.e. the thing that holds the statusCode.

Putting a high level view on the suggestion: To include ResultType as part of or use it in ClousError, github.com/Azure/ARO-RP/pkg/util/log will have to be imported in pkg/api/error.go which might create a circular dependency leading to import cycle.
Another option would be to have ResultType completely under CloudError and defined in pkg/api/error.go itself but that might not be totally accurate as ResultType is meant to be part of logging & thus placed in pkg/util/log.
I might be missing on something, do help me with further suggestions.

Putting a high level view on the suggestion: To include ResultType as part of or use it in ClousError, github.com/Azure/ARO-RP/pkg/util/log will have to be imported in pkg/api/error.go which might create a circular dependency leading to import cycle. Another option would be to have ResultType completely under CloudError and defined in pkg/api/error.go itself but that might not be totally accurate as ResultType is meant to be part of logging & thus placed in pkg/util/log. I might be missing on something, do help me with further suggestions.

No stress. We could put something in log that maps statusCode int to ResultType instead. IMO so long as this logic is in one place I think that's a win. We can do this as part of the pairing on Friday actually (or as a separate PR) since it's gonna branch out beyond this PR

Can we please add a few test cases coverage here? Thank you.

LGTM


Removed dependency from commands module

This PR fixes #51

closed PR as not supposed to go to master


feat: set #[track_caller] on public panicable functions

This attribute sets the panic info location to the caller's location, see https://blog.rust-lang.org/2020/03/12/Rust-1.42.html#useful-line-numbers-in-option-and-result-panic-messages for more information.

Should
https://github.com/twilight-rs/twilight/blob/3fe3a1d9905d189687417f6a2f25289f7551dc1a/lavalink/src/http.rs#L299-L302
not also be marked?

#1787


Feature/472 parsekey quotekey

fix #472

Coverage decreased (-0.01%) to 92.989% when pulling f7ad6b49f6231025d10268a8afa6f19cd0b38410 on im-kulikov:feature/472_parsekey-quotekey into 95c94a9cfda2b5735b0fe82f9bf9173ef3ffce7d on 75team:master.


Docs: Events

Add docs about auth events:

auth:beforeLogin

auth:afterLogin

auth:beforeLogout

auth:afterLogout

https://github.com/sinbadxiii/phalcon-auth/commit/0f51a1228952c5a342bd274a17f3a1cd8eb2838d


Added Hyperf\Coordinator\Timer.

Is it duplicate with #1727 ?

Is it duplicate with #1727 ?

嗯是的。。


fix: unauthorized users for groups and permissions are not redirected to the previous url

Fixes #798

The option for adding arguments to filter configuration has been added to CI v4.4.0, see docs.
This PR might need to be updated to either:

Update the docs and encourage developers to upgrade to v4.4.0
Maintain this PR to cover all use cases.

WDYT?

The option for adding arguments to filter configuration has been added to CI v4.4.0, see docs.

Yes, I remembered, I had reviewed this one😀.

Update the docs and encourage developers to upgrade to v4.4.0

Shield requires CodeIgniter 4.2.7+ at this point I am not very inclined to upgrade to 4.4.0.
The difference between CodeIgniter 4.2.7+ and 4.4.0 is huge, 4.4.0 is great, but I think we shouldn't limit the user community at this point.

Hi @kenjis, @lonnieezell, @MGatner  can you take a look?

@sammyskills is aware of this, I haven't played with the code yet, I'll try to do it tomorrow.


Match run_exports of ncurses

Checklist

[ ] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

https://gist.github.com/isuruf/dfea93249237080356fd9ea8eedc050a

https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/2383 was the one that made me do this PR after looking at run_constrains.


Fixing some broken dependencies and adding material id support

Fixing some broken dependencies and adding material id support.

Martin looked over my shoulder


relative imports of local modules

in this repository, for the imports of local modules it seems to required to define it like this:
https://github.com/ZBT-Tools/pemfc-tkinter-app/blob/d3d6c77ed4674d159a82b16dcbe0861d33306c7b/pemfc_gui/frame.py#L7
just writing import base produces a ModuleNotFoundError. I believe it has to do with the position of the executed py file (gui_app.py) which is located one hierarchy higher compared to the other repos (pemfc-core and pemfc-dash-app). Can you confirm this?

Not relevant atm


Add tab event to searchable selects

I think a good feature would be that when we write in the search field and press the tab, it selects the best match. It is a little tedious to have to select with cursors or the mouse.

Right now is possible to use the arrows keys.

At least I expect behavior like this: http://demos.telerik.com/kendo-ui/dropdownlist/index
I find it faster and more intuitive, you are with me?

Humm, I see that Buefy is missing the select-on-type functionality on non-searchable selects.
The searchable one looks fine for me.

To be honest I'm thinking about removing this custom select and replace with the default one. The searchable is the only thing different and it's not working very well.


Where are the colours? (C Syntax highlighting deviates from CodeMirror demo)

Why is it that e.g. http://gitlist-khornberg.rhcloud.com/git/blob/master/abspath.c is little colourful while it is very colourful when I copy the code to the CodeMirror demo on https://codemirror.net/mode/clike/index.html?
Is there a way to get more colours? The syntax highlighting is little helpful to me at the moment, I'd like to see more colours.

Probably we're running on an older version of CodeMirror. Marking this as bug!

I cannot reproduce this any more, probably it has been fixed.


Relax opflow CODEOWNERS regex

Summary
This commit updates the CODEOWNERS regex for the opflow module. Previously it was set to opflow/ which will match any path with "opflow" as a directory in it. However, there are several documentation files that have opflow in the name. This commit updates the regex to just be the string "opflow" which will match any file or path that contains "opflow" so that the CODEOWNERS for opflow can merge PRs that touch this documentation too.
Details and comments

I pulled this from the merge queue, because this might not be general enough. It might be better to think about the docs/ ownership as a separate quantity (similar to releasenotes)

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
5 unchanged lines in 3 files lost coverage.
Overall coverage increased (+0.01%) to 87.017%

Files with Coverage Reduction
New Missed Lines
%

crates/qasm2/src/expr.rs
1
93.76%

crates/qasm2/src/lex.rs
1
90.91%

qiskit/pulse/library/waveform.py
3
93.75%

Totals

Change from base Build<PHONE_NUMBER>:
0.01%

Covered Lines:
74161

Relevant Lines:
85226

💛  - Coveralls

I was thinking that the whole opflow/algorithms thing will mostly resolve itself in a few weeks' time via their removal from the repo, so I wasn't particularly worried about those rules.  I agree that we'll probably need to re-examine ownership of docs and releasenotes to support more general cross-team stuff.


no problem playing on iphone,but pc has no audio output

possibly it's something wrong with my pc ,but i can't solve this problem.
this software and bluetooth audio receiver both work well,but neither of them can't have audio output

Something I noticed with iPadOS (which is likely similar to regular iOS) is that there's an option to select what type of Bluetooth device you're using. You may need to go into Bluetooth settings in iOS, tap the blue info icon to the right of your computer and then select "headphones."
https://github.com/ysc3839/AudioPlaybackConnector/assets/10189526/3c65f8bc-3c3b-47f8-ae1e-e9429e3ae95f


[concurrency] Key paths to actor members

We plan to allow key paths to non-actor-isolated members of an actor. This primarily refers to let-bound or actorIndependent(unsafe) properties. This PR implements these key paths.
resolves rdar://75120097

@swift-ci please smoke test and merge


Update docs for AMDC REV F

This pull request addresses #92 .
The PR updates the docs with new pages, references, and images for the latest AMDC hardware revision, REV F.
Since I was updating images, I also took care of addressing #26 . A picture of the REV D bring-up page (built locally) with the re-sized images is shown below:

Let's merge this as a squash and merge, i.e. 1 commit.


Should remove this line

https://github.com/pinterest/it-cpe-cookbooks/blob/4646c6b3117bed6c031b6dd98c13f0a3b0be4e9c/cpe_crypt/resources/cpe_crypt_install.rb#L42

Anything else?

You caught my spelling mistakes :P
On Mon, Oct 16, 2017 at 5:48 PM Erik Gomez<EMAIL_ADDRESS>wrote:

Anything else?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/pinterest/it-cpe-cookbooks/issues/3#issuecomment-337084578,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD2KZYNG-PKkymvmg9OG3Nepwa3_jmQHks5ss_lHgaJpZM4P7YeT
.


Lingering TUI is too noisy

Problem
Now that the TUI "lingers" in the terminal (#7801), it becomes more important to avoid noise. Overall the current situation is good, except for 2 specific lines: ModuleSource.resolveFromCaller and ModuleSource.resolveDirectoryFromCaller.
✔ ModuleSource.resolveFromCaller: ModuleSource! 0.4s
✔ ModuleSource.resolveDirectoryFromCaller(path: "..", viewName: "default"): Directory! 0.0s

These lines shouldn't be visible at the default verbosity, because they are implementation details and the user probably won't know what it means.
Solution
Remove those two lines from the lingering TUI output at default verbosity.

FYI @jedevc @vito as discussed yesterday


No module named 'utils.compute_overlap'

Hello, firstly thank you for the great work.
I am having trouble starting training. I get this error:
-- No module named 'utils.compute_overlap' --
I have the keras-retinanet running perfectly on my machine.
What could be done?
Regards!

Hi @nicholasguimaraes , could you replace the compute_overlap.so of this repo with the one from keras-retinanet?
If it still does not work, try to rebuild it by running the setup.py script from keras-retinanet.
Wait for your feedback.

Thank you for your reply.
In fact I don't have a compute_overlap.so file under keras-retinanet/utils.
I do have a compute_overlap.c ,  .pyd  and   .pyx.
I tried replacing the compute_overlap.pyx  from  EfficientDet/utils with the .pyx from RetinaNet/utils and I was able to start training but, I got an AssertionError after the first step.

How to solve the same problem on Windows  @xuannianz ，

I've fixed it by renaming the "compute_overlap.cython-36m-x86_64-linux-gnu.so" to "compute_overlap.so"

@nikola-j Do it really work on Windows?
@HaiLongEmb How about running python setup.py build_ext --inplace to rebuild the module?

@xuannianz I'm on ubuntu 18.04

Hello @xuannianz , I am also on Windows.
After running "python setup.py build_ext --inplace"  I got the file "compute_overlap.cp36-win_amd64.pyd" which I renamed to "compute_overlap.pyd". I added the .pyd from retinanet/utils to efficientdet/utils and I was able to start training but soon I get another Error.
"""
Traceback (most recent call last):
File "train.py", line 352, in
main()
File "train.py", line 347, in main
validation_data=validation_generator
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1433, in fit_generator
steps_name='steps_per_epoch')
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\engine\training_generator.py", line 293, in model_iteration
callbacks._call_batch_hook(mode, 'end', step, batch_logs)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\callbacks.py", line 248, in _call_batch_hook
batch_hook(batch, logs)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\callbacks.py", line 531, in on_train_batch_end
self.on_batch_end(batch, logs=logs)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\callbacks_v1.py", line 362, in on_batch_end
profiler.save(self.log_dir, profiler.stop())
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\eager\profiler.py", line 144, in save
gfile.MakeDirs(plugin_dir)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 438, in recursive_create_dir
recursive_create_dir_v2(dirname)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 453, in recursive_create_dir_v2
pywrap_tensorflow.RecursivelyCreateDir(compat.as_bytes(path))
tensorflow.python.framework.errors_impl.NotFoundError: Failed to create a directory: logs/2019-12-05\plugins\profile\2019-12-05_14-18-11; No such file or directory
"""
Has anyone gotten the same error?

random_misc_group_entry
assert annotations['bboxes'].shape[0] != 0
AssertionError

Regarding the AssertionError I got a few days ago, it happened because I tried using background images with no x1,y1,x2,y2

@nicholasguimaraes You can pull the latest code in which I removed that assertion.

@nicholasguimaraes You can pull the latest code in which I removed that assertion.

I just pulled the latest code.
It seems that I cannot train without validation data.
"""
Traceback (most recent call last):
File "train.py", line 355, in
main()
File "train.py", line 350, in main
validation_data=validation_generator
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1433, in fit_generator
steps_name='steps_per_epoch')
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\engine\training_generator.py", line 220, in model_iteration
batch_data = _get_next_batch(generator, mode)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\engine\training_generator.py", line 362, in _get_next_batch
generator_output = next(generator)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\utils\data_utils.py", line 785, in get
six.reraise(*sys.exc_info())
File "C:\Users\Windows\AppData\Roaming\Python\Python36\site-packages\six.py", line 693, in reraise
raise value
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\utils\data_utils.py", line 779, in get
inputs = self.queue.get(block=True).get()
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\multiprocessing\pool.py", line 644, in get
raise self._value
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\multiprocessing\pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\utils\data_utils.py", line 571, in get_index
return SHARED_SEQUENCES[uid][i]
File "C:\Users\Windows\Documents\EfficientDet\generators\common.py", line 392, in getitem
inputs, targets = self.compute_inputs_targets(group)
File "C:\Users\Windows\Documents\EfficientDet\generators\common.py", line 348, in compute_inputs_targets
image_group = self.load_image_group(group)
File "C:\Users\Windows\Documents\EfficientDet\generators\common.py", line 219, in load_image_group
return [self.load_image(image_index) for image_index in group]
File "C:\Users\Windows\Documents\EfficientDet\generators\common.py", line 219, in
return [self.load_image(image_index) for image_index in group]
File "C:\Users\Windows\Documents\EfficientDet\generators\csv.py", line 225, in load_image
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
cv2.error: OpenCV(4.1.2) C:\projects\opencv-python\opencv\modules\imgproc\src\color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cv::cvtColor'
"""
I'm not passing compute_val_loss as an argument.
This is my line to start training:
python train.py --phi 0 --freeze-backbone csv bin/csv_label/annotations.csv bin/csv_label/classes.csv

Check the image paths in your csv file.

Check the image paths in your csv file.

The image path is correct. I wonder if using .gif images could be the problem.
I use the same dataset to train retinanet and we works fine.

I fixed it!
On generators/csv_.py   I updated the function load_image to load images with PIL instead of cv2, but that was not the solution.
I had to go to
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\Lib\site-packages\tensorflow\python\lib\io
and update file_io.py , more specifically the function recursive_create_dir_v2.
that's what it looks now:
def recursive_create_dir_v2(path):
#print(path)
dest_log = "C:\Users\Windows\Documents\EfficientDet\logs\"
path_new = path.replace("logs/","")
path_new =  path_new.replace(chr(92), "\")
path = dest_log+path_new
Turns out it was simply a path error. I hope this can help all windows users.

Hello @xuannianz , I am also on Windows.
After running "python setup.py build_ext --inplace" I got the file "compute_overlap.cp36-win_amd64.pyd" which I renamed to "compute_overlap.pyd". I added the .pyd from retinanet/utils to efficientdet/utils and I was able to start training but soon I get another Error.
"""
Traceback (most recent call last):
File "train.py", line 352, in
main()
File "train.py", line 347, in main
validation_data=validation_generator
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1433, in fit_generator
steps_name='steps_per_epoch')
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\engine\training_generator.py", line 293, in model_iteration
callbacks._call_batch_hook(mode, 'end', step, batch_logs)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\callbacks.py", line 248, in _call_batch_hook
batch_hook(batch, logs)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\callbacks.py", line 531, in on_train_batch_end
self.on_batch_end(batch, logs=logs)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\callbacks_v1.py", line 362, in on_batch_end
profiler.save(self.log_dir, profiler.stop())
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\eager\profiler.py", line 144, in save
gfile.MakeDirs(plugin_dir)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 438, in recursive_create_dir
recursive_create_dir_v2(dirname)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 453, in recursive_create_dir_v2
pywrap_tensorflow.RecursivelyCreateDir(compat.as_bytes(path))
tensorflow.python.framework.errors_impl.NotFoundError: Failed to create a directory: logs/2019-12-05\plugins\profile\2019-12-05_14-18-11; No such file or directory
"""
Has anyone gotten the same error?
Sorry, I didn’t see the file set.py in the code. Could you send me this? Thank you!

when i run a inference.py i am getting a error as no module named compute_overlap
what to do? when we encounter this in inference.py

please reply back i need urgent reply


Speed up slow cw721_stake_max_claims_works  integration test

Speeds up cw721_stake_max_claims_works test by batching all of the Exec msgs into a single cosmos tx.

Runtime before batching: 28 minutes
Runtime after batching: 29 seconds

Codecov Report
Base: 94.85% // Head: 94.85% // No change to project coverage :thumbsup:

Coverage data is based on head (ea53001) compared to base (3c36ae6).
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #547   +/-   ##
=======================================
  Coverage   94.85%   94.85%
=======================================
  Files          38       38
  Lines        3539     3539
=======================================
  Hits         3357     3357
  Misses        182      182

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


bugfix: remove the DiskQuota useless field from Resource

Signed-off-by: Wei Fu<EMAIL_ADDRESS>
Ⅰ. Describe what this PR did
Remove the DiskQuota useless field from Resource
Ⅱ. Does this pull request fix one issue?

Ⅲ. Describe how you did it
None
Ⅳ. Describe how to verify it
None
Ⅴ. Special notes for reviews
None

Codecov Report

Merging #1212 into master will increase coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1212      +/-   ##
==========================================
+ Coverage   15.57%   15.59%   +0.01%
==========================================
  Files         178      178
  Lines       10454    10441      -13
==========================================
  Hits         1628     1628
+ Misses       8706     8693      -13
  Partials      120      120

Impacted Files
Coverage Δ

daemon/mgr/container.go
0% <0%> (ø)
:arrow_up:

LGTM


[sumo] update meta-rust revision

Closing this as the 1.1 release is approaching EoL.  We'd encourage anyone who needs the latest to look at the 1.4 recipes in the Dunfell branch.


Compare different effort levels

https://storage.googleapis.com/demos.webmproject.org/webp/cmp/2024_03_25/index.html compares JPEG XL effort 7 with AVIF speed 6. How do I get a comparison of JPEG XL 8 vs AVIF speed 6?

Thank you for your interest in codec-compare.
There is nothing yet in the graphical interface to select that, but it can be done by fiddling with the URL:
https://storage.googleapis.com/demos.webmproject.org/webp/cmp/2024_03_25/index.html?batch=batches/avif_6.json&batch=batches/jpegxl_8.json
There is also a preset for comparing all efforts of both codecs, but that may be too high-level for your needs:
https://storage.googleapis.com/demos.webmproject.org/webp/cmp/2024_03_25/index.html?load=jpegxl_vs_avif.json#ref=AVIF+speed+6


libjpeg: modernize

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Relase notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 131530 run on x86_64-linux 1

2 packages failed to build:

libjpeg_original
lightworks

[nix-shell:~/.cache/nixpkgs-review/pr-131530]$ nix log /nix/store/mm9wy5y292mhcfxa6hi2acyigkv8pj0f-libjpeg-9d.drv
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/r1rxg5k8z0xjqq018b4ch0lcvxf4pbv0-jpegsrc.v9d.tar.gz
source root is jpeg-9d
setting SOURCE_DATE_EPOCH to timestamp<PHONE_NUMBER> of file jpeg-9d/README
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
cp: cannot stat '/share/libtool/config.guess': No such file or directory

Result of nixpkgs-review pr 131530 at cb5d32e0 run on aarch64-linux 1

1 package failed to build:

libjpeg_original

1 suggestion:

warning: maintainers-missing
Package does not have a maintainer. Consider adding yourself?
Near pkgs/development/libraries/libjpeg/default.nix:30:5:
   |
30 |     maintainers = with maintainers; [ ];
   |     ^

Note that build failures may predate this PR, and could be nondeterministic or hardware dependent.
Please exercise your independent judgement. Does something look off? Please file an issue or reach out on IRC.

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
Result of nixpkgs-review pr 131530 run on x86_64-linux 1

2 packages built:

libjpeg_original
lightworks

The following issues got detected with the above build packages.
Please fix at least the ones listed with your changed packages:

libjpeg_original:

warning: unknown setting 'extra-trusted-public-keys'
When evaluating attribute ‘libjpeg_original’:
warning: maintainers-missing
Package does not have a maintainer. Consider adding yourself?
Near pkgs/development/libraries/libjpeg/default.nix:19:5:
   |
19 |     maintainers = with maintainers; [ ];
   |     ^

See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/maintainers-missing.md


Enable the use of custom functions in the the debugger's watch window

When working on c++ applications, it's often the case that you have objects which cannot be fully examined by just looking at the debugger. One example are Eigen::Tensor objects:

Where currently the only option to view the actual values is to add a temporary print statement to your program. One idea could be to write a debug function in a custom header, e.g.:
#include <unsupported/Eigen/CXX11/Tensor>

template <typename T, int num_indices>
std::string eigen_tensor_to_string(const Eigen::Tensor<T, num_indices>& tensor) {
    std::stringstream ss;
    Eigen::array<Eigen::DenseIndex, num_indices> indices;
    indices.fill(0);

    for (int i = 0; i < tensor.size(); ++i) {
        ss << "Index: (";
        for (int j = 0; j < num_indices; ++j) {
            ss << indices[j] << (j == num_indices - 1 ? "" : ", ");
        }
        ss << ")\tValue: " << tensor(indices) << std::endl;

        // Increment indices
        for (int j = num_indices - 1; j >= 0; --j) {
            if (++indices[j] < tensor.dimension(j)) {
                break;
            }
            indices[j] = 0;
        }
    }
    return ss.str();
}

And then entering the expression
eigen_tensor_to_string(u_tilde_p_padded)
in the watch window to view the values. I realized then that this is  not possible because custom functions are not accessible from this window, but I believe that it would be a useful feature in many cases where the datastructure of an object doesn't allow for direct examination of the variables in the debgger.

I think this is a request for the c++ debug extension, please file it on that repo, thanks

@roblourens could you maybe link the repo? I wasn't able to find it


Enhance ReviewUnusedParameter to bail out if $MyInvocation.BoundParmeters encountered

PR Summary

PR Checklist

[ ] PR has a meaningful title

Use the present tense and imperative mood when describing your changes

[ ] Summarized changes
[ ] Change is not breaking
[ ] Make sure all .cs, .ps1 and .psm1 files have the correct copyright header
[ ] Make sure you've added a new test if existing tests do not effectively test the code changed and/or updated documentation
[ ] This PR is ready to merge and is not Work in Progress.

If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: manigandan-jegannathan-developer sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Caffe Plugin

@HrWangChengdu I helped you created this pull request to track progress and make some comments.

@tqchen @antinucleon Could you have a look?

@HrWangChengdu Is this currently functional?
Could you give a list of things needed before we can merge in a functional initial version?

Sorry, I'm in the course due of my graduate school, would be available in 1 or 2 days. Thanks for understanding.
Yes. But only forked dmlc/caffe would work, which add several extra interfaces. In the release version, we could provide a patch for those extra codes.
List of things before we can merge:

replace by caffe registry
refine caffe-plugin docs (also interface discussion)
pull request dmlc/caffe to get patch number
python metric modification

What did you add? Is it possible to do it without adding anything to caffe?

Ok I see. You had to access the private blobs. Looks like simple addition of 3 functions.
In that case let's try PR this back to caffe & generate a patch for users. Should be an easy patch.

Yes! that's what i mean :-)

many thanks for the pr, it's very helpful!
I'm not sure caffe will accept the pr quickly. one way is to provide a patch in the readme to tell users how to do the caffe patch manually.
also is there any test code? such as

end-to-end test to check if we can get a reasonable accuray
unittest to check if same results can be get from mx's op and caffe's op

also i'm curious about the performance. ideally, using caffe's op will have the some performance as mxnet's op. is it true?

Thank you!
I think we could do the pr to caffe meanwhile provides manually patch document.
We do have. There're mlp & lenet training on mnist by caffe op in example/caffe folder. Both networks achieve high accuracy. And I double checked caffe op by gradient_checker from dmlc/minpy package.
Oh, that would be really interesting. Let me make some performance comparison results this week.

@piiswrong  could you please check python side code, i.e. example/caffe/caffe_net.py?
I guess
fc = mx.sym.CaffeOp(data=[data0, data1, data2], prototxt=....)
seems better than
fc = mx.sym.CaffeOp(data_0=data0, data_1=data1, data_2=data2, prototxt=....)

data=[data0, data1, data2] is equivalent to *[data0, data1, data2]

LGTM. Fix lint and we can merge.
This is a great effort. Thanks so much for doing this.

Fixed the lint and complement the patch in caffe docs.
Is there anything else to do?
Thank you so much for helping me doing this merge!

Hopefully the test will pass and I'll merge it

@HrWangChengdu Because I created this pull request for you, the commit history now traces this to me. You can make a new PR with two commits where the first one removes every file you added in this PR and the second one adds them back in, so that the history will be traced to you.

you can revert this pr...

How? I probably don't have the privilige

@mli I can't do forced push. Can you? Discard the last commit?

forced push is disabled, which is too dangers. there is a revert button in
the pr page, it basically files another pr to revert all changes
On Tue, Jul 26, 2016 at 10:17 PM, Eric Junyuan Xie<EMAIL_ADDRESS>
wrote:

@mli https://github.com/mli I can't do forced push. Can you? Discard
the last commit?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/mxnet/pull/2746#issuecomment-235489396, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAZv4XQbHpvsAxZG0vkydV4mKqoXJLSKks5qZun3gaJpZM4JOVRn
.

I have created a revert PR

Should I new a PR after the revert one is done?

Yes, Please PR gain

Just create one https://github.com/dmlc/mxnet/pull/2854

It is great that this is merged in. Let us work on making the blogpost to dmlc.ml and put this on the news


How to find the status of configuration if its runing or stopped.

previous version of extension icon update got removed.

I would love to see the status of the extension whether it's running or retrying etc.
It's not necessary for updating the icon but I liked the icon update still.


[7650] Amended to read csv as hash

Context
Failing specs
Changes proposed in this pull request
Amended to read csv as hash
Guidance to review
bundle exec rspec ./spec/services/

Important business

[ ] Does this PR introduce any PII fields that need to be overwritten or deleted in db/scripts/sanitise.sql?
[ ] Does this PR change the database schema? If so, have you updated the config/analytics.yml file and considered whether you need to send 'import_entity' events?
[ ] Do we need to send any updates to DQT as part of the work in this PR?
[ ] Does this PR need an ADR?

NB: Please notify the #twd_data_insights team and ask for a review if new fields are being added to analytics.yml

 Fix flaky tests on code coverage full test runs