dlrs_TaskTrigger: execution of AfterInsert  caused by: System.DmlException: Upsert failed. First exception on row 0; first error: UNABLE_TO_LOCK_ROW

Hi Team!
I've been trying to figure out the best way to handle this UNABLE_TO_LOCK_ROW error and have looked over your forums, but I think our use case is different than the other threads since we're not using bulk API nor do we want to remove this dlrs.
Our users, when trying to send an email from a chatter publisher action, are intermittently hitting this error:
Apex script unhandled trigger exception by user/organization:<PHONE_NUMBER>7h7fV/00D15000000FTXQ

dlrs_TaskTrigger: execution of AfterInsert

caused by: System.DmlException: Upsert failed. First exception on row 0; first error: UNABLE_TO_LOCK_ROW, unable to obtain exclusive access to this record or 1 records: a0D1500000R03quEAB: []

Class.dlrs.RollupService.updateMasterRollupsTrigger: line 1023, column 1
Class.dlrs.RollupService.handleRollups: line 919, column 1
Class.dlrs.RollupService.triggerHandler: line 311, column 1
Trigger.dlrs_TaskTrigger: line 12, column 1

They can get around it by reloading the page and trying again. However, I know they're hitting it frequently based on the number of errors coming to my email.

We have two triggers on the task object: the dlrs one and our internal one. Our internal trigger only has one singular apex class that runs before delete to prevent a user from removing a task record. We do have a few things going on at the case level once the case is updated, but nothing that should immediately fire when this field in the dlrs is updated.

We only have one dlrs config on the task. It's scheduled for real-time (and must remain real-time because it drives our service rep queue priority). We have a filter so that it is only supposed to run when tasks are of a TaskSubtype of Call. (Which is another reason why I'm confused emails being examined in this dlrs and attempting to update the case).

Any advice on what to look for or what to tweak? Again, we need to keep this at real-time. If I need to create some internal custom code, so be it. I just thought it wouldn't hurt to ask.
Here's the metadata record for this:

And a screenshot of the record that's being referred to in the error:

Perhaps what I need to do is switch this to developer mode and merge it with my current trigger framework? I looked around for some documentation on a few examples of Developer mode and the syntax needed to call it from our own custom triggers. Can I rip the current stuff right off the example from the manage child trigger page?

We're getting the same issue.  @kkorynta Did you end up finding a fix for this?

Looking at the error detail from OP, the id of the record concerned is: a0D1500000R03quEAB
Which is not a Task, but a custom object record?
For OP, my question would be which object represents a0D?
For DM28th, what kind of id is mentioned in your scenario?

@horzel The prefix a0D in our org is pointed to the dlrs__LookupRollupSummary__c object, which is what makes this error tricky to manage.

@horzel Thanks for following up!
the id mentioned in my scenario is an account id but it's rolling up information from a Task.  Here is my error
`Apex script unhandled trigger exception by user/organization: 0050y00000D7WoJ/00D80000000KtFf
dlrs_TaskTrigger: execution of AfterInsert
caused by: System.DmlException: Update failed. First exception on row 0 with id 0010y00001ZAYxuAAH; first error: FIELD_CUSTOM_VALIDATION_EXCEPTION, Please fill in the Acct Demo Booked and Acct Demo Type fields: []
Class.dlrs.RollupService.Updater.updateRecords: line 1315, column 1
Class.dlrs.RollupService.UpdateWithoutSharing.updateRecords: line 1358, column 1
Class.dlrs.RollupService.updateRecords: line 1286, column 1
Class.dlrs.RollupService.handleRollups: line 918, column 1
Class.dlrs.RollupService.triggerHandler: line 311, column 1
Trigger.dlrs_TaskTrigger: line 7, column 1`

caused by: System.DmlException: Update failed. First exception on row 0 with id 0010y00001ZAYxuAAH; first error: FIELD_CUSTOM_VALIDATION_EXCEPTION, Please fill in the Acct Demo Booked and Acct Demo Type fields: []

Not only is it mentioning an account, it is also mentioning a different error, it does not concern a row locking error.
For your scenario, I take it you are rolling up information from the Tasks to the Account?
That means the Account is being updated, but a validation rule prevents this, as some fields are not populated.
You would need to either make sure all Accounts have these fields populated, before you start using the roll ups, or you would need to adjust the validation to not fire if only the fields with the roll up results are updated on the account.

@kkorynta Regarding your issue, I am not that deep into this package yet, but looking at your screenshot for the referenced record 'Tasks This Month', is that roll up defined correct? I do not see much info in there?
You might need to check on that roll up instead of the Total Case Calls roll up.

@horzel This record was auto created by the system after I converted the settings to the new metadata format. When I reached out to the SFDC community group (Declarative Lookup Rollup Summaries Tool), I was told this was expected behavior when using the metadata setup as opposed to the custom object setup because the scheduled items needed a place to be housed.

Okay, I would not know how to help further, sorry. The UNABLE_TO_LOCK_ROW, is caused by it being accessed at the same time by multiple actions.

Right, that's why I'm here. The only processes that would access that dlrs__LookupRollupSummary__c  record in our org are those in the dlrs package.

Will this issue be prioritized? is there any additional information I can provide?

Hey There, I help maintain DLRS. I do not think this is necessarily a DLRS specific bug/issue to be fixed. It is as people have said likely something with your configuration and multiple actions blocking each other. DLRS won't trip over itself blocking itself from editing a record.  UNABLE TO LOCK ROW, is always something else is editing the same record at the same time.
You could quickly "bandaid" this by changing from realtime to process builder, and using a scheduled action that has a 0 hours delay, or using scheduled mode and running jobs to update it.
Other than that, you can certainly incorporate the DLRS trigger code into your own trigger, and disable the DLRS trigger, to help control your order of operations.
Trying to roll up Tasks onto Cases is certainly a tricky scenario since the volume of tasks is so large and the relationship WhatId is polymorphic.
You should turn on debugging and run this again, it will tell you want all is running at the same time that is trying to edit the same record.

@kkorynta why don't you delete that DLRS record that the row lock is happening on? it's odd to have a DLRS with a bunch of N/A typed into it like that.

I’ve tried to delete it before, the package just ends up recreating another
record like it. No other process is accessing dlrs records except for this
package.
The dlrs triggers were already merged into our existing framework for
several objects and we’re still having the issue, albeit less frequently.
We cannot switch the calculation to a process builder or to scheduled since
our support processes rely on this calculation being real-time currently.
If anything, i’d Have to convert the real-time ones to our own code and rip
this package out. That’s a hefty project to convert everything and i’d like
to avoid it.
Also, I’m reaching out again because it’s a similar issue on our email
message object, which removes the polymorphic id issue.
I can pull some logs. It might take me a day or two because this is an
intermittent problem and there’s a lot on my plate at work. If I can show
that it’s the dlrs roll up service causing the issue, who should I reach
out to?
Thanks,
Kelly
On Mon, Jul 15, 2019 at 2:26 PM Jim Bartek<EMAIL_ADDRESS>wrote:

@kkorynta https://github.com/kkorynta why don't you delete that DLRS
record that the row lock is happening on? it's odd to have a DLRS with a
bunch of N/A typed into it like that.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/afawcett/declarative-lookup-rollup-summaries/issues/788?email_source=notifications&email_token=AE57YJFO6FUKYFNVQAV5VR3P7S6NPA5CNFSM4GUP5KJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZ6R4LQ#issuecomment-511516206,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE57YJHDR5PNKCT4KM2SPPDP7S6NPANCNFSM4GUP5KJQ
.

Kelly,  OK thanks for the details.  Let me know if you can produce some logs showing what is going on. I can discuss with Andy and see if we can pinpoint a bug in the product, however there are thousands of orgs using DLRS.
Along with the debug logs:

What version is installed?
Lame request, but have you tried deactivating/removing the DLRS task trigger, and recreating it?

Also, you said "when trying to send an email from a chatter publisher action":

what happens if you just manually create and edit tasks on cases that match the DLRS criteria?
I am wondering if somehow DLRS gets called twice in the same transaction due to what the Action is doing to the Task.

Again even with a Process Builder you can get it to run near instantaneous, like a Future method.  Or you might be able to move the DLRS trigger call into your trigger wrapped in a future.

Hey Jim,
We are currently using version 2.11.1.
In regards to the task trigger, yes, sort of. Originally when I started at this company, they had two triggers on the task object - the dlrs one and an internal one. I've removed the dlrs one and merged it into our trigger framework. If it's ok, I'd like to focus on the EmailMessage dlrs trigger. I'm getting the same error message, but this has a much bigger impact on our org as it affects inbound emails. I've provided the trigger code at the bottom and its dispatcher.
Here is the error message:
CANNOT_INSERT_UPDATE_ACTIVATE_ENTITY : EmailMessageTrigger: execution of AfterInsert

caused by: System.DmlException: Upsert failed. First exception on row 0; first error: UNABLE_TO_LOCK_ROW, unable to obtain exclusive access to this record or 1 records: a0D1C00000TYVEeUAP: []

Class.dlrs.RollupService.updateMasterRollupsTrigger: line 1026, column 1
Class.dlrs.RollupService.handleRollups: line 922, column 1
Class.dlrs.RollupService.triggerHandler: line 311, column 1
Class.EmailMessageTriggerDispatcher.afterInsert: line 55, column 1
Class.TriggerHandler.run: line 44, column 1
Trigger.EmailMessageTrigger: line 5, column 1

Here's what that record Id looks like when I pull it up - note it's not active and we don't have a matching metadata config that matches either. Mr. Rodriguez is also not a system administrator and does not have access to the dlrs app or its objects.

As far as what roll ups we have for the email message object as the child, we currently have three. They are all configured under the metadata side of things and are all System Calculation + realtime. One counts the total number of emails, one for inbound, and the other for outbound. They all aggregate up to the case object.
Metadata configs

dlrs configs

As mentioned, I'll work on getting those logs. It's going to take a little bit of coordination as I'm not the support process owner, so I don't receive the errors directly. I do have the logs set up and will get a sample for you within 2 days.
Thanks,
Kelly
trigger EmailMessageTrigger on EmailMessage (before insert, before update, before delete, after insert, after update, after delete, after undelete) {
    Map<String, Trigger_Settings__c> triggerSettings = Trigger_Settings__c.getAll();

    if(triggerSettings.get('EmailMessageTrigger') <> null && triggerSettings.get('EmailMessageTrigger').Execute_Trigger__c)
        new EmailMessageTriggerDispatcher().run();
}

public without sharing class EmailMessageTriggerDispatcher extends TriggerHandler {
    //Trigger context variables are still available but need to be cast: trigger.new, trigger.newMap, trigger.oldMap
    List<EmailMessage> emailMessages = (List<EmailMessage>) trigger.new;
    Map<Id, EmailMessage> emailMessageOldMap = (Map<Id, EmailMessage>) trigger.oldMap;
    Map<Id, EmailMessage> emailMessageNewMap = (Map<Id, EmailMessage>) trigger.newMap;
    Map<String, Trigger_Settings__c> triggerSettings = Trigger_Settings__c.getAll();

    protected override void beforeInsert() {
        //declare variables
        //loop through to find qualifying records
        //execute methods
        if(!TriggerHandler.isBypassed('BI-dlrs_EmailMessageTrigger') && triggerSettings.get('dlrs_EmailMessageTrigger') <> null && triggerSettings.get('dlrs_EmailMessageTrigger').Execute_Trigger__c) {
            TriggerHandler.bypass('BI-dlrs_EmailMessageTrigger');
            dlrs.RollupService.triggerHandler(EmailMessage.SObjectType);
        }
    }

    protected override void beforeUpdate() {
        //declare variables
        //loop through to find qualifying records
        //execute methods
        if(!TriggerHandler.isBypassed('BU-dlrs_EmailMessageTrigger') && triggerSettings.get('dlrs_EmailMessageTrigger') <> null && triggerSettings.get('dlrs_EmailMessageTrigger').Execute_Trigger__c) {
            TriggerHandler.bypass('BU-dlrs_EmailMessageTrigger');
            dlrs.RollupService.triggerHandler(EmailMessage.SObjectType);
        }
    }

    protected override void beforeDelete() {
        //declare variables
        //loop through to find qualifying records
        //execute methods
        //if(!emailMessageOldMap.isEmpty() && !TriggerHandler.isBypassed('PreventDeleteHandler') && triggerSettings.get('PreventDeleteHandler') <> null && triggerSettings.get('PreventDeleteHandler').Execute_Trigger__c) {
        //    TriggerHandler.bypass('PreventDeleteHandler');
        if(!emailMessageOldMap.isEmpty() && triggerSettings.get('PreventDeleteHandler') <> null && triggerSettings.get('PreventDeleteHandler').Execute_Trigger__c) {
            PreventDeleteHandler.preventDelete(emailMessageOldMap.values());
        }

        if(!TriggerHandler.isBypassed('BD-dlrs_EmailMessageTrigger') && triggerSettings.get('dlrs_EmailMessageTrigger') <> null && triggerSettings.get('dlrs_EmailMessageTrigger').Execute_Trigger__c) {
            TriggerHandler.bypass('BD-dlrs_EmailMessageTrigger');
            dlrs.RollupService.triggerHandler(EmailMessage.SObjectType);
        }
    }

    protected override void afterInsert() {
        //declare variables
        //loop through to find qualifying records
        //execute methods
        if(!emailMessages.isEmpty() && !TriggerHandler.isBypassed('CaseCloneHandler') && triggerSettings.get('CaseCloneHandler') <> null && triggerSettings.get('CaseCloneHandler').Execute_Trigger__c) {
                TriggerHandler.bypass('CaseCloneHandler');
                CaseCloneHandler.processEmailMessages(emailMessages);
        }

        if(!TriggerHandler.isBypassed('AI-dlrs_EmailMessageTrigger') && triggerSettings.get('dlrs_EmailMessageTrigger') <> null && triggerSettings.get('dlrs_EmailMessageTrigger').Execute_Trigger__c) {
            TriggerHandler.bypass('AI-dlrs_EmailMessageTrigger');
            dlrs.RollupService.triggerHandler(EmailMessage.SObjectType);
        }
    }

    protected override void afterUpdate() {
        //declare variables
        //loop through to find qualifying records
        //execute methods
        if(!TriggerHandler.isBypassed('AU-dlrs_EmailMessageTrigger') && triggerSettings.get('dlrs_EmailMessageTrigger') <> null && triggerSettings.get('dlrs_EmailMessageTrigger').Execute_Trigger__c) {
            TriggerHandler.bypass('AU-dlrs_EmailMessageTrigger');
            dlrs.RollupService.triggerHandler(EmailMessage.SObjectType);
        }
    }

    protected override void afterDelete() {
        //declare variables
        //loop through to find qualifying records
        //execute methods
        if(!TriggerHandler.isBypassed('AD-dlrs_EmailMessageTrigger') && triggerSettings.get('dlrs_EmailMessageTrigger') <> null && triggerSettings.get('dlrs_EmailMessageTrigger').Execute_Trigger__c) {
            TriggerHandler.bypass('AD-dlrs_EmailMessageTrigger');
            dlrs.RollupService.triggerHandler(EmailMessage.SObjectType);
        }
    }

    protected override void afterUndelete() {
        //declare variables
        //loop through to find qualifying records
        //execute methods
        if(!TriggerHandler.isBypassed('AUD-dlrs_EmailMessageTrigger') && triggerSettings.get('dlrs_EmailMessageTrigger') <> null && triggerSettings.get('dlrs_EmailMessageTrigger').Execute_Trigger__c) {
            TriggerHandler.bypass('AUD-dlrs_EmailMessageTrigger');
            dlrs.RollupService.triggerHandler(EmailMessage.SObjectType);
        }
    }
}

I am not sure why you have Scheduled Items when it is realtime. Are those old records?
But i think the main culprit is you call DLRS in both BEFORE INSERT and AFTER INSERT, so you are calling it twice in the same transaction.  You should put comments in there so you can see in the logs if it succeeded the first time, and then failed in the after insert.

It's likely an old record, but it doesn't make sense why current transactions are referencing it.
I'm calling dlrs in both the before insert and after insert as that's what the default apex class references. Happy to adjust it if we're certain that's the culprit.
/**
 * Auto Generated and Deployed by the Declarative Lookup Rollup Summaries Tool package (dlrs)
 * Deactivated as it's been merged into the core trigger framework
 **/
trigger dlrs_EmailMessageTrigger on EmailMessage (before delete, before insert, before update, after delete, after insert, after undelete, after update) {
    dlrs.RollupService.triggerHandler(EmailMessage.SObjectType);
}

Yeah you are right, no processing is done in the before phase anyways:
https://github.com/afawcett/declarative-lookup-rollup-summaries/blob/master/force-app/main/classes/RollupService.cls
I think debug logs are the next step, specifically seeing what is trying to
touch that record.
I would also try clearing out all that old scheduled items, as when it runs
it may being trying to process those as well.
BTekVentures.com
On Tue, Jul 16, 2019 at 8:02 AM kkorynta<EMAIL_ADDRESS>wrote:

It's likely an old record, but it doesn't make sense why current
transactions are referencing it.
I'm calling dlrs in both the before insert and after insert as that's what
the default apex class references. Happy to adjust it if we're certain
that's the culprit.
/**

Auto Generated and Deployed by the Declarative Lookup Rollup Summaries Tool package (dlrs)
Deactivated as it's been merged into the core trigger framework
**/
trigger dlrs_EmailMessageTrigger on EmailMessage (before delete, before insert, before update, after delete, after insert, after undelete, after update) {
dlrs.RollupService.triggerHandler(EmailMessage.SObjectType);
}

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/afawcett/declarative-lookup-rollup-summaries/issues/788?email_source=notifications&email_token=AAI7GG2MPFFPUQOSVW3GDATP7W2DRA5CNFSM4GUP5KJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2AT4XI#issuecomment-511786589,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAI7GG77GK6FVQATUOAOFQTP7W2DRANCNFSM4GUP5KJQ
.

Hey Jim - still working with Salesforce support on getting the logs and vetting the information that they're telling us. I'll be sure to update you as soon as I know more.

@kkorynta you can simply turn on Debug Logs in Salesforce, and then go cause the error to happen to capture a log

Yep, I know how to run a debug log. The problem is, I'm getting logs like
this on full detail, which is why I've involved salesforce support to take
a look at server side logs and see if they can make heads or tails of
what's going on.
On Mon, Jul 22, 2019 at 5:20 PM Jim Bartek<EMAIL_ADDRESS>wrote:

@kkorynta https://github.com/kkorynta you can simply turn on Debug Logs
in Salesforce, and then go cause the error to happen to capture a log
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/afawcett/declarative-lookup-rollup-summaries/issues/788?email_source=notifications&email_token=AE57YJCTMNU3PSM44KLDKULQAYQATA5CNFSM4GUP5KJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2RGK4Y#issuecomment-513959283,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE57YJAVI5MQHHN7IJOVDBLQAYQATANCNFSM4GUP5KJQ
.

Cool, FYI that log has some Identifiable info in the email.
I notice the entire log is only running BeforeInsert.  While the trigger passes Before context to the TriggerHandler for DLRS, it does nothing in the Before context:
https://github.com/afawcett/declarative-lookup-rollup-summaries/blob/master/force-app/main/classes/RollupService.cls
Your original log shows an AfterInsert error.
The parent Case is already getting updated by the fact of the Email getting Attached to it, and you are trying to also have DLRS update the same case at the same time. Also you are running CaseCloneHandler.processEmailMessages(emailMessages); at the same time.
Are you able to reproduce it when bypassing CaseCloneHandler?

So .... for Schedule Mode triggers there is (in the current release) a master-detail relationship between LookupRollupSummaryScheduleItems__c and LookupRollupSummary__c. This means a lock scenario can occur under high concurrent load. If locking is due to this it can be confirmed by checking the ID in the error message and relating it back to the source object. I noticed at least one reference to this being the case in the thread above. Good news.... the code in master (yet to be release in package form) removes this master detail relationship..... and also those pesky shadow rollup records in the custom object for those using custom metadata objects only. There is a packaging complication getting this change I need to work through to get a feature enabled with Salesforce. I'll work to get these changes and others out in a new release asap!

Thanks Andrew - hoping to see that the fix will improve our issues.
@JimBTek - As far as your question, we're not able to bypass the CaseCloneHandler as that's a critical piece of code we use to support our clients. Basically, if an inbound email comes in and attaches to a closed case, we clone that email into a new case so our support team can properly handle it.
In general, with the dispatchers, I have set dlrs to run at the end of all of the executing methods. Hopefully, it shouldn't be running at the same time, but there could be something asynchronous occuring in the transaction because dlrs is being called for all DML events.
As far as the investiagation with salesforce has gone, we've engaged their CATT team due to numerous issues with perforamnce in the lightning platform. Now that we're past most of the javascript based issues with lightning, they're poking into server slowness. They identified that a lot of the slowness we're receiving server side is coming from how heavy we use dlrs.  Recently, we did start running into governor limits with concurrent processing as we were attempting to execute an archive process (deleting email messages). The logs from salesforce point back to high trigger CPU run-time, which lines up with the information the CATT team has provided us.
Internally, we're going to see if some of the jobs can be removed/deactivated so there's not as much running. However, salesforce has said that what ever the fix is that Andrew is planning, it should greatly improve our server side processing. Fingers crossed it resolves a lot of the row locking issues as we're still having issues with this in various parts of our application.

@JimBTek @afawcett - Thanks for the info here. We too are running into these errors at bulk loading. We have a lot hitting Campaign Member records and users are importing lists via Apsona and getting these errors often with bulk loads. I just got 1,600 errors from 31k records loaded.
"Rejected: Salesforce said: dlrs_CampaignMemberTrigger: execution of AfterInsert
caused by: System.DmlException: Upsert failed. First exception on row 200; first error: UNABLE_TO_LOCK_ROW, unable to obtain exclusive access to this record or 1 records: a1Gf4000006vhpZEAQ: []
Class.dlrs.RollupService.updateMasterRollupsTrigger: line 1023, column 1
Class.dlrs.RollupService.handleRollups: line 919, column 1
Class.dlrs.RollupService.triggerHandler: line 311, column 1
Trigger.dlrs_CampaignMemberTrigger: line 7, column 1"

I have all (5) jobs scheduled at 2am,230am,3am,330am, and 4am. They are also creating new Lookup Rollup Summary records with thousands of child records. I mass deleted all the child records but the app is creating them over and over again. The creator (of who created those add'l records) are not me and the users (of course) are not aware of triggering it.
In terms of operation - If my schedule is not currently happening - are the records getting queued at create? Trying to unpack this. Thanks!

So .... for Schedule Mode triggers there is (in the current release) a master-detail relationship between LookupRollupSummaryScheduleItems__c and LookupRollupSummary__c. This means a lock scenario can occur under high concurrent load. If locking is due to this it can be confirmed by checking the ID in the error message and relating it back to the source object. I noticed at least one reference to this being the case in the thread above. Good news.... the code in master (yet to be release in package form) removes this master detail relationship..... and also those pesky shadow rollup records in the custom object for those using custom metadata objects only. There is a packaging complication getting this change I need to work through to get a feature enabled with Salesforce. I'll work to get these changes and others out in a new release asap!

Hello @afawcett ,
Hope you are doing good.
I recently started using DLRS package and faced Lock Issue on DLRS Objects for scheduled mode. Is there any estimated timeline for release of this fix in the package?

All RUS fields are defined in Scheduled Mode.
These custom objects gets synchronized by offline Mobile app and hence there can be scenarios for concurrent update. Since CPU timeout limit was the concern , I opted the approach for Schedule (batch calculation only), but this gives Lock issues intermittently.

Do you suggest any alternative if fix is not coming in package version soon?
Thanks in advance for your reply.
Navita

@afawcett Hey Andrew, I work with the OP. Is this resolved in the most recent update? I'm not seeing this in the release notes, though I see you were hoping to have this released per your comment last year.

@Martinch713  yeah if you are talking about my comment here https://github.com/afawcett/declarative-lookup-rollup-summaries/issues/788#issuecomment-570104597. Then yes the latest release includes those changes. Can you retest?

@Martinch713 yeah if you are talking about my comment here #788 (comment). Then yes the latest release includes those changes. Can you retest?

I've finally been able to update and test and we're still getting the Unable to Lock Row error.

@Martinch713 can you confirm that you have completed all the post install steps for the latest release?

Hi @afawcett - We did recently complete the post install steps for the latest release.  We are still getting some lock row issues, but not nearly as frequently. They do seem to 'retry' the record and clear out the next time the RollupJob runs.
Error: Delete failed. First exception on row 82 with id a0C1C00001gtAsrUAE; first error: UNABLE_TO_LOCK_ROW, unable to obtain exclusive access to this record or 1 records: a0C1C00001gtAsrUAE: []. Review Rollup Summary Schedule Items a0C1C00001gtAtZUAU,a0C1C00001gtAtYUAU,a0C1C00001gtAtXUAU,a0C1C00001gtAtWUAU,a0C1C00001gtAtVUAU,a0C1C00001gtAtUUAU,a0C1C00001gtAtTUAU,..... [omitted for brevity]
The only other odd issues that come up are:
Upsert failed. First exception on row 0; first error: DUPLICATE_VALUE, duplicate value found: dlrs__QualifiedParentID__c duplicates value on record with id: a0C1C00001hXzRt: []
and
Failed to process batch for class 'dlrs.RollupJob' for job id '7071C0000B6AuvQ' caused by: System.LimitException: dlrs:Too many query rows: 50001
The query rows I have an idea for a record blacklist that might fix the issue of the scheduled item from being created in the first place. The duplicate value, I'm not sure what's going on with this since the a0C record is the Scheduled Item object and would be managed by the code.

Hi @afawcett and @JimBTek,
I have just updated my org's dlrs package to the newest release. In doing so, we deployed the case child trigger because we are transitioning to a scheduled system instead of a scheduled full calculate. Currently, I have the RollUpJob scheduled to run at 9AM, 12 PM, 3 PM, and 6PM. There were no issues with this until a scheduled triggered flow attempted to run and failed, producing the following errors:
"The flow tried to update these records: 5004x00000Dw0A5AAJ. This error occurred: CANNOT_INSERT_UPDATE_ACTIVATE_ENTITY: dlrs_CaseTrigger: execution of AfterUpdate caused by: System.DmlException: Upsert failed. First exception on row 0; first error: UNABLE_TO_LOCK_ROW, unable to obtain exclusive access to this record: [] (dlrs) Trigger.dlrs_CaseTrigger: line 7, column"
and
"Error element Update_Status_Code (FlowRecordUpdate).
The flow tried to update these records: 5004x00000HeQGsAAN. This error occurred: CANNOT_INSERT_UPDATE_ACTIVATE_ENTITY: dlrs_CaseTrigger: execution of AfterUpdate caused by: System.DmlException: Upsert failed. First exception on row 1; first error: DUPLICATE_VALUE, duplicate value found: dlrs__QualifiedParentID__c duplicates value on record with id: a1bQP0000000MW1YAM: [] (dlrs) Trigger.dlrs_CaseTrigger: line 7, column 1. "
The scheduled triggered flow runs daily at 1:15 AM to check if a formula field is evaluated to true. The formula field is set to only evaluate to true on the 15th of every month. When it does, it updates a Case record's status and associated status code to "Successful".
Our dlrs field is on the Case level rolling up to the Account. It evaluates the MIN of a formula field that ranks a Case based on its status. When its ranking moves up, this Case is populated in the dlrs field at the Account level. The status "Successful" is the highest ranking case at 1. So each of these case updates impact what the dlrs field will change to on the Account level.
I'm posting here because these errors are very similar to the ones listed above. What would be next steps in trying to resolve this? Thanks,

@kemacdougall96 I think it would be best to treat this as an independent issue. I'll provide some level of guidance but I think you'd be best to start in the Trailblazer Community Group for DLRS looking for help resolving these errors.
These errors don't necessarily indicate a problem with DLRS but more a conflict between running processing. Figuring out where and how that conflict is occurring can be difficult, deciding what to do about it is another layer of difficulty.
"UNABLE_TO_LOCK_ROW" is something using SOQL to request a record FOR UPDATE, or essentially calling "dibs" on that record until it is done. There are a few other ways "locks" in the database can be requested but it is a mechanism to say "I'm going to change the record, nobody else change the record until I'm done"
In your first example I would have expected a set of record IDs. "While updating the User with ID 5004x00000Dw0A5AAJ we were unable to lock record..." but the values are blank. So something on the Case trigger was seemingly trying to rollup a value to the User record and that process failed.
I can tell you that DLRS attempts to lock parent User records when it is updating values. (https://github.com/SFDO-Community/declarative-lookup-rollup-summaries/blob/02752a724ac2063fab16ef399270e2c241bb6e44/dlrs/main/classes/RollupService.cls#L686) but I don't know why it is limited to User records. I'd have to dig through more history to figure that out. Is it possible that multiple concurrent case updates are trying to write to the same User record at the same time and this is causing the record contention?
As for your second error, this is similar and we actually have another open issue more closely related to this. You have a rollup configured in Scheduled mode so when DLRS notice a change it tries to write Scheduled Item records to the database using Upsert. Something about that process seemed to having a timing or visibility issue and the Upsert command tried to create a new record but failed because there is already an existing record holding the same dlrs__QualifiedParentID__c value. I haven't been able to replicate this issue on demand for testing but I have some guesses as to how we can handle it better. I believe it is caused by concurrent edits, very similar to the above just a different action in the code.
It seems both of your problems are likely concurrent edits, likely caused by batch updates in your Flow. You will need to try and figure out how to order records better or how to handle the batch updates so they can manage this situation better.
I'm hoping to be able to replicate and work on a fix eventually for your second issue but the first issue is likely not something DLRS can eventually be smart enough to make go away. It will be more of an issue in your environment and your concurrent edits conflicting because they want to update the same record at the same time.
A lot of this is conjecture on my part and I'm sorry if I'm missing the point or overlooked some aspect of your specific issues. Right now the best thing to do is see that the root cause is concurrency and try and figure out how to manage that aspect of it better.

Hi ,
I'm getting the same error. First I received the below error when I bulk-inserted the users as contacts in salesforce. After re-try inserting the contacts, the issue was gone.
Now we have Okta salesforce integration, Okta integration creates contacts in salesforce. Last week okta integration tried to create 40 contacts but we received below error. I've researched and this error is related to declarative lookup rollup summary manage tool manage packages, our org using this package since 2019. We can't uninstall this package because we are using it. How we can resolve this error issue? Do you have any idea? Thank you
'' dlrs_ContactTrigger: execution of AfterInsert\n\ncaused by: System.DmlException: Update failed. First exception on row 0 with id 001Qo000009kMurIAE; first error: UNABLE_TO_LOCK_ROW, unable to obtain exclusive access to this record or 1 records: 001Qo000009kMurIAE: []\n\n(dlrs)\nTrigger.dlrs_ContactTrigger: line 7, column 1",
"errorCode": "CANNOT_INSERT_UPDATE_ACTIVATE_ENTITY",


TODO: Make exception handling optional

To support embedded devices or environments where RTTI and exception are not available.

tinygltf itself does not use exception.
Replacing picojson with this PR https://github.com/kazuho/picojson/pull/99 will make tinygtlf exception free.


Allow taking multiple photos

I just tested this, by commenting out the line below in takePicture():
self.stillImageOutput = nil // Just taking only one image.
and do not dismiss the view controller in didFinishCapturingImage(), it actually supports taking multiple photos!
It is a nice feature. Can you add this as an official feature, so others do not need to change the code directly?
I cannot override the takePicture() or make a new function for this, because stillImageOutput is fileprivate. Making it non-private will work as well.

Updated.


Make failing

Hi,
I just tried do build the latest commit
$ git rev-parse HEAD
42fb5b4f2ff825b2339a8e7b254ec400e822130c

but received the following error:
$ make
  AR       libdeflate.a
  CXX      programs/gunzip.o
In file included from programs/../lib/deflate_decompress.hpp:61:0,
                 from programs/../lib/gzip_decompress.hpp:40,
                 from programs/gunzip.cpp:28:
programs/../lib/input_stream.hpp: In member function ‘bool InputStream::set_position_bits(size_t)’:
programs/../lib/input_stream.hpp:233:16: warning: declaration of ‘bits’ shadows a member of ‘InputStream’ [-Wshadow]
         size_t bits  = bit_pos & 7;
                ^~~~
programs/../lib/input_stream.hpp:282:39: note: shadowed declaration is here
     template<typename T = uint32_t> T bits(bitbuf_size_t n = 8 * sizeof(T)) const
                                       ^~~~
In file included from programs/../lib/deflate_decompress.hpp:58:0,
                 from programs/../lib/gzip_decompress.hpp:40,
                 from programs/gunzip.cpp:28:
programs/../lib/memory.hpp: In instantiation of ‘unique_span<T, D>::unique_span() [with T = unsigned char; D = lock_releaser<std::mutex>]’:
programs/../lib/deflate_decompress.hpp:946:42:   required from here
programs/../lib/memory.hpp:227:21: error: no matching function for call to ‘lock_releaser<std::mutex>::lock_releaser()’
       , _end(nullptr)
                     ^
programs/../lib/memory.hpp:436:39: note: candidate: lock_releaser<std::mutex>::lock_releaser(std::unique_lock<std::mutex>::mutex_type&)
     using std::unique_lock<Lockable>::unique_lock;
                                       ^~~~~~~~~~~
programs/../lib/memory.hpp:436:39: note:   candidate expects 1 argument, 0 provided
programs/../lib/memory.hpp:436:39: note: candidate: lock_releaser<std::mutex>::lock_releaser(std::unique_lock<std::mutex>::mutex_type&, std::defer_lock_t)
programs/../lib/memory.hpp:436:39: note:   candidate expects 2 arguments, 0 provided
programs/../lib/memory.hpp:436:39: note: candidate: lock_releaser<std::mutex>::lock_releaser(std::unique_lock<std::mutex>::mutex_type&, std::try_to_lock_t)
programs/../lib/memory.hpp:436:39: note:   candidate expects 2 arguments, 0 provided
programs/../lib/memory.hpp:436:39: note: candidate: lock_releaser<std::mutex>::lock_releaser(std::unique_lock<std::mutex>::mutex_type&, std::adopt_lock_t)
programs/../lib/memory.hpp:436:39: note:   candidate expects 2 arguments, 0 provided
programs/../lib/memory.hpp:436:39: note: candidate: template<class _Clock, class _Duration> lock_releaser<std::mutex>::lock_releaser(std::unique_lock<std::mutex>::mutex_type&, const std::chrono::time_point<_Clock, _Duration1>&)
programs/../lib/memory.hpp:436:39: note:   template argument deduction/substitution failed:
programs/../lib/memory.hpp:227:21: note:   candidate expects 2 arguments, 0 provided
       , _end(nullptr)
                     ^
programs/../lib/memory.hpp:436:39: note: candidate: template<class _Rep, class _Period> lock_releaser<std::mutex>::lock_releaser(std::unique_lock<std::mutex>::mutex_type&, const std::chrono::duration<_Rep1, _Period1>&)
     using std::unique_lock<Lockable>::unique_lock;
                                       ^~~~~~~~~~~
programs/../lib/memory.hpp:436:39: note:   template argument deduction/substitution failed:
programs/../lib/memory.hpp:227:21: note:   candidate expects 2 arguments, 0 provided
       , _end(nullptr)
                     ^
programs/../lib/memory.hpp:438:5: note: candidate: lock_releaser<Lockable>::lock_releaser(std::unique_lock<_Mutex>&&) [with Lockable = std::mutex]
     lock_releaser(std::unique_lock<Lockable>&& lock) noexcept
     ^~~~~~~~~~~~~
programs/../lib/memory.hpp:438:5: note:   candidate expects 1 argument, 0 provided
programs/../lib/memory.hpp:434:49: note: candidate: lock_releaser<std::mutex>::lock_releaser(lock_releaser<std::mutex>&&)
 template<typename Lockable = std::mutex> struct lock_releaser : private std::unique_lock<Lockable>
                                                 ^~~~~~~~~~~~~
programs/../lib/memory.hpp:434:49: note:   candidate expects 1 argument, 0 provided
make: *** [programs/gunzip.o] Error 1

My compiler:
$ g++ --version
g++ (GCC) 6.4.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Not sure what I missed here.
Thank you for your input on this issue!
Best,
Cedric

Thanks for the report.
It should be fixed in master.
It's a suitability in constructor inheritance with older version of GCC. Not sure what happened exactly...

Is the problem solved on your side ?
(closing for now, assuming it's resolved)

Yes, I could compile and run the bigger_benchmarck.sh.
On our uni.lu HPC on a node with 28 cores, I got 255 MB/s (decompressing with pugz) and 2154 MB/s (counting lines with pugz).
Nice tool :)

Thanks !
Yes it shows how costly in wall time it is to synchronize threads for output. I think this could be improved.


Mangago download error

Source information
Mangago 1.4.15
Source language
English
Steps to reproduce

Open mangago manga
Click download

Expected behavior
Chapter should be downloaded
Actual behavior
The error always happen after 5th page and download progress stopped.
"Expected URL scheme 'http' or 'https' but no scheme was found for" notice appear.
This error happened in any different title with more than 5 pages. Webview working just fine.

Mihon/Tachiyomi version
0.16.5
Android version
Android 13
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] I have updated the app to version 0.15.3.
[X] I have updated all installed extensions.
[X] I have tried the troubleshooting guide.
[X] If this is an issue with the app itself, I should be opening an issue in the app repository.
[X] I will fill out all of the requested information in this form.

Login to your own account in webview and tap Refresh


[1866] Blacklist UICompatibilityInputViewController on iOS 13+

This CL fixes the following issue:  https://github.com/google/EarlGrey/issues/1866
It Blacklists UICompatibilityInputViewController on iOS 13 and above.

This issue has been fixed internally, I'm closing this PR.


[Feature] Generate rayStartParam resource flag from container resource(e.g. cpu,memory)

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Description
Using container.resource.cpu.limit as  num-cpus default value in rayStartParams,  it will help to reduce configuration redundancy. When we want to change the container resource config, we don't need to update rayStartParams.
It's the same with memory,gpu.
Use case
No response
Related issues
No response
Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

Agree. container.resource.cpu.limit should be the source the truth. Let's do this

Perhaps this should be set from requests instead of limits? It's sometimes  beneficial to not set the CPU limit, as it can introduce unwanted throttling.

I think ray is not aware of the (limit-request) resources which leads wasting?  @chenk008 do you know exact behavior on ray side?

It has been addressed in issue https://github.com/ray-project/kuberay/pull/170


Overhaul GeoStyler

@jansule and @KaiVolland talked about an overhaul of the component concept.
Currently the user of a component is stuck to its child components, or has to pass properties via an ObjectProperty like sldRendererProps in the Style component. This feels a bit hackish and doesn't fit the approach of React.
An alternative approach would be to work with renderers:
<Style
  style={style}
  nameRenderer={<SpecialStyleNamefield  onChange={/*TODO*/} />}
  rulesRenderer={
    <RuleTable
      onChange={/*TODO*/}
      ruleRenderer={
        <Rule
          onChange={/*TODO*/}
          nameRenderer={<NameSpecial onChange={/*TODO*/} />}
          symbolizersRenderer={<SpecialSymbolizersRenderer onChange={/*TODO*/} />}
          filterRenderer={undefined} // Use default FilterRenderer
          scaleDenominatorRenderer={false} // Will not be shown
        />
      }
    />
  }
/>

Paradigms:

A component would have a renderer for its (main) child components.
If you don't specifiy a renderer (or undefined) a default component is used instead.
If you provide false nothing will be renderer.

Open Questions:

How to handle the changeListeners and its default behaviour?

Please share your thoughts!

With #1179 we show an approach to customize the fields of an Editor:
The fieldConfig property allows to replace or hide single Fields of an Editor.
It doesn't allow to change the order of the Elements of an Editor, which might be achieved with a children-based approach. The disadvantag of the childre-based approach would that you'd have to add every field even if no change is needed.

See also #1186 for another approach to customize fields. There, react's context feature was used.
This allows the customisation for all customisable components from a global point. The big advantage here is that no props have to be propagated through all parent components. So each component checks for itself if it should be customised. This should avoid errors and keep the code clean. Also, if no provider is being used, the component falls back to its default state.


Upgrade near api js 0.43.1

Upgrade near-api-js to latest.
Continues work done in https://github.com/near/near-wallet/pull/1728 and adds .call to maintain this context here removes v1 usages of account signAndSendTransaction and functionCall in favor of v2 APIs.

@ihaid Please smoke test this PR; we've upgraded near-api-js to the latest here.
Primary possible sources of problems are transaction submission and interactions with lockup accounts and 2fa-enabled accounts.


[Event Request] Codeunit 7324 OnBeforeTempWhseActivLineModify

we Need a new Event trigger OnBeforeTempWhseActivLineModify in codeunit 7324 in procedure InsertTempWhseActivLine:
LOCAL InsertTempWhseActivLine(WhseActivLine : Record "Warehouse Activity Line";ItemTrackingRequired : Boolean)
OnBeforeInsertTempWhseActivLine(WhseActivLine,ItemTrackingRequired);
WITH WhseActivLine DO BEGIN
TempWhseActivLine.SetSourceFilter(
"Source Type","Source Subtype","Source No.","Source Line No.","Source Subline No.",FALSE);
IF TempWhseActivLine.FIND('-') THEN BEGIN
TempWhseActivLine."Qty. to Handle" += "Qty. to Handle";
TempWhseActivLine."Qty. to Handle (Base)" += "Qty. to Handle (Base)";
//++++++++++++++++++++++
OnBeforeTempWhseActivLineModify(VAR TempWhseActivLine,WhseActivLine);
//----------------------
TempWhseActivLine.MODIFY;
END ELSE BEGIN
TempWhseActivLine.INIT;
TempWhseActivLine := WhseActivLine;
OnBeforeTempWhseActivLineInsert(TempWhseActivLine,WhseActivLine);
TempWhseActivLine.INSERT;
IF ItemTrackingRequired AND
("Activity Type" IN ["Activity Type"::"Invt. Pick","Activity Type"::"Invt. Put-away"])
THEN
ItemTrackingMgt.SynchronizeWhseActivItemTrkg(WhseActivLine);
END;
END;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


Added Compound Interest Problem Function

Function to generate Compound Interest Problem.

I'm getting some issues with merge conflicts. Can you remove changes to readme.md?

Which changes?
Can please share details

Remove changes to README.md

Remove changes to README.md
Which changes?
Can please share details


_MODBUSINO_RTU_MAX_ADU_LENGTH is wrong

I started using rather larger modbus tables, and starting hitting all sorts of buffer overflows :(  (I have a very old fork, that was missing some of the earlier fixes for this)
But immediately, the define for RTU_MAX_ADU is just wrong.  Setting this to 256 properly, as per page 5 of modbus app proto 1.1b3 results in requests for more than 64 registers succeeding, but requesting 128 registers for instance, passes validation but overflows.  I know this is the "very very very cut down" version, so I'm not hanging out for a heap of fixes, just letting you know.

OK thank you for your report. I'll just leave a comment about that in the code.


How to compile Compass?

I uses Compass in my project.
When I build .scss file, there is an error.
Error: file to import not found or unreadable: compass/css3
How should I solve this problem?
BTW I can compile .scss with compass in sublime text 3.
I don't what's wrong
My OS is mac os.

Can you post a minimal example of your usage of compass so I can test your problem?


[Match] Changing capabilities on an appId causes match to created duplicate Provisioning Profiles

Issue Description
Apple changed the way the provisioning profiles are invalidated.
Before, we you added a capacity like Push, the Provisioning Profile was not invalidated but it was up to the developer to regenerated it to be sure the functionality worked in his app.
Lately Apple changed this and the PP is now invalidated.
When running match it does not refresh an Invalid profile (by design?) and recreates one called match ... Provisioning Profile <timestamp>
and printing The name 'match ... ' is already taken, using another one.
https://i.imgur.com/Y3S4lFD.png
I believe match should edit or delete+redo this profile.
Currently the only way to fix this is to manually delete the PP from both Apple and the match repo.
I see there is a sigh option to skip fetched profile, but I am not sure what it exactly does and how to pass it to match.
NB: I saw #7099. I think both tickets are linked because we use to use this bug to force a PP renewal after activating a capability
Complete output when running fastlane, including the stack trace and command used
14:35:26 +-----------------------+---------------------------------------------+
14:35:26 |                      Summary for match 0.10.0                       |
14:35:26 +-----------------------+---------------------------------------------+
14:35:26 | git_url               | XXXX |
14:35:26 | git_branch            | whiteroom                                   |
14:35:26 | type                  | enterprise                                  |
14:35:26 | username              | XXXX                     |
14:35:26 | team_id               | XXX                                  |
14:35:26 | team_name             | XXXX                              |
14:35:26 | app_identifier        | com.applidium.radiofrance                   |
14:35:26 | shallow_clone         | false                                       |
14:35:26 | force_for_new_devices | true                                        |
14:35:26 | readonly              | false                                       |
14:35:26 | keychain_name         | login.keychain                              |
14:35:26 | verbose               | false                                       |
14:35:26 | force                 | false                                       |
14:35:26 | skip_docs             | false                                       |
14:35:26 +-----------------------+---------------------------------------------+
14:35:26
14:35:26 [14:35:26]: Enterprise profiles are currently not officially supported in _match_, you might run into issues
14:35:26 [14:35:26]: Cloning remote git repo...
14:35:35 [14:35:35]: Checking out branch whiteroom...
14:35:37 [14:35:37]: 🔓  Successfully decrypted certificates repo
14:35:37 [14:35:37]: Verifying that the certificate and profile are still valid on the Dev Portal...
14:35:41 [14:35:41]: Installing certificate...
14:35:44 +-------------------------------------+-----------------------------------------+
14:35:44 |                            Summary for sigh 1.11.2                            |
14:35:44 +-------------------------------------+-----------------------------------------+
14:35:44 | app_identifier                      | com.applidium.radiofrance               |
14:35:44 | username                            | XXX                 |
14:35:44 | force                               | true                                    |
14:35:44 | cert_id                             | XXX                             |
14:35:44 | provisioning_name                   | match InHouse com.applidium.radiofrance |
14:35:44 | ignore_profiles_with_different_name | true                                    |
14:35:44 | team_id                             | U5M7X5V6ZG                              |
14:35:44 | adhoc                               | false                                   |
14:35:44 | development                         | false                                   |
14:35:44 | skip_install                        | false                                   |
14:35:44 | skip_fetch_profiles                 | false                                   |
14:35:44 | skip_certificate_verification       | false                                   |
14:35:44 +-------------------------------------+-----------------------------------------+
14:35:44
14:35:44 [14:35:44]: Starting login with user 'XXX'
14:35:46 [14:35:46]: Successfully logged in
14:35:46 [14:35:46]: Fetching profiles...
14:35:47 [14:35:47]: Provisioning Profile 'match InHouse com.applidium.radiofrance' is not valid, skipping this one...
14:35:47 [14:35:47]: No existing profiles found, that match the certificates you have installed locally! Creating a new provisioning profile for you
14:35:50 [14:35:50]: The name 'match InHouse com.applidium.radiofrance' is already taken, using another one.
14:35:50 [14:35:50]: Creating new provisioning profile for 'com.applidium.radiofrance' with name 'match InHouse com.applidium.radiofrance<PHONE_NUMBER>'

Environment
[09:19:48]: Generating fastlane environment output, this might take a few seconds...
🚫 fastlane environment 🚫
Stack

Key
Value

OS
10.12.1

Ruby
2.3.1

Bundler?
false

Git
git version 2.10.0

Installation Source
/Users/nicolasbraun/.rbenv/versions/2.3.1/bin/fastlane

Host
Mac OS X 10.12.1 (16B2555)

Ruby Lib Dir
/Users/nicolasbraun/.rbenv/versions/2.3.1/lib

OpenSSL Version
OpenSSL 1.0.2j  26 Sep 2016

Is contained
false

Xcode Path
/Applications/Xcode.app/Contents/Developer/

Xcode Version
8.1

System Locale

Error

No Locale with UTF8 found 🚫

fastlane files:
No Fastfile found
No Appfile found
fastlane gems

Gem
Version
Update-Status

credentials_manager
0.16.2
✅ Up-To-Date

fastlane_core
0.56.0
🚫 Update availaible

spaceship
0.37.0
✅ Up-To-Date

deliver
1.15.1
✅ Up-To-Date

snapshot
1.16.3
🚫 Update availaible

frameit
3.0.0
✅ Up-To-Date

pem
1.4.0
✅ Up-To-Date

cert
1.4.4
✅ Up-To-Date

sigh
1.11.2
✅ Up-To-Date

produce
1.3.0
✅ Up-To-Date

gym
1.12.0
🚫 Update availaible

pilot
1.12.1
✅ Up-To-Date

scan
0.14.1
🚫 Update availaible

supply
0.7.1
✅ Up-To-Date

match
0.11.0
✅ Up-To-Date

screengrab
0.5.6
✅ Up-To-Date

fastlane
1.110.0
✅ Up-To-Date

Loaded fastlane plugins:
No plugins Loaded
Loaded gems

Gem
Version

did_you_mean
1.0.0

highline
1.7.8

commander
4.4.0

terminal-table
1.4.5

json
1.8.3

dotenv
2.1.1

net-sftp
2.1.2

rubyzip
1.1.7

security
0.1.3

krausefx-shenzhen
0.14.11

slack-notifier
1.5.1

xcpretty
0.2.4

terminal-notifier
1.7.1

multipart-post
2.0.0

word_wrap
1.0.0

colored
1.2

babosa
1.0.2

excon
0.54.0

gh_inspector
1.0.2

credentials_manager
0.16.2

fastlane_core
0.56.0

bundler
1.13.1

multi_xml
0.5.5

unf_ext
<IP_ADDRESS>

unf
0.1.4

domain_name
0.5.20161021

http-cookie
1.0.3

faraday-cookie_jar
0.0.6

fastimage
1.6.8

mini_magick
4.5.1

xcpretty-travis-formatter
0.0.4

i18n
0.7.0

thread_safe
0.3.5

tzinfo
1.2.2

activesupport
<IP_ADDRESS>

plist
3.2.0

claide
1.0.1

xcode-install
2.0.9

CFPropertyList
2.3.3

xcodeproj
1.4.1

io-console
0.4.5

I think this issue is still relevant


Update _index.md

👷 Deploy Preview for gladia processing.

Name
Link

🔨 Latest commit
8eec7845f40b00f48510a7d495490dbbafe6b167

🔍 Latest deploy log
https://app.netlify.com/sites/gladia/deploys/63dbbac16450060009c9fccd


Add stpl ("super template") engine

https://github.com/dpc/stpl
https://docs.rs/stpl
https://crates.io/crates/stpl

@bee-git last release was Dec 31, 2017, is it still maintained?

It doesn't look very active: there's only one issue on gh, last commit was on Jul 2018, and the downloads are low. I'm closing it, but feel free to reopen if the things change.

I guess I misunderstood the purpose of AreWeWebYet. I thought it was a resource to surface, in as comprehensive a way as possible, all the resources someone might want to use if they're using Rust for web-related development. I already can use Google to find the leading, most active projects for web-related work. What saves me time is when a resource exists that I can trust to lead me to just about everything out there for a particular task. If you want, you can divide sections into "active projects" and "less active, or inactive, projects". But just limiting the lists to active, popular projects doesn't make the list very helpful to me (and, I suspect, others). It also can prevent projects from getting attention that might deserve attention, and might become active with a bit more interest.

@bereddy thanks for writing! I'd like to clarify what low means

What do you think about the following paragraph?
https://blog.rust-lang.org/inside-rust/2019/10/03/Keeping-secure-with-cargo-audit-0.9.html#new-feature-unmaintained-crate-warnings

I can see how it might be useful to list things like Github stars, number of downloads, and time since last code update in some sort of brief way next to each crate name in each section of AreWeWebYet. If that information is used to also develop a "unmaintained" flag, that's fine.
All I can say is that when I realized that a popularity filter was being put in place as to what crates would be listed on AreWeWebYet, I stopped using AreWeWebYet, and I stopped making suggestions for additions to the lists on the site.
To me, in an age where Google is so good at surfacing what is popular, the purpose of AreWebWebYet should be its comprehensive nature. This is not only useful to people looking for crates to serve their needs. A comprehensive list also gives people interested in creating new crates a better sense of what already has been tried.
I think, for example, that this templating crate, DisplayAs, is very original and could be useful to many people, and might spark others to develop crates similar to it. It's not popular, however, so after the reaction to the less popular crates suggested here, I decided not to bother suggesting it for AreWeWebYet.
In a similar vein, there was a time when Ructe was not particularly well known as a templating engine. Some people pointed it out on Reddit, however, and even though it hardly was known, it started to attract some attention.
It's up to you and the other maintainers of AreWeWebYet to decide what filter you want to use for items to be included in your directories. I simply was surprised, and disappointed, when I found out that the goal was NOT to be as comprehensive as possible.


Fix bug in llama-cloud integration

Disable llama-parse when creating index from framework integration

@Disiok I think your branch was slightly out of date. But I could use this for inspiration


GGUF support

Supports loading Q4_0 models, which use both Q4_0 and Q6_K quantization.
Example
import os
from extra.gguf import load_gguf
from tinygrad import Tensor, dtypes, Device

fn = "Meta-Llama-3-8B-Instruct.Q4_0.gguf"
gguf_tensor = Tensor.empty(os.stat(fn).st_size, dtype=dtypes.uint8, device=f"disk:{fn}").to(Device.DEFAULT)
kv_data, state_dict = load_gguf(gguf_tensor)

TODO

[x] Q8_0 support
[x] end to end tests/testing data alignment
[x] move to nn/state.py
[x] more cleanup
[?] testing assumptions in ggml_tensor_to_numpy

Q8_0 should work also

Now supporting Q4_0, Q4_1, Q6_K and Q8_0 + tests.
I am testing everything besides the assumptions made about the tensor layout in ggml_tensor_to_numpy (test_disk_tensor.py).
The only way I was able to test these assumptions is with the official gguf python package, which is a reimplementation of the libggml logic with numpy. Unfortunately this reimplementation is not feature complete and does not implement quantization for Q6_K, it also only supports numpy < 2 and does not free memory after closing gguf files, which is why I think its better to use unofficial low level bindings to the actual libggml, which allows us to test almost everything.
I have tested everything but Q6_K with the official high level python package, by reimplementing _test_dequantization and _test_load_gguf with it (tests pass). But I think dealing with the gguf package as a dev dependency may induce headaches.
If we want both libraries for testing, I would be happy to include this.

Looking good!
Can you get this llama running? https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF
That'll be my end to end test, then the bounty is yours

I have tried getting LLama3.2 1B to work, but the preexisting Transformer in llama3.py is not compatible with llama3.2. The major part that seems to be missing is RoPE scaling. I have implemented this but still only got gibberish (with the safetensor version). Due to my lack of memory, it is very time consuming to debug the llama examples on my machine.
I have gotten gpt2 running and included it as an example. You can try different quantization methods and compare against the unquantized version.
I hope this is sufficient to validate the change.
If getting llama3.2 to run is a requirement I will have to invest more time into this.
$ PYTHONPATH="." python examples/gpt2-gguf.py --temperature 0 --quant Q4_0
ram used:  0.60 GB, lm_head.weight                                    : 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 149/149 [00:01<00:00, 75.43it/s]
loaded weights in 1975.72 ms, 0.65 GB loaded at 0.33 GB/s
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:01<00:00, 71.77it/s]
What is the answer to life, the universe, and everything?

The answer to life is that we are all connected.

The answer to life is that we are all connected.

The answer to life is that we are all connected.

The answer to life is that we are all connected.

The answer to life is that we are all connected.

The answer to life is that we are all connected.

The answer to life is that we are all connected.

The answer to life is that we

$ PYTHONPATH="." python examples/gpt2.py --model_size "gpt2" --temperature 0
using NV backend
using gpt2
ram used:  0.50 GB, lm_head.weight                                    : 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 149/149 [00:00<00:00, 318.59it/s]
loaded weights in 467.99 ms, 0.65 GB loaded at 1.39 GB/s
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:01<00:00, 76.95it/s]
Generating text...
What is the answer to life, the universe, and everything?

The answer to life, the universe, and everything is that we are all connected.

The answer to life, the universe, and everything is that we are all connected.

The answer to life, the universe, and everything is that we are all connected.

The answer to life, the universe, and everything is that we are all connected.

The answer to life, the universe, and everything is that we are all connected.

The answer to

GPT2 is fine, testing locally now

Congrats, bounty is yours<EMAIL_ADDRESS>to claim, can pay either via PayPal or USDC on ethereum

So it's merged, but I have a bunch of nitpits.

It should be gguf_load, not load_gguf to match torch and safe
It should raise if a GGUF format type is not supported. Right now it returns None
The supported types should be documented

Agree with naming and docs.  I should have seen the naming inconsistency.
"Right now it returns None" - it should raise a KeyError, since there is no block size defined for it.


Pinned comments are duplicated

Comments posted on Odysee posts are duplicated. The error is also displayed in the mobile browser version.

This is a known bug on the comment server side and will be fixed soon, sorry!

It's been fixed!

@tzarebczan missed opportunity not responding "It's been fixed" twice


remove doughbies

doughbies pivoted and shuttered the product i built for them that used this package

I'm sorry for that!
...in case you could share more details I'd like to ear about them (via email?)
Thanks,
Luca


Should throttled be integrated into thinkpad-tools?

This tools keeps the CPU under its highest possible performance by overwriting MSR registers. I think this tool should be implemented under thinkpad-tools to ensure high system performance.
throttled is liscensed under MIT, so it can be integrated into this application. The configuration would be under the same file as the other configurations.
https://github.com/erpalma/throttled

Keep in mind this could be based on platform so the throttled part is only implemented on newer ThinkPads

throttled is also used on machines made by Dell and Microsoft though. And I think many users of thinkpad-tools would not appreciate having throttled on their system if they don't need it (i.e. on older thinkpads). The two projects have different purposes and different use cases.
Are you suggesting that thinkpad-tools should include a fork of throttled, or that the entire project should be moved to thinkpad-tools? I think either option would go against the Unix philosophy.


Method "read_chunk" of "BodyPartReader" returns zero bytes before eof

Long story short
I've implemented a multipart file upload handler inspired on code from the docs. My code is truncating part's data. I believe the problem is in the method _read_chunk_from_stream, which is used by read_chunk of BodyPartReader. That method is returning a zero-length bytearray before the part's EOF. This is the pseudo-code.
reader = await request.multipart()
part = await reader.next()
arr = bytearray()

while True:
    chunk = await part.read_chunk()  # 8192 bytes by default.

    if not chunk:
        break

    arr.extend(chunk)

Expected behaviour
The loop ends when all the part's data has been read.
Actual behaviour
The loop ends before the part's data is exhausted, i.e., chunk becomes a zero-length bytearray prematurely.
Steps to reproduce
The code is part of a large web application so it's hard for me to give reproducible steps. But replacing the break condition to if not part._at_eof made the problem go away.
reader = await request.multipart()
part = await reader.next()
arr = bytearray()

while True:
    chunk = await part.read_chunk()  # 8192 bytes by default.

    if not part._at_eof:  # This fixed the problem.
        break

    arr.extend(chunk)

Your environment
Aiohttp 1.1.5
Python 3.5.1 from PSF
macOS Sierra 10.12.1

I think I've traced the problem back to lines 335 to 338 of multipart.py.
From revision 6edd8256, it seems those lines deal with the case of a small multipart body, but I'm not sure about this. In lines 337 and 338 an empty bytes object is returned explicitly with self._at_eof == False, which is clearly a violation of the self.read_chunk protocol: return zero byte only at EOF.
Removing those lines did not affect the outcome of the test battery in my environment.
I can provide a PR if you like.

@kxepal ?

@viotti
Nice found! Indeed _read_chunk_from_stream may return nothing in some cases fooling you about all content read.
While it looks as a good idea to switch to at_eof flag to break the loop, I think it still not good idea to be able to read nothing and keep looping on wasting CPU.
Feel free to submit a patch anyway! The _read_chunk_from_stream starts to be overly complicated thought /:

Fixed by #1431
The fix is released as aiohttp 1.1.6


Update session.from_id()

Summary
backend should not be a parameter in from_id() because sessions do not support multiple backends. Also, the passed in session_id must be an existing session_id .
Although I'm not sure how useful this will be because a new session created from an existing session id will likely not be accepting jobs (because this happens automatically when the original session's context manager exits).
Details and comments
Addresses this comment https://github.com/Qiskit/qiskit-ibm-runtime/issues/1134#issuecomment-1753198038

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

8 of 9 (88.89%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.03%) to 55.953%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

qiskit_ibm_runtime/session.py
8
9
88.89%

Totals

Change from base Build<PHONE_NUMBER>:
0.03%

Covered Lines:
2472

Relevant Lines:
4418

💛  - Coveralls

@merav-aharoni @jyu00 thoughts on the from_id method? I know we briefly talked about this before but I don't see how this method would be useful after the accepting_jobs change. The original use case (here) was for a way to create a session from a previous session id and then run jobs again - this isn't possible unless we set accepting_jobs back to True

@merav-aharoni @jyu00 thoughts on the from_id method? I know we briefly talked about this before but I don't see how this method would be useful after the accepting_jobs change. The original use case (here) was for a way to create a session from a previous session id and then run jobs again - this isn't possible unless we set accepting_jobs back to True

There were 2 use cases from the original issue:

(This was an offline discussion) A user wants to run multiple jobs in parallel using different processor / container, but within the same session. from_id is still useful in this case as long as they don't use a context manager.
A user is submitting consecutive jobs to the backend within a session, and their local system crashes. This is also still a valid  use case since the crash may happen before the session is closed.


Template

This module doesn't support <1.3.x terraform though. 😄

This module doesn't support <1.3.x terraform though. 😄

True that! Updating!


Integration Test - Release - 2020/08/12 10:00:41

Auto create for integration test. IssueCommandFlow

#pri1

Just type '#pri3'

#pri2

#pri3

#assign PRMergerTest1

#assign NonAllowedUser

NonAllowedUser is not a valid GitHub ID, or is not a collaborator on this repo.

#reassign PRMergerTest2

GitHub user PRMergerTest2 has been removed from the MicrosoftDocs organization, so they were automatically removed as an assignee.

#reassign NonAllowedUser

NonAllowedUser is not a valid GitHub ID, or is not a collaborator on this repo.

#please-close


[MXNET-1110] find the path to include header files

Description
This feature is required by Horovod integration.
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[X] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
[X] Changes are complete (i.e. I finished coding on this PR)
[X] Code is well-documented:
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
[X] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

[X] a new python method

@szha Please help to review if the path is consistent with pip package. Thanks!

@mxnet-label-bot Add [pr-awating-review]

@szha @marcoabreu @anirudh2290 Please help to merge this if no other comment.

@mxnet-label-bot update [pr-awaiting-merge]


Acceptance test ignore fail to wait for PV deletion

…exit

This change is

Review status: 0 of 2 files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


libavcodec.so: error: undefined reference to 'log2'

libavcodec/libavcodec.so: error: undefined reference to 'log2'
libavcodec/libavcodec.so: error: undefined reference to 'log2'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [ffprobe_g] Error 1
make: *** Waiting for unfinished jobs....
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [ffmpeg_g] Error 1
How to solve this problem?
NDK:21.3
What's the reason I haven't found the problem for a long time..

INSTALL	doc/libswresample.3
INSTALL	doc/libavcodec.3
INSTALL	doc/libavformat.3
INSTALL	doc/libavdevice.3
INSTALL	doc/libavfilter.3
LD	ffmpeg_g
libavcodec/libavcodec.so: error: undefined reference to 'log2'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [ffmpeg_g] Error 1

Hi @SagittariusSum ,
I assume you are adding a new external library support.
The log2() is a function from libm library. Some external libraries require a -lm' linker flag to be added to the --extra-ldflags` argument of the FFmpeg's configure script.
I already added a support of adding such a flag in case it is actually needed. Have a look here:
https://github.com/Javernaut/ffmpeg-android-maker/blob/master/scripts/libtwolame/build.sh#L17

@Javernaut
谢谢，我没有加新的第三方插件，不过我先试试加上 "-lm" 试试。

@Javernaut
hi。I didn't add a new third-party plug-in, but I'll try to add "- LM" first.Thank you.

@Javernaut

Is it related to my computer system?     My computer is MacOS Catalina.

@SagittariusSum , are you using the ffmpeg-android-maker or just trying to create your own script for compilation?
If you are using my script, then please specify the command how you got the 'log2' error.
As for the naming for libavcodec.so.XX - you have to add --target-os=android argument to FFMpeg's configure script and that should do the trick removing those digits.

我在使用你的ffmpeg-android-maker

@SagittariusSum , let's stick to English language.
Can you provide me the command that caused the 'log2' error?
And again about the libavcodec.so.XX. I've just checked on macOS Catalina. The terminal prints the name with digits in the end, but actual SO files are installed to the build properly (i.e. without these digits).
Do you need more help on this?

The other third-party libraries are OK, but the libaom reference reports an error.

./ffmpeg-android-maker.sh -all

This command assembles literally everything that is supported right now. Specifically it assembles the encoder of libaom. The thing is the encoder requires at least Android API 18 to be compiled. So the proper command to assemble everything would be ./ffmpeg-android-maker.sh -all -android=18

I think the -all flag is rarely needed, I think. It was made for testing purposes mostly.
The idea of ffmpeg-android-maker is to configure the build scripts to produce the smallest binaries that yet solve your problem.

Thank you very much for your patience. For someone like me who has just learned ffmpeg, your script is very good.
I use - all because the app I want to make is a video editing product.
I'll add a lot of third-party libraries later.
Thanks again


Reloader option. Some ideas

I had some issues using reloader option. I am still trying to discover what happens. I have been reading the bottle.py code and other codes for reloading like these ones:
https://github.com/pallets/werkzeug/blob/master/werkzeug/_reloader.py
https://github.com/django/django/blob/master/django/utils/autoreload.py
https://github.com/lepture/python-livereload/blob/master/livereload/watcher.py
https://github.com/gorakhargosh/watchdog
One potential drawback that I see in the current implementation is that is watching all the python modules, including the modules in the standard library.
One possible idea is to filter the modules in the standard library:
for name, module in sys.modules.items():
    if not module or name in sys.builtin_module_names or not hasattr(module, '__file__'):
        # an import sentinel, built-in module or not a real module, really
        continue

    path = module.__file__
    if path[-4:] in ('.pyo', '.pyc'): path = path[:-1]
    if path and exists(path): files[path] = mtime(path)

Source:
https://stackoverflow.com/questions/22195382/how-to-check-if-a-module-library-package-is-part-of-the-python-standard-library
Another possible idea is to watch changes in a directory, like here:
https://github.com/lepture/python-livereload/blob/master/livereload/watcher.py
One possible improvement that is using django is to create a set of loaded modules, instead of a list, like this:
for module in set(sys.modules.values()):

Bottle is doing this:
for module in list(sys.modules.values()):

Another possible improvement is to use watchdog or pyinotify if they are already installed. Otherwise to use the current implementation as a fallback.
Django is providing a customized solution just in case pyinotify is not installed. Otherwise it uses pyinotify.
Werkzeug is using a similar implementation to the current solution of bottle.py and it provides also a solution using watchdog:
https://github.com/pallets/werkzeug/blob/master/werkzeug/_reloader.py
Werkzeug also does this extra check:
    for module in list(sys.modules.values()):
        if module is None:
            continue
        filename = getattr(module, '__file__', None)
        if filename:
            if os.path.isdir(filename) and \
               os.path.exists(os.path.join(filename, "__init__.py")):
                filename = os.path.join(filename, "__init__.py")

Is there a concrete problem that needs to be solved? I do not see any noticeable delay from watching all modules, instead of just the user provided. It's unnecessary work, but as long as it runs fast enough and does not impact the application performance, I'd prefer a simple and short implementation over a more sophisticated one.

I will try to discover what is the problem and I will report with more details.
The possibility to use watchdog or pyinotify if they are installed and to use the current implementation as a fallback like django or Werkzeug is doing is also interesting because then the code is relying in third party well tested modules specialized to this purpose.

Hi!
Maybe this could be the problem.  The method interrupt_main() only raise  KeyboardInterrupt in the main thread,as the docs say:
thread.interrupt_main()
Raise a KeyboardInterrupt exception in the main thread. A subthread can use this function to interrupt the main thread.

https://docs.python.org/2/library/thread.html
What does it happens if the code of some server catch the exception unintentionally? The thread checking changes in files raise and exception in the main thread and exits. The main thread catch the exception, maybe logging that some exception happened but it continue its executions. Now the web server continue serving pages but there is no thread checking changes in files because that thread exited before, and no new process is created.
For some reason, maybe this is happening to me:
Exception happened during processing of request from ('<IP_ADDRESS>', 39254)
Traceback (most recent call last):
  File "/usr/lib/python2.7/SocketServer.py", line 290, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python2.7/SocketServer.py", line 318, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python2.7/SocketServer.py", line 331, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.7/SocketServer.py", line 652, in __init__
    self.handle()
  File "/usr/lib/python2.7/wsgiref/simple_server.py", line 116, in handle
    self.raw_requestline = self.rfile.readline(65537)
  File "/usr/lib/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
KeyboardInterrupt
----------------------------------------

I analyzed the reloader code of  werzkzeug:
https://github.com/pallets/werkzeug/blob/master/werkzeug/_reloader.py
In that code the server is running in a thread and the code checking for changes in python modules and exiting the application for a new reload is in the main thread. It's the opposite of the current implementation in bottle. I made this changes in bottle and it works for me. I have no problem:
def run(app=None,
        server='wsgiref',
        host='<IP_ADDRESS>',
        port=8080,
        interval=1,
        reloader=False,
        quiet=False,
        plugins=None,
        debug=None,
        config=None, **kargs):
    """ Start a server instance. This method blocks until the server terminates.

        :param app: WSGI application or target string supported by
               :func:`load_app`. (default: :func:`default_app`)
        :param server: Server adapter to use. See :data:`server_names` keys
               for valid names or pass a :class:`ServerAdapter` subclass.
               (default: `wsgiref`)
        :param host: Server address to bind to. Pass ``<IP_ADDRESS>`` to listens on
               all interfaces including the external one. (default: <IP_ADDRESS>)
        :param port: Server port to bind to. Values below 1024 require root
               privileges. (default: 8080)
        :param reloader: Start auto-reloading server? (default: False)
        :param interval: Auto-reloader interval in seconds (default: 1)
        :param quiet: Suppress output to stdout and stderr? (default: False)
        :param options: Options passed to the server adapter.
     """
    if NORUN: return
    if reloader and not os.environ.get('BOTTLE_CHILD'):
        import subprocess
        lockfile = None
        try:
            fd, lockfile = tempfile.mkstemp(prefix='bottle.', suffix='.lock')
            os.close(fd)  # We only need this file to exist. We never write to it
            while os.path.exists(lockfile):
                args = [sys.executable] + sys.argv
                environ = os.environ.copy()
                environ['BOTTLE_CHILD'] = 'true'
                environ['BOTTLE_LOCKFILE'] = lockfile
                p = subprocess.Popen(args, env=environ)
                while p.poll() is None:  # Busy wait...
                    os.utime(lockfile, None)  # I am alive!
                    time.sleep(interval)
                if p.poll() != 3:
                    if os.path.exists(lockfile): os.unlink(lockfile)
                    sys.exit(p.poll())
        except KeyboardInterrupt:
            pass
        finally:
            if os.path.exists(lockfile):
                os.unlink(lockfile)
        return

    try:
        if debug is not None: _debug(debug)
        app = app or default_app()
        if isinstance(app, basestring):
            app = load_app(app)
        if not callable(app):
            raise ValueError("Application is not callable: %r" % app)

        for plugin in plugins or []:
            if isinstance(plugin, basestring):
                plugin = load(plugin)
            app.install(plugin)

        if config:
            app.config.update(config)

        if server in server_names:
            server = server_names.get(server)
        if isinstance(server, basestring):
            server = load(server)
        if isinstance(server, type):
            server = server(host=host, port=port, **kargs)
        if not isinstance(server, ServerAdapter):
            raise ValueError("Unknown or unsupported server: %r" % server)

        server.quiet = server.quiet or quiet
        if not server.quiet:
            _stderr("Bottle v%s server starting up (using %s)...\n" %
                    (__version__, repr(server)))
            _stderr("Listening on http://%s:%d/\n" %
                    (server.host, server.port))
            _stderr("Hit Ctrl-C to quit.\n\n")

        if reloader:
            import signal

            signal.signal(signal.SIGTERM, lambda *args: sys.exit(0))

            t = threading.Thread(target=server.run, args=(app,))
            t.setDaemon(True)
            t.start()

            lockfile = os.environ.get('BOTTLE_LOCKFILE')

            exists = os.path.exists
            mtime = lambda p: os.stat(p).st_mtime
            files = dict()

            for module in list(sys.modules.values()):
                path = getattr(module, '__file__', '')
                if path[-4:] in ('.pyo', '.pyc'): path = path[:-1]
                if path and exists(path): files[path] = mtime(path)

            while True:
                if not exists(lockfile)\
                or mtime(lockfile) < time.time() - interval - 5:
                    sys.exit(3)

                for path, lmtime in list(files.items()):
                    if not exists(path) or mtime(path) > lmtime:
                        sys.exit(3)

                time.sleep(interval)

        else:
            server.run(app)
    except KeyboardInterrupt:
        pass
    except (SystemExit, MemoryError):
        raise
    except:
        if not reloader: raise
        if not getattr(server, 'quiet', quiet):
            print_exc()
        time.sleep(interval)
        sys.exit(3)

I deleted FileCheckerThread because now the code checking changes in python modules is in the main thread and the server is running in a thread:
            t = threading.Thread(target=server.run, args=(app,))
            t.setDaemon(True)
            t.start()

I did this in /usr/lib/python2.7:
$ grep -rn . -e "Exception happened during processing of request from"

The print of that exception is done here:
./SocketServer.py:348:        print 'Exception happened during processing of request from',

No server software should catch and ignore KeyboardInterrupt in the main thread. The exception represents a SIGINT signal (Ctrl+c) and catching it without shutting down the server would prevent a user from closing the server from command-line, also. Are you sure that you really cannot close a wsgiref-server with Ctrl+c? That's really strange.

Hmm, it catches KeyboardInterrupt while handling a request, but between requests it works. This is why this bug is not easily reproduce-able. I'd say it should count as a bug in BaseServer, but getting that fixed in 2.7 is very unlikely.
Starting the server in a thread different than main has other problems, unfortunately. For example, killing it by setting it to deamon and exiting the main-thread would ignore any exception handlers or cleanup code in the application itself :/

KeyboardInterrupt is also captured in  WSGIServer of python 3.6 when a requests is processed. This is the code for the same method _handle_request_noblock(self) of the class BaseServer in socketserver.pymodule, which is inherited by WSGIServer :
    def _handle_request_noblock(self):
        """Handle one request, without blocking.

        I assume that selector.select() has returned that the socket is
        readable before this function was called, so there should be no risk of
        blocking in get_request().
        """
        try:
            request, client_address = self.get_request()
        except OSError:
            return
        if self.verify_request(request, client_address):
            try:
                self.process_request(request, client_address)
            except Exception:
                self.handle_error(request, client_address)
                self.shutdown_request(request)
            except:
                self.shutdown_request(request)
                raise
        else:
            self.shutdown_request(request)

What do you think about this solution?

Customized handle_error method in WSGIServer server. If the exception is KeyboardInterruptthen raise again. Otherwise execute inherited handle_error for the default behaviour. It's possible to check the last exception this way: sys.exc_info()[0] ==  KeyboardInterrupt

For example, changing WSGIRefServer this way in bottle:
class WSGIRefServer(ServerAdapter):
    def run(self, app):  # pragma: no cover
        from wsgiref.simple_server import make_server
        from wsgiref.simple_server import WSGIRequestHandler, WSGIServer
        import socket

        class FixedHandler(WSGIRequestHandler):
            def address_string(self):  # Prevent reverse DNS lookups please.
                return self.client_address[0]

            def log_request(*args, **kw):
                if not self.quiet:
                    return WSGIRequestHandler.log_request(*args, **kw)

        handler_cls = self.options.get('handler_class', FixedHandler)
        server_cls = self.options.get('server_class', WSGIServer)
        if server_cls is WSGIServer:
            def handle_error(self, *args, **kwargs):
                if sys.exc_info()[0] ==  KeyboardInterrupt:
                    raise
                else:
                    return super(WSGIServer, self).handle_error(*args, **kwargs)

            server_cls.handle_error = handle_error

        if ':' in self.host:  # Fix wsgiref for IPv6 addresses.
            if getattr(server_cls, 'address_family') == socket.AF_INET:

                class server_cls(server_cls):
                    address_family = socket.AF_INET6

        self.srv = make_server(self.host, self.port, app, server_cls,
                               handler_cls)
        self.port = self.srv.server_port  # update port actual port (0 means random)
        try:
            self.srv.serve_forever()
        except KeyboardInterrupt:
            self.srv.server_close()  # Prevent ResourceWarning: unclosed socket
            raise

I only added this to WSGIRefServer class:
        if server_cls is WSGIServer:
            def handle_error(self, *args, **kwargs):
                if sys.exc_info()[0] ==  KeyboardInterrupt:
                    raise
                else:
                    return super(WSGIServer, self).handle_error(*args, **kwargs)

            server_cls.handle_error = handle_error

I really hesitate to merge a workaround (#1098) for an obvious bug in a different library. The core problem is that wsgiref catches and ignores KeyboardInterrupt in the main thread. Did you file a bug against python? What did they say?


port lint workflows from CircleCI to GHA

Per title. I don't expect any problems here so I'm ok with removing the lint workflows on CircleCI right away. LMK if you disagree so I can revert d31f3bbb77457cb664277736792557d203df53d0.
cc @seemethere

Can you make these jobs fail so we can get a sense of the workflow for checking the errors?

With c0cb6374e63c6fc98c4fae6b107f5e5cab5345f9:

Python and configs: https://github.com/pytorch/vision/actions/runs/4375903581/jobs/7657254310
C: https://github.com/pytorch/vision/actions/runs/4375903581/jobs/7657253653
mypy: https://github.com/pytorch/vision/actions/runs/4375903581/jobs/7657255081

I looked into why the C lint job emits quite few of these warnings: https://github.com/pytorch/vision/actions/runs/4375903581/jobs/7657253653#step:10:196

./clang-format: /lib64/libtinfo.so.5: no version information available (required by ./clang-format)

According to this SO answer, ncurses from conda defaults channel simply doesn't provide this information. The apparent solution is to install from conda-forge instead.
Since we are not building or testing here, this shouldn't be an issue and I tried that in c0006d0e33c7a2637ac1edc172c3448f1791301d. However the clang-format we are using depends on libtinfo.so.5 that is provided by ncurses=5, which is over 6 years old at this point and also doesn't contain the version info. So this is patched in libtinfo.so.6 from conda-forge, but that doesn't help us here.
This was not an issue on CircleCI since we were able to install it as system package on Ubuntu:
https://github.com/pytorch/vision/blob/e59cf64bb6eb4c3a50e0a76d8019fa4c4d5f2a15/.circleci/config.yml#L295-L297
The new runners use CentOS 7 and I was unable to install it through the system package manager.
Thus, since this is only a warning and doesn't affect the job otherwise, I'm ok to keep it as is. Any concerns @NicolasHug? @osalpekar I don't think there is an option to specify a docker image for the generic Linux job, is there?

still OK for me. We rely on the clang job once in a blue moon anyway


Support for working in the AdiIRC client

As of today, this does not work with AdiIRC Apparently because of the strict binding of DLL only for work in mIRC dialogs. I would like such support to appear in the near future as an alternative, to be able to use this DLL at the same time both in mIRC and in such a good analogue as the AdiIRC client. Both of these programs use the same "mSL" scripting language.
If it is possible to do this make, then I will wait for an update where such extended support will be added. Thanks.

Already suggested on #7 , AdiIRC is much better IRC client and i also wish to see that DLL supported there.


#661 multi-pages subscription

It would be cool to have at least an exemple or better a unit test

@gsoumier @NainJaune  @elebescond done. Please review :)

1/ and 2/ fixed. Also improve bot activation/deactivation methods

When you can't find a connector associated with a pageId, we can throw a InvalidConfigurationException ?

I've just added an error log ;). Also fix a null postback error due to standby webhook


Add remoteEndpoint in RPC spans

This is an enhancement request to support zipkin remoteEndpoint in auto-instrumented RPC spans, something like examples from the docs at https://cloud.spring.io/spring-cloud-sleuth/multi/multi__features.html#_rpc_tracing:
// before you send a request, add metadata that describes the operation
span = tracer.newTrace().name("get").type(CLIENT);
span.tag("clnt/finagle.version", "6.36.0");
span.tag(TraceKeys.HTTP_PATH, "/api");
span.remoteEndpoint(Endpoint.builder()        // <=======   remoteEndpoint
    .serviceName("backend")
    .ipv4(127 << 24 | 1)
    .port(8080).build());

// when the request is scheduled, start the span
span.start();

// if you have callbacks for when data is on the wire, note those events
span.annotate(Constants.WIRE_SEND);
span.annotate(Constants.WIRE_RECV);

// when the response is complete, finish the span
span.finish();

This would be a nice feature and would open up opportunities for users to stream through the spans, and build up service performance analytics without cross-joining with other spans.
For clients, this would require all RPC clients, e.g. FeignClient, to build spans with additional target service names. For server side spans, not sure it requires according changes.
Let me know if this makes sense.

Can you elaborate exactly on what in your opinion is missing in the current instrumentation? I don't think I follow.

For example, from zipkin spans we collected,
  "_source": {
    "traceId": "cd4c1ce935007431",
    "duration": 6000,
    "localEndpoint": {             <==  can we have remoteEndpoint in addition to localEndpoint?
      "serviceName": "user-api",
      "ipv4": "<IP_ADDRESS>",
      "port": 9900
    },
    "timestamp_millis":<PHONE_NUMBER>010,
    "kind": "CLIENT",
    "name": "http:/creditsystem/getusertotalcredit",
    "id": "aed0510b80f4a6f7",
    "parentId": "cd4c1ce935007431",
    "timestamp":<PHONE_NUMBER>010000,
    "tags": {
      "http.host": "<IP_ADDRESS>",
      "http.method": "GET",
      ...
    }
  }

Above span has client-side service name as user-api in its localEndpoint, but lacks target service name in remoteEndpoint.
Similarly in SERVER spans, we only have localEndpoint as the server-side service name, but not knowing which service is calling it. Maybe we can set remoteEndpoint as the calling service name in this case.

Isn't it Zipkin that has the data on both sides (client and server) and can come to such conclusions properly?

True, but we would like to conclude the whole RPC in source -> destination form in a single span. The advantage is that we can subscribe to the sleuth/zipkin MQ broker and streaming through spans to compute analysis data without saving and joining with other spans..
The background is that we want to build performance analysis functions based on zipkin data. Zipkin can query traces from the whole trace aspect vertically. This is nice, but it would be even better if we can look at service endpoint aggregations to give us horizontal statistics.
Hope this explains what we want to achieve.

WDYT @adriancole  ?

in HttpTracing component, you can scope spans to a service with httpTracing.clientOf("your-serviceName").. this is helpful when the remote service name is static for the client. of course you can override this explicitly with Span.remoteEndpoint. However, it might not be ideal as setting this could remove better data already set. Will be interesting to see the api or code impact of what you are interested in.

If we could read Span.remoteEndpoint we could set it only if it wasn't set already.

two things here.. in the next patch of sleuth, Span.remoteServiceName(your_name) will be exposed for writes. In the next minor (hopefully) brave itself will have a handler for aggregation https://github.com/openzipkin/brave/issues/557
either way I think there's no action to take in sleuth at the moment