Error:  Cannot cast array data from dtype('O') to dtype('int64') according to the rule 'safe'

Hello
This is my first time trying NanoRTax and am new to nanopore data analysis.  I am getting an error that I don't see how to fix or workaround.  To move past the bottleneck created by the error I can move the related directories out of the work folder.  The error occurs when running the kraken_push process and seems to be related to when the counts are to be tabulated for the diversity calculations.
Any help is appreciated.
Not sure if this is related but for the read_binning process I had to add the --drop flag.  This error looked to be due to some of the rows in the kraken_report_full that had a value for a count but was otherwise empty -- no information on kingdom, phylum, class, order, family, genus, or species as in the rest of the rows.
Command error:
executor >  local (12)
[d6/648673] process > QC (61)                   [ 14%] 61 of 434, cached: 59
[79/f5a961] process > qc_reporting (33)         [ 55%] 33 of 60, cached: 31
[e3/41d78b] process > read_binning_kraken (60)  [100%] 60 of 60, cached: 58
[b6/4d0d4a] process > agg_kraken (49)           [ 84%] 49 of 58, cached: 46
[52/cba34b] process > kraken_push (24)          [ 52%] 25 of 48, cached: 23, failed: 1
[4e/030042] process > agg_kraken_diversity (23) [100%] 23 of 23, cached: 22
[bc/b73cab] process > output_documentation      [100%] 1 of 1, cached: 1 ✔
Execution cancelled -- Finishing pending tasks before exit
-[nf-core/rtnanopipeline] Pipeline completed with errors-
WARN: There's no process matching config selector: get_software_versions
Error executing process > 'kraken_push (23)'
Caused by:
Process kraken_push (23) terminated with an error exit status (1)
Command executed [/athena/home/beatond/Tools/NanoRTax/templates/kraken_push.py]:
#!/usr/bin/env python3
import datetime
import re
import pandas as pd
import skbio
df = pd.read_csv("kraken_report_full.txt", delimiter="        ", names=['seq_id', 'tax_id', 'kingdom', 'phylum', 'class', 'order', 'family', 'genus', 'species'])
tax_table = df['class'].value_counts()
tax_table_class = pd.DataFrame(list(zip(tax_table.index.tolist(), tax_table.tolist())), columns =['tax_id', 'read_count'])
tax_table_class.to_csv("kraken_report_class.csv", header=1, columns=["tax_id", "read_count"], index=False)
tax_table = df['order'].value_counts()
tax_table_order = pd.DataFrame(list(zip(tax_table.index.tolist(), tax_table.tolist())), columns =['tax_id', 'read_count'])
tax_table_order.to_csv("kraken_report_order.csv", header=1, columns=["tax_id", "read_count"], index=False)
tax_table = df['family'].value_counts()
tax_table_family = pd.DataFrame(list(zip(tax_table.index.tolist(), tax_table.tolist())), columns =['tax_id', 'read_count'])
tax_table_family.to_csv("kraken_report_family.csv", header=1, columns=["tax_id", "read_count"], index=False)
tax_table = df['genus'].value_counts()
tax_table_genus = pd.DataFrame(list(zip(tax_table.index.tolist(), tax_table.tolist())), columns =['tax_id', 'read_count'])
tax_table_genus.to_csv("kraken_report_genus.csv", header=1, columns=["tax_id", "read_count"], index=False)
tax_table = df['species'].value_counts()
tax_table_species = pd.DataFrame(list(zip(tax_table.index.tolist(), tax_table.tolist())), columns =['tax_id', 'read_count'])
tax_table_species.to_csv("kraken_report_species.csv", header=1, columns=["tax_id", "read_count"], index=False)
shannon = skbio.diversity.alpha.shannon(tax_table_class['read_count'])
simpson = skbio.diversity.alpha.simpson(tax_table_class['read_count'])
file1 = open("kraken_diversity_class.csv","w")
file1.write(str(tax_table_class['read_count'].sum()) + "," + str(round(shannon,3)) + "," + str(round(simpson,3)) + "\n")
file1.close()
shannon = skbio.diversity.alpha.shannon(tax_table_order['read_count'])
simpson = skbio.diversity.alpha.simpson(tax_table_order['read_count'])
file1 = open("kraken_diversity_order.csv","w")
file1.write(str(tax_table_order['read_count'].sum()) + "," + str(round(shannon,3)) + "," + str(round(simpson,3)) + "\n")
file1.close()
shannon = skbio.diversity.alpha.shannon(tax_table_family['read_count'])
simpson = skbio.diversity.alpha.simpson(tax_table_family['read_count'])
file1 = open("kraken_diversity_family.csv","w")
file1.write(str(tax_table_family['read_count'].sum()) + "," + str(round(shannon,3)) + "," + str(round(simpson,3)) + "\n")
file1.close()
shannon = skbio.diversity.alpha.shannon(tax_table_genus['read_count'])
simpson = skbio.diversity.alpha.simpson(tax_table_genus['read_count'])
file1 = open("kraken_diversity_genus.csv","w")
file1.write(str(tax_table_genus['read_count'].sum()) + "," + str(round(shannon,3)) + "," + str(round(simpson,3)) + "\n")
file1.close()
shannon = skbio.diversity.alpha.shannon(tax_table_species['read_count'])
simpson = skbio.diversity.alpha.simpson(tax_table_species['read_count'])
file1 = open("kraken_diversity_species.csv","w")
file1.write(str(tax_table_species['read_count'].sum()) + "," + str(round(shannon,3)) + "," + str(round(simpson,3)) + "\n")
file1.close()
#diversity = pd.DataFrame(data={"Shannon index": [round(shannon,3)], "Simpson index": [round(simpson, 3)]})
#diversity.to_csv("kraken_diversity.csv", sep=',',index=False, header=0)
Command exit status:
1
Command output:
(empty)
Command error:
/athena/home/beatond/Data_analysis/Nanopore_data/crosswise/NanoRTax_test/work/conda/nanortax-8a6750e7c29e704fe98c2ea094c85a6b/lib/python3.6/site-packages/skbio/util/_testing.py:16: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
import pandas.util.testing as pdt
Traceback (most recent call last):
File ".command.sh", line 30, in
shannon = skbio.diversity.alpha.shannon(tax_table_class['read_count'])
File "/athena/home/beatond/Data_analysis/Nanopore_data/crosswise/NanoRTax_test/work/conda/nanortax-8a6750e7c29e704fe98c2ea094c85a6b/lib/python3.6/site-packages/skbio/diversity/alpha/_base.py", line 868, in shannon
counts = _validate_counts_vector(counts)
File "/athena/home/beatond/Data_analysis/Nanopore_data/crosswise/NanoRTax_test/work/conda/nanortax-8a6750e7c29e704fe98c2ea094c85a6b/lib/python3.6/site-packages/skbio/diversity/_util.py", line 26, in _validate_counts_vector
counts = counts.astype(int, casting='safe', copy=False)
TypeError: Cannot cast array data from dtype('O') to dtype('int64') according to the rule 'safe'
Work dir:
/athena/home/beatond/Data_analysis/Nanopore_data/crosswise/NanoRTax_test/work/71/d8b8dda7cc3212135eda44829a8a67
Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out

Hello -- I have done more digging.  The error described above appears to follow from the conversion of the krkinput.fastq file to the krakenreport.txt file.
kraken_report_annotated_otu.txt has otu information from kingdom to species.
kraken_report_annotated.txt only has  information to the level of kingdom.
The diversity analysis in kraken_push.py starts that the level of class so the report files have no data to analyze and create the the diversity files.
The seqid for the first taxon id in the list is shown in the file to be on row 2 rather than row 1.
Any suggestions on how to fix or workaround and see if it occurs in other qced.fastq files in the sequencing run?

After much looking, the issue here is with the fats5 conversion to fastq.


feature/SO-4919_FHIR_operation_definition

Codecov Report

Merging #899 (94c63f6) into 8.x (41bc2a5) will increase coverage by 0.26%.
The diff coverage is 83.82%.

@@             Coverage Diff              @@
##                8.x     #899      +/-   ##
============================================
+ Coverage     62.65%   62.91%   +0.26%
- Complexity    11541    11677     +136
============================================
  Files          1712     1721       +9
  Lines         55144    55747     +603
  Branches       5281     5328      +47
============================================
+ Hits          34548    35073     +525
- Misses        18406    18455      +49
- Partials       2190     2219      +29

Impacted Files
Coverage Δ

...tional/snowowl/fhir/core/model/DomainResource.java
77.27% <ø> (ø)

...onal/snowowl/fhir/core/model/MetadataResource.java
90.00% <ø> (-0.43%)
:arrow_down:

...l/fhir/core/request/FhirResourceSearchRequest.java
84.18% <0.00%> (-6.73%)
:arrow_down:

...rnational/snowowl/core/rest/OpenAPIExtensions.java
0.00% <0.00%> (ø)

...al/snowowl/fhir/rest/FhirCodeSystemController.java
90.90% <ø> (ø)

.../rest/FhirCodeSystemLookupOperationController.java
65.00% <ø> (ø)

...ternational/snowowl/fhir/core/model/Extension.java
66.66% <42.85%> (-17.95%)
:arrow_down:

...ore/model/capabilitystatement/BackboneElement.java
45.45% <45.45%> (ø)

...l/snowowl/fhir/core/codesystems/OperationKind.java
57.14% <57.14%> (ø)

...fhir/core/codesystems/CapabilityStatementKind.java
62.50% <62.50%> (ø)

... and 98 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 41bc2a5...94c63f6. Read the comment docs.


Add a plugin to get remote IP from HTTP-Header

As in #1827 discussed the remote IP  from a client could be different the the remoteAddr Variable. Some Applications, nowadays, run behind a reverse proxy and send the real client IP in http headers like X-Forwarded-For or Forwarded HTTP Extension which should be parsed and set the remoteAddr, or another Variable, with that header value.
There are two to-do's from my point of view.
[ ] Implement Forwarded and/or X-Forwarded-For Parser in a fresh plugin
[ ] Create a example to show the usage of the plugin.

No, as described in #1827 this should not live in the Fresh code base at all, neither directly in Fresh, nor as a plugin. Adding it to the Fresh repository would imply that we would keep it up to date and maintain it. We won't do that. It should either live in the project's code directly as a middleware or as a plugin in a separate repository. I'm happy to point to that from the docs once a repository for a plugin exists or something like that.


pkg/trace/{api,agent,pb}: introduce new client payload formats: TracerPayload & TraceChunk

What does this PR do?
1. Updates the trace payload from agent to backend
Before:
message TracePayload {
        string hostName = 1;
        string env = 2;
        repeated APITrace traces = 3;
        repeated Span transactions = 4;
}

After:
message TracePayload {
        string hostName = 1;
        string env = 2;
        repeated Span transactions = 4;
        repeated TracerPayload tracerPayloads = 5;
        map<string, string> tags = 6;
        string agentVersion = 7;
}

On backend, we would like to receive the payloads from the tracer as they are.
We also updated the payload from tracer to agent.
Before:
[][]*Span

After:
message TraceChunk {
	repeated Span spans = 1;
	map<string, string> tags = 2;
	int32 priority = 3;
	string origin = 4;
}

message TracerPayload {
	repeated TraceChunk chunks = 1;
	map<string, string> tags = 2;
	string service = 3;
	int64 traceCount = 4;
	string containerID = 5;
	string lang = 6;
	string langVersion = 7;
	string langInterpreter = 8;
	string langInterpreterVendor = 9;
	string tracerVersion = 10;
	bool computedTopLevel = 11;
	bool computedStats = 12;
	int32 sequence = 13;
	string runtimeID = 14;
}

2. Adds a new endpoint to receive the payload in new format from tracers

/v0.6/traces

3. [serverless] Sets global tags at agent payload level instead of each span

we can propagate the global tags on backend
it helps the customers to reduce their traffic to us

4. [container] Sets container tags at tracer payload level instead of each top level span

we can propagate container tags on backend
it helps the customers to reduce their traffic to us

5. Removes redundant data

Removes _dd.origin tag from each spans as it's set at trace chunk level
Removes _sampling_priority_v1  tag from each spans as it's set at trace chunk level

6. Removed Transactions from the payload from agent to backend
Spans in Transactions are a subset of spans in TracerPayloads. Instead of duplicating them in Transactions, we decided to keep them in TracerPayload, and just mark them with _dd.analyzed.
Motivation
We would like receive the following information from the tracer to enrich our features.

agent tags
tracer tags
trace tags

To do that, we need to receive two level of structured data from tracer instead of [][]*Span.
Additional Notes
This PR doesn't contain any logical change. But, there will be 2 more PRs after this one gets merged.

A new endpoint in trace-agent that receives structured payload from traces
A new trace payload format we are going to use in future intake

Describe how to test your changes
As this PR doesn't introduce a new feature,

[ ] making sure it didn't break anything
[ ] making sure the performance wasn't degraded

in the usual environment should enough for the test.
Checklist

[ ] A release note has been added or the changelog/no-changelog label has been applied.
[ ] The need-change/operator and need-change/helm labels has been applied if applicable.
[ ] The appropriate team/.. label has been applied, if known.
[ ] If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
[ ] The config template has been updated if applicable.

Note: Adding GitHub labels is only possible for contributors with write access.

Hi,
As discussed offline, this PR changes are coupled to (and blocked by) this backend changes: https://github.com/DataDog/logs-backend/pull/25249
We need to ensure backend changes are merged before we merge Agent changes.


[BugFix]: Potential data errors in primary tables using persistent indexes

What type of PR is this：

[x] bug
[ ] feature
[ ] enhancement
[ ] refactor
[ ] others

Which issues of this PR fixes ：

Fixes #7416
Problem Summary(Required) ：

In function PrimaryIndex::_insert_into_persistent_index(), if values in rowids is not continuous, we will get error value after call function PrimaryIndex::_build_persistent_values which may cause data error of primary table using persistent index.

@decster @chaoyli PTAL if you are available

It seems issue #7525 is related to this bug, i will check it later.


Wishlist: reconect GCM on connectivity change

When I've seen that correctly, an automatic reconnect doesn't take place when the connectivity, e.g. from mobile to WIFI changes, right? Is it possible to add this?

Interestingly, I have not noticed any problems when it comes to this. The app Autostarts lists uG services as one of the apps that is notified when the connectivity changes, and as far as I can tell, the uG services always reconnected when I switched from mobile to WiFi and vice versa. (OS Monitor showed that the connection was re-established.)
Did you have any problems, @bonanza123 ? (An honest question, I do not doubt that you have a serious reason to post your question here. :-) )

I don't have any real problems. But in my build I extended the heartbeat interval to 10 minutes to reduce the wakeups ( if I remember correctly, googles default is something around 15 - 20 minutes). And additionally, I have often switches from mobile to Wifi and back.
In the case the app would automatically reconnect on a connectivity change and would not be waiting for the next interval, the "dead period" would be reduced. (note that I'm not sure whether the app doesn't reconnect on connectivity change, meaning I haven't had a deep look into the code, I just didn't notice it yet)

An option to increase the heartbeat interval was in fact a suggestion I wanted to post. :+1:
My guess (based on my observations) is that a connectivity change lets uG reconnect immediately: I did not notice any delay.
Even if the reconnect on connectivity change works, I am not sure whether the automatic and immediate reconnect is triggered when the uG services were terminated (and restarted) by Android because of the lack of system resources (or because someone has restarted the service with OS Monitor). In this case, uG may indeed wait for the next interval.

I just had a look at the 'catlog' and the heartbeat messages that are logged there. Based on them, I was not able to notice immediate reconnects after the connectivity change.
From my very limited android knowledge, I guess the 'connectivity change' is a broadcast to which apps can listen. So I guess that would also enable µg to restart and reconnect (but I'm not sure).

Oh, so this is the issue I'm experiencing! I've described it in #23. Tested with Push Notifications tester. Sometimes it wont receive push messages, so I restart my phone and the messages arrive immediately. Now I know why. I've set wifi sleep policy to off, so when my phone is in deep sleep, wifi is off. This obviously affects microG.

Some logs. Procedure:

restart phone
check if push messages arrive on system - yes
turn off wifi
turn wifi back on
check if push messages arrive on system  - NO, they don't

u0_a52@p880:/ $ su
root@p880:/ # logcat | grep Gms
D/GmsWearSvc(  884): onBind: Intent { act=com.google.android.gms.wearable.BIND pkg=com.google.android.gms }
D/GmsServiceBroker(  884): onTransact [unknown]: 26, android.os.Parcel@1a4d4740, 16
D/GmsGcmMcsSvc(  884): Heartbeat initiated, reason: Intent { act=org.microg.gms.gcm.mcs.HEARTBEAT flg=0x4 cmp=com.google.android.gms/org.microg.gms.gcm.McsService (has extras) }
D/GmsGcmMcsSvc(  884): Outgoing message: HeartbeatPing{last_stream_id_received=9}
D/GmsGcmMcsSvc(  884): Incoming message: HeartbeatAck{last_stream_id_received=9}
D/GmsGcmMcsSvc(  884): Heartbeat initiated, reason: Intent { act=org.microg.gms.gcm.mcs.HEARTBEAT flg=0x4 cmp=com.google.android.gms/org.microg.gms.gcm.McsService (has extras) }
D/GmsGcmMcsSvc(  884): Outgoing message: HeartbeatPing{last_stream_id_received=10}
D/GmsGcmMcsSvc(  884): Incoming message: HeartbeatAck{last_stream_id_received=10}
D/GmsGcmMcsSvc(  884): Heartbeat initiated, reason: Intent { act=org.microg.gms.gcm.mcs.HEARTBEAT flg=0x4 cmp=com.google.android.gms/org.microg.gms.gcm.McsService (has extras) }
D/GmsGcmMcsSvc(  884): Outgoing message: HeartbeatPing{last_stream_id_received=11}
D/GmsGcmMcsSvc(  884): Incoming message: HeartbeatAck{last_stream_id_received=11}
D/GmsGcmRegisterSvc(  884): onHandleIntent: Intent { act=com.google.android.c2dm.intent.REGISTER pkg=com.google.android.gms (has extras) }
D/GmsGcmRegisterSvc(  884): register[req]: Bundle[{sender=197533215920, app=PendingIntent{1e074e9a: android.os.BinderProxy@282b93cb}, google.messenger=android.os.Messenger@3b97f7a8}]
D/GmsHttpFormClient(  884): -- Request --
D/GmsHttpFormClient(  884): X-GOOG.USER_AID=4563711268479711284&device=4563711268479711284&app=com.firstrowria.pushnotificationtester&cert=1acbe3831cb1827bee6a78879501d7c3bc074987&app_ver=7&sender=197533215920
D/GmsHttpFormClient(  884): -- Response --
D/GmsHttpFormClient(  884): token=APA91bE_RRUddutPhB8fp8hUnxaETLiFFsAI7H_kHzlORSKy-4V_MYv7oA-UJISiWTZx8aPJS4jrtQMRdyEcJ1LlFVOuFrJLeD-03HFZSV8fgfytKifJpNr6hy-6-L_0L87Zmytnpb3kto9X5DHSLASCvWy04p-R_w
D/GmsGcmRegisterSvc(  884): received response: RegisterResponse{token='APA91bE_RRUddutPhB8fp8hUnxaETLiFFsAI7H_kHzlORSKy-4V_MYv7oA-UJISiWTZx8aPJS4jrtQMRdyEcJ1LlFVOuFrJLeD-03HFZSV8fgfytKifJpNr6hy-6-L_0L87Zmytnpb3kto9X5DHSLASCvWy04p-R_w', retryAfter='null', deleted='null'}
D/GmsGcmRegisterSvc(  884): register[res]: Intent { act=com.google.android.c2dm.intent.REGISTRATION pkg=com.firstrowria.pushnotificationtester (has extras) } extras=Bundle[{registration_id=APA91bE_RRUddutPhB8fp8hUnxaETLiFFsAI7H_kHzlORSKy-4V_MYv7oA-UJISiWTZx8aPJS4jrtQMRdyEcJ1LlFVOuFrJLeD-03HFZSV8fgfytKifJpNr6hy-6-L_0L87Zmytnpb3kto9X5DHSLASCvWy04p-R_w}]
D/GmsGcmMcsSvc(  884): Incoming message: DataMessageStanza{id=462BC431, from=197533215920, category=com.firstrowria.pushnotificationtester, token=do_not_collapse, app_data=[AppData{key=serverTime, value=1444297978421}, AppData{key=title, value=Test notification}], from_trusted_server=false, persistent_id=0:1444297978446601%c5da679ff9fd7ecd, sent=1444297978427}
D/GmsGcmRegisterSvc(  884): onHandleIntent: Intent { act=com.google.android.c2dm.intent.UNREGISTER pkg=com.google.android.gms (has extras) }
D/GmsHttpFormClient(  884): -- Request --
D/GmsHttpFormClient(  884): X-GOOG.USER_AID=4563711268479711284&device=4563711268479711284&app=com.firstrowria.pushnotificationtester&cert=1acbe3831cb1827bee6a78879501d7c3bc074987&app_ver=7&delete=1
D/GmsHttpFormClient(  884): -- Response --
D/GmsHttpFormClient(  884): deleted=com.firstrowria.pushnotificationtester
D/GmsGcmRegisterSvc(  884): received response: RegisterResponse{token='null', retryAfter='null', deleted='com.firstrowria.pushnotificationtester'}
D/GmsGcmRegisterSvc(  884): unregister[res]: Intent { act=com.google.android.c2dm.intent.REGISTRATION pkg=com.firstrowria.pushnotificationtester (has extras) } extras=Bundle[{unregistered=com.firstrowria.pushnotificationtester}]
D/GmsGcmMcsSvc(  884): Heartbeat initiated, reason: Intent { act=org.microg.gms.gcm.mcs.HEARTBEAT flg=0x4 cmp=com.google.android.gms/org.microg.gms.gcm.McsService (has extras) }
D/GmsGcmMcsSvc(  884): Outgoing message: HeartbeatPing{last_stream_id_received=13}
D/GmsGcmMcsSvc(  884): Incoming message: HeartbeatAck{last_stream_id_received=12}
D/GmsGcmMcsSvc(  884): Heartbeat initiated, reason: Intent { act=org.microg.gms.gcm.mcs.HEARTBEAT flg=0x4 cmp=com.google.android.gms/org.microg.gms.gcm.McsService (has extras) }
D/GmsGcmMcsSvc(  884): Outgoing message: HeartbeatPing{last_stream_id_received=14}
D/GmsGcmMcsSvc(  884): Incoming message: HeartbeatAck{last_stream_id_received=13}
D/GmsGcmMcsSvc(  884): Heartbeat initiated, reason: Intent { act=org.microg.gms.gcm.mcs.HEARTBEAT flg=0x4 cmp=com.google.android.gms/org.microg.gms.gcm.McsService (has extras) }
D/GmsGcmMcsSvc(  884): Outgoing message: HeartbeatPing{last_stream_id_received=15}
D/GmsGcmMcsSvc(  884): Incoming message: HeartbeatAck{last_stream_id_received=14}
D/GmsGcmMcsSvc(  884): Heartbeat initiated, reason: Intent { act=org.microg.gms.gcm.mcs.HEARTBEAT flg=0x4 cmp=com.google.android.gms/org.microg.gms.gcm.McsService (has extras) }
D/GmsGcmMcsSvc(  884): Outgoing message: HeartbeatPing{last_stream_id_received=16}
D/GmsGcmMcsSvc(  884): Incoming message: HeartbeatAck{last_stream_id_received=15}
D/GmsGcmMcsSvc(  884): Heartbeat initiated, reason: Intent { act=org.microg.gms.gcm.mcs.HEARTBEAT flg=0x4 cmp=com.google.android.gms/org.microg.gms.gcm.McsService (has extras) }
D/GmsGcmMcsSvc(  884): Outgoing message: HeartbeatPing{last_stream_id_received=17}
D/GmsGcmMcsSvc(  884): Incoming message: HeartbeatAck{last_stream_id_received=16}
D/GmsGcmMcsSvc(  884): Heartbeat initiated, reason: Intent { act=org.microg.gms.gcm.mcs.HEARTBEAT flg=0x4 cmp=com.google.android.gms/org.microg.gms.gcm.McsService (has extras) }
D/GmsGcmMcsSvc(  884): Outgoing message: HeartbeatPing{last_stream_id_received=18}
D/GmsGcmMcsSvc(  884): Incoming message: HeartbeatAck{last_stream_id_received=17}
D/GmsGcmMcsSvc(  884): I/O error: javax.net.ssl.SSLException: Read error: ssl=0xaf837200: I/O error during system call, Connection timed out
D/GmsGcmMcsSvc(  884): Teardown initiated, reason: javax.net.ssl.SSLException: Read error: ssl=0xaf837200: I/O error during system call, Connection timed out
D/GmsGcmTrigger(  884): Ignoring Intent { act=org.microg.gms.gcm.RECONNECT flg=0x10 cmp=com.google.android.gms/org.microg.gms.gcm.TriggerReceiver }: network is offline, scheduling new attempt.
D/GmsGcmTrigger(  884): Ignoring Intent { act=android.net.conn.CONNECTIVITY_CHANGE flg=0x4000010 cmp=com.google.android.gms/org.microg.gms.gcm.TriggerReceiver (has extras) }: network is offline, scheduling new attempt.
D/GmsGcmTrigger(  884): Ignoring Intent { act=android.net.conn.CONNECTIVITY_CHANGE flg=0x4000010 cmp=com.google.android.gms/org.microg.gms.gcm.TriggerReceiver (has extras) }: network is offline, scheduling new attempt.
D/GmsGcmTrigger(  884): Ignoring Intent { act=org.microg.gms.gcm.RECONNECT flg=0x14 cmp=com.google.android.gms/org.microg.gms.gcm.TriggerReceiver (has extras) }: network is offline, scheduling new attempt.
D/GmsGcmTrigger(  884): Ignoring Intent { act=org.microg.gms.gcm.RECONNECT flg=0x14 cmp=com.google.android.gms/org.microg.gms.gcm.TriggerReceiver (has extras) }: network is offline, scheduling new attempt.
D/GmsGcmMcsSvc(  884): Connect initiated, reason: Intent { act=org.microg.gms.gcm.mcs.CONNECT cmp=com.google.android.gms/org.microg.gms.gcm.McsService (has extras) }
D/GmsGcmMcsSvc(  884): Starting MCS connection...

W/GmsGcmMcsSvc(  884): javax.net.ssl.SSLHandshakeException: SSL handshake aborted: ssl=0xb4488200: I/O error during system call, Connection refused
W/GmsGcmMcsSvc(  884):  at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(Native Method)
W/GmsGcmMcsSvc(  884):  at com.android.org.conscrypt.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:318)
W/GmsGcmMcsSvc(  884):  at com.android.org.conscrypt.OpenSSLSocketImpl.waitForHandshake(OpenSSLSocketImpl.java:623)
W/GmsGcmMcsSvc(  884):  at com.android.org.conscrypt.OpenSSLSocketImpl.getInputStream(OpenSSLSocketImpl.java:585)
W/GmsGcmMcsSvc(  884):  at org.microg.gms.gcm.McsService.connect(McsService.java:177)
W/GmsGcmMcsSvc(  884):  at org.microg.gms.gcm.McsService.handleMessage(McsService.java:316)
W/GmsGcmMcsSvc(  884):  at android.os.Handler.dispatchMessage(Handler.java:98)
W/GmsGcmMcsSvc(  884):  at org.microg.gms.gcm.McsService.onHandleIntent(McsService.java:157)
W/GmsGcmMcsSvc(  884):  at android.app.IntentService$ServiceHandler.handleMessage(IntentService.java:65)
W/GmsGcmMcsSvc(  884):  at android.os.Handler.dispatchMessage(Handler.java:102)
W/GmsGcmMcsSvc(  884):  at android.os.Looper.loop(Looper.java:135)
W/GmsGcmMcsSvc(  884):  at android.os.HandlerThread.run(HandlerThread.java:61)
D/GmsGcmMcsSvc(  884): Teardown initiated, reason: javax.net.ssl.SSLHandshakeException: SSL handshake aborted: ssl=0xb4488200: I/O error during system call, Connection refused
D/GmsCheckinSvc(  884): Checked in as 3f558f38172e3834
^C
130|root@p880:/ #

listening on connectivity changes and using it to restart the GCM service was always implemented. I guess the problem here is something else. 766a6a1 hopefully fixes these problems, else please report over in #31 or create a corresponding bug report (with logs).


Parse output of dafny --version to omit build information in dafnyVersionCheck

Dafny started outputting strings like 4.9.0+25fa1000744d8d8a9a6a84c0712149daeda6f67e which isn't an exact match for a string like 4.9.0.

I have an existing DafnyVersion class here that could be useful for this (as I couldn't find any semver 2.0 parser in a small Java dependency anywhere): https://github.com/smithy-lang/smithy-dafny/blob/main-1.x/codegen/smithy-dafny-codegen/src/main/java/software/amazon/polymorph/smithydafny/DafnyVersion.java


NativeProxy for native class don't represent the erased type (used T instead)

fixed in 895cd8bf3e416161ccf363f6aa99cd7f04840199


CoverageScript.command_line wipes out 1st sys.path entry.

This can be problematic if the coverage distribution itself is not the first item on the classpath:
https://github.com/nedbat/coveragepy/blob/401471c1c51c2a99b31b0508578d04b4da03eebb/coverage/cmdline.py#L468-L470
Is there any reason this couldn't be a sys.path.insert(0, '')?
I'm happy to do the work but I just wanted to run this by you first.

I'd feel better about changing this if you could show me a bad behavior that the current code causes.

OK.
CI is here, but its a wall of text: https://travis-ci.org/pantsbuild/pants/jobs/437335398
The relevant bug presentation is configuring coverage with a plugin, that plugin being on the sys.path (in the 1st slot as it turns out) and then coverage mysteriously not being able to find the plugin::
11:20:19 00:02       [coverage-html]
                     pex: PEX.run invoking /home/jsirois/dev/pantsbuild/jsirois-pants/build-support/pants_dev_deps.venv/bin/python2.7 /home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/bdd83c141bb9cfffde291c49aa9e41781cc24326 html -i --rcfile /home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest/testprojects.tests.python.pants.constants_only.constants_only/tmpW10vvZ -d /home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest/testprojects.tests.python.pants.constants_only.constants_only/coverage
Traceback (most recent call last):
                       File "/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/bdd83c141bb9cfffde291c49aa9e41781cc24326/.bootstrap/_pex/pex.py", line 351, in execute
                         self._wrap_coverage(self._wrap_profiling, self._execute)
                       File "/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/bdd83c141bb9cfffde291c49aa9e41781cc24326/.bootstrap/_pex/pex.py", line 281, in _wrap_coverage
                         runner(*args)
                       File "/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/bdd83c141bb9cfffde291c49aa9e41781cc24326/.bootstrap/_pex/pex.py", line 313, in _wrap_profiling
                         runner(*args)
                       File "/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/bdd83c141bb9cfffde291c49aa9e41781cc24326/.bootstrap/_pex/pex.py", line 390, in _execute
                         return self.execute_entry(self._pex_info_overrides.entry_point)
                       File "/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/bdd83c141bb9cfffde291c49aa9e41781cc24326/.bootstrap/_pex/pex.py", line 514, in execute_entry
                         return runner(entry_point)
                       File "/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/bdd83c141bb9cfffde291c49aa9e41781cc24326/.bootstrap/_pex/pex.py", line 532, in execute_pkg_resources
                         return runner()
                       File "/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/coverage-4.5.1-cp27-cp27mu-manylinux1_x86_64.whl/coverage/cmdline.py", line 755, in main
                         status = CoverageScript().command_line(argv)
                       File "/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/coverage-4.5.1-cp27-cp27mu-manylinux1_x86_64.whl/coverage/cmdline.py", line 507, in command_line
                         self.coverage.load()
                       File "/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/coverage-4.5.1-cp27-cp27mu-manylinux1_x86_64.whl/coverage/control.py", line 676, in load
                         self._init()
                       File "/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/coverage-4.5.1-cp27-cp27mu-manylinux1_x86_64.whl/coverage/control.py", line 227, in _init
                         self.plugins = Plugins.load_plugins(self.config.plugins, self.config, self.debug)
                       File "/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/coverage-4.5.1-cp27-cp27mu-manylinux1_x86_64.whl/coverage/plugin_support.py", line 41, in load_plugins
                         __import__(module)
                     ImportError: No module named __pants_backend_python_tasks_pytest_prep__

At the handoff point from executing code, the particular sys.path is:
 >>> Executing coverage.cmdline:main using sys.path:
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/bdd83c141bb9cfffde291c49aa9e41781cc24326
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/build-support/pants_dev_deps.venv/lib/python27.zip
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/build-support/pants_dev_deps.venv/lib/python2.7
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/build-support/pants_dev_deps.venv/lib/python2.7/plat-linux2
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/build-support/pants_dev_deps.venv/lib/python2.7/lib-tk
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/build-support/pants_dev_deps.venv/lib/python2.7/lib-old
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/build-support/pants_dev_deps.venv/lib/python2.7/lib-dynload
                     	/usr/lib64/python2.7
                     	/usr/lib/python2.7
                     	/usr/lib/python2.7/plat-linux2
                     	/usr/lib64/python2.7/lib-tk
                     	/usr/lib/python2.7/lib-tk
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/pyprep/requirements/CPython-2.7.15/5e0b9a73f7766ef6381bac7241ece3a52aefebf3-DefaultFingerprintStrategy_f4bc9a770d1b
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/pyprep/sources/e9d7f65130ec5c9a2a5d05db01a0075427938983
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/pytest-3.6.4-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/setuptools-40.4.3-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/funcsigs-1.0.2-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/linecache2-1.0.0-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/pytest_timeout-1.2.1-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/attrs-18.2.0-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/pytest_cov-2.4.0-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/more_itertools-4.3.0-py2-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/unittest2-1.1.0-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/six-1.11.0-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/atomicwrites-1.2.1-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/py-1.6.0-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/coverage-4.5.1-cp27-cp27mu-manylinux1_x86_64.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/pluggy-0.7.1-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/argparse-1.4.0-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/traceback2-1.4.0-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/pyprep/requirements/CPython-2.7.15/5e0b9a73f7766ef6381bac7241ece3a52aefebf3-DefaultFingerprintStrategy_f4bc9a770d1b/.deps/six-1.11.0-py2.py3-none-any.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/pyprep/requirements/CPython-2.7.15/5e0b9a73f7766ef6381bac7241ece3a52aefebf3-DefaultFingerprintStrategy_f4bc9a770d1b/.deps/thrift-0.11.0-cp27-cp27mu-linux_x86_64.whl
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/bdd83c141bb9cfffde291c49aa9e41781cc24326/.bootstrap

Here the 1st entry, /home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/bdd83c141bb9cfffde291c49aa9e41781cc24326 is the executing code and it contains the plugin.
Coverage is on the sys.path in the /home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/coverage-4.5.1-cp27-cp27mu-manylinux1_x86_64.whl entry lower down.
I went down the rabbit hole and things were ok at coverage/cmdline.py line 755:
>>> In coverage.cmdline with sys.path:
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/bdd83c141bb9cfffde291c49aa9e41781cc24326
                     	/home/jsirois/dev/pantsbuild/jsirois-pants/build-support/pants_dev_deps.venv/lib/python27.zip
...

But not ok here:
>>> In coverage.cmdline about to call self.coverage.load() with sys.path:

                     	/home/jsirois/dev/pantsbuild/jsirois-pants/build-support/pants_dev_deps.venv/lib/python27.zip
...

FWIW - I'm hacking around presently by ensuring the 1st sys.path entry (the user code), is on the sys.path twice.

Also probably a wall of text, but the fix commit downstream is here: https://github.com/pantsbuild/pants/pull/6594/commits/2d957560ff9f977561e4b5f6ca15d1a2a0112296

I do have a workaround - so, bottom line, no worries. That said, commenting out the sys.path[0] mutation leads to the same sys.path via python test.py or coverage run test.py since python places CWD at the head of sys.path already. IOW sys.path[0] = '' is a no-op in this vanilla case. Presumably it fixes up a sys.path in a less standard case. The comment references django which I have little familiarity with. It would seem though that there is an assumption that whatever is in the 0th slot is unimportant. I'm not sure how coverage can be confident what is in that slot. Perhaps the django case puts a safely-nukeable path element there?

@jsirois Hmm, commenting out that line might be fine: the coverage.py tests all pass.  Does it solve your problem?  There is another sys.path manipulation later when we exec the file.

I'm not positive, but I don't think it will affects us. Here I assume?:
https://github.com/nedbat/coveragepy/blob/7a099a2204e6863824f11432c807442f3ad5306c/coverage/execfile.py#L174-L176
We use pytest-cov to actually collect coverage data under the pytest runner and they appear to use the coverage API (ie: coverage.run(), etc) - not execfile. We only have this issue trying to produce reports after the coverage run using the coverage command reporting tools.

@jsirois Could you try removing the line?

Will do. If I don't get a PR out tonight I will tomorrow. Thanks Ned!

A PR isn't necessary (the change is simple enough), but I'd like to know how your scenario behaves with the change.

Ah, gotcha. I can guaranty it solves our problem. The digging I hinted at above with the injected print statements proved that. That said, I can back up that guaranty with a full test by tomorrow. I'll report back.

@jsirois #678 is also about this.

Aha, yes I duped #678.
I just had a chance to burn a local coverage dist with:
$ git diff coverage/cmdline.py
diff --git a/coverage/cmdline.py b/coverage/cmdline.py
index edbc1d25..e6ea6e23 100644
--- a/coverage/cmdline.py
+++ b/coverage/cmdline.py
@@ -465,10 +465,6 @@ class CoverageScript(object):
         if self.do_help(options, args, parser):
             return OK

-        # We need to be able to import from the current directory, because
-        # plugins may try to, for example, to read Django settings.
-        sys.path[0] = ''
-
         # Listify the list options.
         source = unshell_list(options.source)
         omit = unshell_list(options.omit)

That did in fact solve the problem pants was having and coverage now works without hacks.

Thanks @nedbat!

@jsirois Looking into this more, I have a few questions.  The sys.path docs (https://docs.python.org/3/library/sys.html#sys.path) say, "As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter."  What is sys.path[0] for you, and why is it different than this?
Also, I am just curious about your coverage plugin: what does it do?

@nedbat we run pytest and coverage from within a pex - roughly a zipapp, and so the sys.path[0] entry points to our pex.
In other words - in the debug output above snipped and annotated here:
[0] /home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/bdd83c141bb9cfffde291c49aa9e41781cc24326
/usr/lib64/python2.7
...
/usr/lib/python2.7/lib-tk
[12] /home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4
[13] /home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/pyprep/requirements/CPython-2.7.15/5e0b9a73f7766ef6381bac7241ece3a52aefebf3-DefaultFingerprintStrategy_f4bc9a770d1b
[14] /home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/pyprep/sources/e9d7f65130ec5c9a2a5d05db01a0075427938983
/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/pytest-3.6.4-py2.py3-none-any.whl
...
/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/py-1.6.0-py2.py3-none-any.whl
[27] /home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/coverage-4.5.1-cp27-cp27mu-manylinux1_x86_64.whl
/home/jsirois/dev/pantsbuild/jsirois-pants/.pants.d/test/pytest-prep/CPython-2.7.15/15d60b3366ef43c7a4ae6b5f5e2d7567a95fe5f4/.deps/pluggy-0.7.1-py2.py3-none-any.whl
...

The 0th entry is our pex, the a bunch of system python entries, then more bits of our pex (we combine 4) in 12, 13 & 14 - then 3rdparty dependencies of the pex - including coverage down in entry 27. The critical point which applies here afaict is coverage is not guaranteed to be in the 0th slot if some other tool is running coverage. As such coverage should not blow away any sys.path entry it is not sure it owns. In this case the 0th entry is us, the bootstrap code that runs coverage and contains the coverage plugins we wish to activate. More generally, inserting or appending a sys.path entry as is appropriate to the case at hand is almost always what you want to do afaict. If there might be multiple versions of the thing and you need to have your version seen, insert 0, if you know you're unique and just need to be visible, append.
The plugin just attempts to remap filenames. The scenario is as follows:

A user wants to run tests and get coverage data for those tests.
Pants (a build tool), gathers up the relevant test code and production code under test and forms a hermetic pex containing the user code as well as pytest and coverage.
Pants executes pytest and coverage against the user code in the context of the pex.

Because the coverage is gathered in step 3, it sees and reports on paths different from the original loose filesystem paths the user expects; and as such we have a coverage plugin that remaps pex pathnames back to user-space.

@jsirois Thanks for the explanation.
BTW, your description of your plugin sounds like what the [paths] configuration setting does: https://coverage.readthedocs.io/en/v4.5.x/config.html#paths .   Or is your situation more complex?

The situation is a bit more complex. See a comment here: https://github.com/pantsbuild/pants/issues/5426#issuecomment-363593177 and our discussion in #646.
The issue boils down to this code here:
https://github.com/nedbat/coveragepy/blob/e6b883903e6f0eb396d35d43ff82c3c5644af84c/coverage/control.py#L559-L565
Pants runs tests for projects that can be split over multiple source trees; ie:
project/
  widget/src/python/
    widget/__init__.py  # A namespace package
    widget/module1.py
  util/src/python
    widget/__init__.py  # A namespace package
    widget/util.py

As such, a test against the widget package in this example can produce coverage that is spanned multiple different local source roots. The PathAliases code does not allow for this as things stand and 1 local source root "wins" leading to trampling of data from the losers. So my options were to contribute a patch to support the concept of multiple local source roots or else write a plugin. I'd be happy to ditch the plugin in favor of using combine if you think a patch to add support for this concept is desirable.


Attacking enemies causes textures to be tinted

Attacking slimes causes the player (and sometimes other GUI textures such as the hearts) to be random colors. Seems like the only colors as of right now are: Green, Yellow, Red.

Found the issue source:
Creature.renderHealthBar()
		if (hp < 0.3f) {
			Graphics.batch.setColor(1, 0, 0, 1);
		} else if (hp < 0.6f) {
			Graphics.batch.setColor(1, 1, 0, 1);
		} else {
			Graphics.batch.setColor(0, 1, 0, 1);
		}

This seems to be applying to the player and not the slimes from the pic.

Closed in pull https://github.com/egordorichev/LastTry/pull/71


Results: When disaggregated values do not add up to total, I am prompted to give a reason for the discrepancy

This ticket was peeled off of the work described in #2293.
Issue
From the MEL team -- related to their request to allow results to be submitted when the disaggregated values don't add up to the total actual value: We spoke about how it may be useful to track how this is working. Would it be possible to track this in something like the change log? If the change log isn't appropriate, then maybe another method? If people have to put in a reason for why the numbers don't match it could give us a better idea of how the system is being used. If people don't provide substantive reasons then it at least gives us a bunch of cases to dig into with teams so we can spot check and better understand the trend in use.
Acceptance criteria
Affects:

Results submitted for number indicators only, not % indicators.
Submission of new results AND edited results.

Scenario 1: The user creates OR edits a saved result and submits the form with one or more invalid or incomplete required fields

[x] 1. Regardless of whether or not the disaggregated values match the total, the user sees the exact same warnings and instructions they see today. We do not show any special prompts related to disaggregation discrepancies until the other issues are resolved.

Scenario 2: The user creates a new result with all required fields complete and valid AND the disaggregated values do not add up to the total. The fact that this is a newly created result is a notable distinction; today we do not have any rationale prompts associated with a new result.

[x] 1. We display a yellow rationale prompt. It says:

Warning
The sum of disaggregated values does not match the actual value.
For future reference, please share your reason for these variations.

Please ignore the styling differences between my mockup and the current implementation of PNotify warnings. We have a Poblano ticket to update styles globally.

[x] 1.1. If I click "Cancel", changes are not saved and the PNotify prompt goes away.

[x] 1.2. If I attempt to submit without entering anything into the text field, I see the standard red message: A reason is required.

[x] 2. If I type something in the text field and click "OK", the following things occur:

[x] 2.1. We maintain the same behavior described in #2293 -- the submission is accepted, with a warning message:

[x] 2.2. Whatever the user typed into the "reason for change" text box appears in the "Reason for change" column of the change log.

Scenario 3: The user edits a saved result in any combination of ways, WITHOUT creating any disaggregation discrepancies.

[x] 1. We show the following reason for change prompt: Modifying results will affect program metrics for this indicator and should only be done to correct a data entry error. For future reference, please provide a reason for modifying this result.

NOTE: We are currently displaying the wrong message as per #1320. Please replace with the above message.
Scenario 4: The user edits a saved result in any combination of ways, INCLUDING a disaggregation discrepancy.

[x] 1. We display a yellow "reason for change" prompt. It says:
Warning
The sum of disaggregated values does not match the actual value.
Modifying results will affect program metrics for this indicator and should only be done to correct a data entry error. For future reference, please provide a reason for modifying this result.

[x] 1.1. If I click "Cancel", changes are not saved and the PNotify prompt goes away.

[x] 1.2. If I attempt to submit without entering anything into the text field, I see the standard red message: A reason is required.

[x] 2. If I type something in the text field and click "OK", the following things occur:

[x] 2.1. We maintain the same behavior described in #2293 -- the submission is accepted, with a warning message:

[x] 2.2. Whatever the user typed into the "reason for change" text box appears in the "Reason for change" column of the change log.

Housekeeping

[ ]  Unit test/s written wherever applicable
[ ]  i18n tags included wherever applicable
[ ]  Functionality is accessible to people using screen readers

@jennyhavah The basics of presenting the modal and creating the log entry are pretty quick and easy.  The place it could get bogged down is in the diffing in and display.  The diffing in particular is under-tested and therefore fragile.
I'm assuming that we're not making any changes to the structure of the change log.  It's meant to show previous and new entries, which is not really what we're doing.  If we decide to change the structure of the change log to accommodate this new type of rationale prompting, this will become a bigger project.

@sanjuroj By "structure" do you mean layout? No, the only change I'm suggesting is really to the logic -- a new situation where a prompt would appear. The rationale itself would be stored in the same place that we store all other change log rationales -- in the Comments column.

@sanjuroj Ticket updated, including a new scenario where the user edits a saved result in multiple ways, including disaggregation changes that cause a discrepancy.
Please let me know if you see any other gaps.

@jennyhavah Looks good.  I think I know the answer but just to be sure, in Scenario 4, I assume the warning message should be "Your changes will be recorded in a change log. For future reference...", rather than "Your changes will be recorded in a change log. future reference..."

I assume the warning message should be "Your changes will be recorded in a change log. For future reference...", rather than "Your changes will be recorded in a change log. future reference..."

@sanjuroj Thanks for catching that -- mistake fixed.

@jennyhavah As I was looking through the code, I realized that there is a different warning message if the user changes the date:

Modifying results will affect program metrics for this indicator and should only be done to correct a data entry error. For future reference, please provide a reason for modifying this result.

This seems a little strange since it doesn't actually reference that the date has changed, though it is a bit stronger than the message provided if the user changes the value or the evidence link:

Your changes will be recorded in a change log. For future reference, please share your reason for these changes.

Does this change the scenarios listed above?

@sanjuroj I thought it was weird that this message was tied to a change in the result date, so did a bit of digging and realized that we should be using this messaage...

Modifying results will affect program metrics for this indicator and should only be done to correct a data entry error. For future reference, please provide a reason for modifying this result.

... for any type of result modification, not only for the result date. This is not a Goat Tree regression -- there was just a misunderstanding or oversight related to the "modify result record" section of ticket #1320.
I updated scenarios 3 and 4 to include the correct text anytime a result is changed, with or without a disaggregation issue.

@sanjuroj The PNotify behavior all works as expected. However, for Scenario 2, my reason is not showing up in the change log. It says "N/A".

@jennyhavah I've fixed the problem of the rationale not appearing but there are a couple of edge cases I hadn't considered.

Start with an existing result with a disagg/value mismatch.  The user changes the value to match the disagg sum.  Only the changed value will appear in the change log, not the disaggs since they haven't changed.  But it's significant that the value has been changed to match the disaggs (and this might be reflected in the rationale).  Do we want to show the disaggs values in the change log, even though they haven't changed?

Again start with an existing result with a disagg/value mismatch. The user changes the evidence link.  Once again, they may reference the disagg mismatch in the rationale (as they are prompted to), but neither the actual value nor the disaggs will appear in the change log because neither has changed.

These seem fairly unlikely but just wanted to make sure we handled them if needed.

Start with an existing result with a disagg/value mismatch. The user changes the value to match the disagg sum. Only the changed value will appear in the change log, not the disaggs since they haven't changed. But it's significant that the value has been changed to match the disaggs (and this might be reflected in the rationale). Do we want to show the disaggs values in the change log, even though they haven't changed?

@sanjuro In general, I think it's best to have consistent rules for what fields we show in the change log -- either show every field regardless of whether or not it's been changed... or only show fields that have changed. I remember you and I going back and forth over whether we ought to always show the total value even if it has not changed... and in the end we thought it might be more confusing than helpful to make that one exception. I lean the same way for disaggregation values...
Let's leave it as is for now. Just to be safe, I'll double check with Marco to make sure we're on the same page about only showing total and disagg values that have changed, generally speaking.

Again start with an existing result with a disagg/value mismatch. The user changes the evidence link. Once again, they may reference the disagg mismatch in the rationale (as they are prompted to), but neither the actual value nor the disaggs will appear in the change log because neither has changed.

In this case, I'm not sure that displaying the fields adds any clarity... They're still providing an explanation for a discrepancy that occurred in the past... not something related to what they're editing in the present. If we include all the fields, then the auditor is going to be looking for what's different and may wonder if something got mis-logged. So again, I think it boils down to needing consistent rules -- show everything or only show what's changed, always.

My rationale is now saved, for Scenario 2. Passing this ticket. @sanjuroj I will open a new ticket if Marco wants to change any behavior around what values we display in the change log, as discussed above.


TTL support for firestore-send-email

Extension name
This feature request is for extension: firestore-send-email
What feature would you like to see?
Now with TTL Support for Firestore would be great if firestore-sen-email implement a configuration possibility to determine expiration time of sended emails.
In configuration whe set the time, like 1 day, 1 week..... and would be created a expireAt field, and then we would create a TTL Policy.
How would you use it?
The log of sended email from firestore-send-email normaly is not necessary and we have to create routines to clear the unecessary data.

I was just about to write the same feature request. This would be very useful!


issue 54

start issue-54
fixes issue-54
fixes issue-54

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (3815ba3) to head (637b06a).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff            @@
##              main       #56   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           16        15    -1
  Lines          928       944   +16
=========================================
+ Hits           928       944   +16

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[visualization] Change color of arms depending on angle

Summary
We want the users (patient and care giver) to easily see the level of movement the patient has.
This will start very simply with just the arms, and also monitoring the angle between arm and core-body.
Basic example
The main idea is to have the arm change color depending on the said angle. If this is too complicated, maybe we overlay something else that shows that data and with colors
Details
We've been don't to not use red/orange/green color-set.

I believe this might be our solution: https://wiki.ros.org/rviz/DisplayTypes/Marker

No, unfortunately this is not applicable on xacro/urdf. You can create and modify additional markers in different colors and shapes. You can spawn markers on the link (upper arm link etc.) and move them attached to the respective link like a mask. It is how I implemented in the HRC project. It works but not perfect and a bit cumbersome to set up.
We can think of this as plan-B or we might reconsider the visual design - do we really need 'the arm' to change color?
The ideal would be if we find a way to change xacro material color, let say a launch argument which is linked to a rosparam, and modify it on the go.

Yes I agree, the markers were supposed to be a plan B, but it seems like changing the color of a xacro material on the fly is not possible. That's why I'm looking into plan B


Older version is broken

Clicking the link in the README to the older version gives this 404 page.

Removed link


Errors in REST API Reference

I'm busy writing a client library against the REST API. There is a few places where the reference is wrong e.g. best_header_timestamp in /v1/getinfo is supposed to be an int64 but actually a string, some fields seem to be optional.
If the reference is generated from https://github.com/lightningnetwork/lnd/blob/master/lnrpc/rpc.swagger.json and this is a hand-written file then I'm happy to create a pull request to correct the issues that I find. Let me know.

What reference are you referring to? That's just a set of swagger json docs. If you're actually getting json, many times it's the case that integers are encoded as strings, as JS doesn't have proper integers.

Are you aware of https://api.lightning.community/rest/?

That's the reference I am using and which is wrong. Was thinking it was generated from the swagger docs.
Assuming we want reference to match implementation and not the other way around. Want help fixing it?

There's no need to fix it: it's JSON, integers are usually encoded as strings.

Also the swagger file is automatically generated.

"1" specified as type <Int64> instead of <string> is problematic for something that is supposed to be machine readable (even if in JSON). It should not take much effort to correct and I'm happy to do so, if I'm pointed in the right direction. I will close the issue, but feel free to leave a comment.

I personally scratched my head at this as well, it seems that sometimes you get back numbers where you'd expect to remain small enough (e.g. v1/getinfo's numPeers) but you get back strings for numbers you'd expect to get big (e.g. v1/balance/blockchain's balance numbers).
It's great that this is done automatically as you'd expect, but it'd be even better if the documentation reflected that. Given that somewhere this conversion is happening explicitly, it'd be great to have this issue re-opened & the documentation corrected.

I fully agree with you, but will leave it to one of the maintainers to reopen if acknowledged that it is an issue they care about.


Invokable controller option

You can now create an invokable Controller by passing --invokable option.

Thank you for your review, that should be good now!


Have you tried adding support for cache?

That is, the ability to use the cache and tool-cache from PowerShell...
I have been playing with some actions in PoshCode/Actions and ended up writing about half of it in node js because I want to cache the install of modules...

I haven't played with caching in GH Workflows yet.  Are you trying to write a version of the cache/tool-cache actions, but just for PWSH modules?

I haven't played with caching in GH Workflows yet.  Are you trying to write a version of the cache/tool-cache actions, but just for PWSH modules?


Reset Quests when starting new game

I have a problem resetting the quests. Due to my lack of Godot experience, I can't solve the problem even after several hours.
I have the following scenario: The player starts a new game and completes the first quest. He then switches back to the main menu and starts a new game again. If he now wants to start the first quest a second time, it is automatically completed.
I suspect the error (if it's not just me) in the quest_resource.gd -> line 33. The QuestResource is loaded as a fresh copy using duplicate(true). However, according to the documentation and tests, the duplicate function cannot handle resource arrays. So although the QuestResource itself is freshly loaded, its nodes are still the old, already completed ones.
Can you confirm an error here or is it just me?
Thank you very much for your work! I can't wait to use your asset in my game.

Edit: I‘m using Godot 4.3

Hmm, it seemed to work before 4.3. I'm away from my computer atm, but will investigate it tomorrow

Ok, this should be fixed now. You were right - arrays in resources cannot be duplicated, so I had to manually duplicate each node and edge, but this seems to do the trick.
Let me know if you have any other problems or ideas for improvement

Awesome! Now everything works as expected. Saving, loading, resetting - I was able to try everything out successfully! 😊
Thank you so much for looking at this so quickly!


Update the component for beta release

Please verify the version of auro-backtotop you have installed
No response
Please describe the bug
Component needs to be updated to use beta versions of dependency components and do tag versioning.
Reproducing the error
None
Expected behavior
Component needs to be updated to use beta versions of dependency components and do tag versioning.
What browsers are you seeing the problem on?
No response
Additional context
No response
Exit criteria
No response

This PR has been merged.


Updated Artwork

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/example-scenario/apps/media/architecture-diagram-hpc-saas.png
:white_check_mark:Succeeded
View

docs/example-scenario/apps/media/architecture-diagram-sap-production.png
:white_check_mark:Succeeded
View

docs/example-scenario/apps/media/decentralized-trust/DecentralizedTrust.png
:white_check_mark:Succeeded
View

docs/example-scenario/apps/media/ecommerce-search/architecture-ecommerce-search.png
:white_check_mark:Succeeded
View

docs/example-scenario/data/media/architecture-diagram-big-data-with-iot.png
:white_check_mark:Succeeded
View

docs/example-scenario/data/media/architecture-diagram-realtime-analytics-vehicle-data1.png
:white_check_mark:Succeeded
View

docs/example-scenario/infrastructure/ServiceFabricMicroservices.md
:white_check_mark:Succeeded
View

docs/example-scenario/infrastructure/media/Hybrid-HPC-Ref-Arch.png
:white_check_mark:Succeeded
View

docs/example-scenario/infrastructure/media/service-fabric-microservices/clip_image002.png
:white_check_mark:Succeeded

docs/example-scenario/infrastructure/media/service-fabric-microservices/clip_image004.png
:white_check_mark:Succeeded

docs/example-scenario/infrastructure/media/service-fabric-microservices/clip_image006.png
:white_check_mark:Succeeded

docs/example-scenario/infrastructure/media/service-fabric-microservices/service-fabric-complete.png
:white_check_mark:Succeeded
View

docs/example-scenario/infrastructure/media/service-fabric-microservices/service-fabric-midway.png
:white_check_mark:Succeeded
View

docs/example-scenario/infrastructure/media/service-fabric-microservices/service-fabric-monolith.png
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Bug fix

HeatPumpEconIHX_closed and HeatPumpIHXEcon_closed labeling error. There was no label for the connection A11, from econ to merge.
Also corrected transcritical pressure value in params file of HeatPumpSimpleTrans. It has to be greater than the T_crit of wf.

Good eye. Thank you for the improvements.


Multiple Tenant Support

Is the scenario supported where one ADFS farm serves multiple tenants?  Specifically, we are one company with presence in both Azure Commercial and Azure Government, and need to support MFA from both Azure tenants on the same ADFS farm.  Using two different ADFS farms is not an easy option since we have other shared third-party relying parties that do not support multiple federation endpoints.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f333e8cb-c046-ee06-d33d-d7dc8e8af695
Version Independent ID: 10838695-f60e-42a7-b0e8-eed7820fd333
Content: Configure AD FS 2016 and Azure MFA
Content Source: WindowsServerDocs/identity/ad-fs/operations/configure-ad-fs-and-azure-mfa.md
Product: windows-server
Technology: ad-fs
GitHub Login: @billmath
Microsoft Alias: billmath

#assign:xelu86

Hey @tristan-rice,
Thanks again for contacting Microsoft. We value your feedback.
As we understand it, you're seeking deeper knowledge of the product compared to an issue with the documentation. GitHub isn't quite the right location if searching for technical support, deeper specifics of the product or feature requests as it is completely outside our scope. If there are any issues within the content itself, please let us know.
To get assistance or support for your product, we recommend that you engage with the product community or open a ticket with Microsoft Support.
Additional support forums listed below:

TechNet: https://social.technet.microsoft.com/Forums/en-US/home
MSDN: https://social.msdn.microsoft.com/Forums/en-US/home
Microsoft Community: https://answers.microsoft.com/

Hopefully one of the community experts may be able to better assist. We will now be closing this thread. Thank you and have a most pleasant day.
#please-close


modularize build windows

NOT FOR MERGE
This is an experimental branch to get Windows builds of the runtime working.
The plan is to productionize and then git cherry-pick from here

Closing in favor of #573


Remove dup SPRING_DATASOURCE_PLATFORM env

Fix #328

@paderlol please help to merge.

一同考虑 https://github.com/nacos-group/nacos-docker/issues/327 ？

一同考虑 #327 ？

Yes, commented in issue, waiting for reviewer opinion.

@paderlol please help to review, thanks!


Does this script support Caddy 2 beta?

Hi, I've been using Caddy-Web-Server-Installer for a while and very happy with it. But I wonder does it support CaddyServer 2?
Caddy 2 is still in beta, but it's fine for me to use it in my personal project, I tried to set it up manually, it's pretty good.
Also, Caddy 2 is listed on Digital Ocean for one-click install, I think it's pretty stable and it's time to implement it.
Thank you~

Not yet but once Caddy 2 is released under stable branch it should just work fine.

Thank you. I'll wait till Caddy 2 release.


Using Function in generics without new breaks things

Consider this code:
class StoreConnector<T, S> {
}

void main() {
  final connector = new StoreConnector<int, int Function(String)>();
}

This compiles and runs fine.
Now, remove the optional new keyword:
class StoreConnector<T, S> {
}

void main() {
  final connector = StoreConnector<int, int Function(String)>();
}

This doesn't show any static errors in the IDE (at least not in my case). But it does result in a compiler error (when trying to run a Flutter app):
compiler message: lib/main.dart:4:45: Error: A 'final' variable must be initialized.
compiler message:   final connector = StoreConnector<int, int Function(String)>();
compiler message:                                             ^
compiler message: lib/main.dart:4:41: Error: Can't declare 'int' because it was already used in this scope.
compiler message:   final connector = StoreConnector<int, int Function(String)>();
compiler message:                                         ^^^
compiler message: lib/main.dart:4:36: Context: Previous use of 'int'.
compiler message:   final connector = StoreConnector<int, int Function(String)>();
compiler message:                                    ^^^
compiler message: lib/main.dart:4:45: Error: Expected ';' before this.
compiler message:   final connector = StoreConnector<int, int Function(String)>();
compiler message:                                             ^^^^^^^^
compiler message: lib/main.dart:4:45: Error: The class 'Function' is abstract and can't be instantiated.
compiler message:   final connector = StoreConnector<int, int Function(String)>();
compiler message:                                             ^^^^^^^^
compiler message: lib/main.dart:4:63: Error: Expected an identifier, but got ')'.
compiler message:   final connector = StoreConnector<int, int Function(String)>();
compiler message:                                                               ^
compiler message: lib/main.dart:4:63: Error: Getter not found: ''.
compiler message:   final connector = StoreConnector<int, int Function(String)>();
compiler message:                                                               ^
compiler message: lib/main.dart:4:35: Error: The method '<' isn't defined for the class 'dart.core::Type'.
compiler message: Try correcting the name to the name of an existing method, or defining a method named '<'.
compiler message:   final connector = StoreConnector<int, int Function(String)>();
compiler message:                                   ^
Compiler failed on lib/main.dart

Dart SDK Version (dart --version)

Dart VM version: 2.0.0-dev.48.0.flutter-fe606f890b (Mon Apr 16 21:21:13 2018 +0000) on "macos_x64"
(Bundled with Flutter Beta, v0.3.1)

Whether you are using Windows, MacOSX, or Linux (if applicable)

macOS

@danrubel Is this a failure in the Fasta parser, or is it somewhere later?

https://dart-review.googlesource.com/c/sdk/+/52920 addresses the fasta parser portion of  this issue
but I'm hoping someone else can address the dart2js and kernel generation portions.

Curious, why was the P1 label removed here? This seems like an important bug to get resolved quickly...

It was fixed on April 27.  @danrubel is there any work left on this, or can we close it?
The code in the issue compiles, and runs without error in the VM and dart2js.

Apparently @peter-ahe-google and crew have fixed they other part of this issue, so yes, we can close. Please reopen if you find this is not the case.


Update sbt-assembly to 2.1.0

Updates com.eed3si9n:sbt-assembly from 1.2.0 to 2.1.0.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Files still referring to the old version number
The following files still refer to the old version number (1.2.0).
You might want to review and update them manually.
CHANGELOG.md
docs/sections/versions/baker-3.md

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.eed3si9n", artifactId = "sbt-assembly" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "@monthly" },
  dependency = { groupId = "com.eed3si9n", artifactId = "sbt-assembly" }
}]

labels: sbt-plugin-update, early-semver-major, semver-spec-major, old-version-remains, commit-count:1

Superseded by #1432.


http return 302 redirect when i click button to upload file from ADMIN

Python 2.7. Mezzazine version 4.01.
os. windows7
#model
image = FileField(verbose_name=_("Upload Image"),upload_to=upload_to("demoapp.Article.image", ‘uploads/%Y%m/%d’),format="Image", max_length=255, null=True, blank=True)
#html
javascript:FileBrowser.show('id_image', '/admin/media-library/browse/?pop=1&dir=uploads\201512\07&type=Image');
windows CMD:
"GET /admin/media-library/browse/?pop=1&dir=uploads%C2%81512%07&type=Image HTTP/1.1" 302 0
"GET /admin/media-library/browse/?type=Image&pop=1 HTTP/1.1" 200 27872

Dynamic upload directories aren't supported with the filebrowser field - you might like to use the regular Django file field which does.

Thank you very much.


cmd/cgo: cgo changes broke golang.org/x/text/collate/tools/colcmp on Darwin

The golang.org/x/text/collate/tools/colcmp binary no longer compiles at Go tip. (It works at Go 1.9 and Go 1.8)
https://build.golang.org/log/6a0073f83d8b33849ca6b8f57120b1161e849c32
# golang.org/x/text/collate/tools/colcmp
/var/folders/dx/k53rs1s93538b4x20g46cj_w0000gn/T/workdir/gopath/src/golang.org/x/text/collate/tools/colcmp/darwin.go:41: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFStringCreateWithBytes
/var/folders/dx/k53rs1s93538b4x20g46cj_w0000gn/T/workdir/gopath/src/golang.org/x/text/collate/tools/colcmp/darwin.go:48: cannot use nil as type _Ctype_CFAllocatorRef in argument to func literal
/var/folders/dx/k53rs1s93538b4x20g46cj_w0000gn/T/workdir/gopath/src/golang.org/x/text/collate/tools/colcmp/darwin.go:76: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFStringCreateWithCharactersNoCopy
/var/folders/dx/k53rs1s93538b4x20g46cj_w0000gn/T/workdir/gopath/src/golang.org/x/text/collate/tools/colcmp/darwin.go:79: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFStringCreateWithCharactersNoCopy
/var/folders/dx/k53rs1s93538b4x20g46cj_w0000gn/T/workdir/gopath/src/golang.org/x/text/collate/tools/colcmp/darwin.go:82: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFStringCreateWithCharactersNoCopy
/var/folders/dx/k53rs1s93538b4x20g46cj_w0000gn/T/workdir/gopath/src/golang.org/x/text/collate/tools/colcmp/darwin.go:85: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFStringCreateWithCharactersNoCopy
/var/folders/dx/k53rs1s93538b4x20g46cj_w0000gn/T/workdir/gopath/src/golang.org/x/text/collate/tools/colcmp/darwin.go:93: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFStringCreateWithBytesNoCopy
/var/folders/dx/k53rs1s93538b4x20g46cj_w0000gn/T/workdir/gopath/src/golang.org/x/text/collate/tools/colcmp/darwin.go:98: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFStringCreateWithBytesNoCopy
/var/folders/dx/k53rs1s93538b4x20g46cj_w0000gn/T/workdir/gopath/src/golang.org/x/text/collate/tools/colcmp/darwin.go:101: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFStringCreateWithBytesNoCopy
/var/folders/dx/k53rs1s93538b4x20g46cj_w0000gn/T/workdir/gopath/src/golang.org/x/text/collate/tools/colcmp/darwin.go:106: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFStringCreateWithBytesNoCopy
/var/folders/dx/k53rs1s93538b4x20g46cj_w0000gn/T/workdir/gopath/src/golang.org/x/text/collate/tools/colcmp/darwin.go:106: too many errors

This is due to https://golang.org/cl/66332.  It needs to be fixed in x/text.

Ah, great. I had a fix mostly done but wasn't sure why it was needed.


False positive for "blank line" missing after a function declaration

This wasn't happening before v1.6 but now it is. The code below will give me a FunctionDeclaration must be succeeded by 1 blank line
/// @notice this function returns something important
/// @return description
function someFunction() constant returns(uint256) {
    return 1;
}

  /// @notice gets something else important
  function anotherFunction() constant returns(uint256)
  {
    return 2;
  }

Basically seems to be due to the comments section which somehow confuse it. If comments are removed, then the rule passes.

Solved in 0.2.1, thanks to @cgewecke


Add test about enip stats with _udp prefix

Ticket
Redmine ticket: https://redmine.openinfosecfoundation.org/issues/6304
Suricata PR coming

Replaced by https://github.com/OISF/suricata-verify/pull/1546


Add golang CI lint

Description

Add golang CI lint

Added as part of https://github.com/iptables-viz/iptables-viz/pull/11


"disallowMultipleVarDecl": {"allExcept": ["require"]} is ignored by grunt-jscs

the option "disallowMultipleVarDecl": {"allExcept": ["require"]}, is part of jscs 2.0 and above, but is ignored when using jcsc via grunt.

Very much doubt that, but if you could provide a test repo to review, i can re-open, sorry it took so long to review this btw


Card not available

Before submitting a bug
✅ I updated to the latest version available
✅ I cleared the cache of my browser
Describe the bug
I tried both, installing it via HACS and manually, neither of those options worked.
I tried both multiple times, I coudn't get the card to show or the yaml-config-card to work, bc. it says the card isn't available.
I restarted to whole HA, cleared the browser data. Every other card works fine. Is there maybe a problem witht the latest HA version?
Screenshots
If applicable, add screenshots to help explain your problem.
For example, JavaScript errors are shown in the web inspector (if applicable).
Versions:

Release: 2.3.1
Browser: Chrome
HomeAssistant version: 2023.6.2

Same here, vacuum card is working just fine.

Please, try the latest version. Should be fixed in there.
Pleaes don't forget to hard-reload your browser with cleaning the cache.

Please, try the latest version. Should be fixed in there.
Pleaes don't forget to hard-reload your browser with cleaning the cache.

Hey, not with me, problem still persists.


Add user search

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#147  👈
#146
#145
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @illarionov and the rest of your teammates on  Graphite


Added more tests

Added more tests as referenced here. #475
Screenshot of all the tests clearing:

I don't think this understands that issue. The issue may be stale.


PostgresNIO/ListenStateMachine.swift:182: Fatal error: Invalid state: initialized

Description
ListenStateMachine reaches an invalid state during concurrent operation of addListener on a pool that is shutting down.
I'm not sure if this behavior is expected given the obvious concurrent execution described in the reproducer below.
At any rate, a fatalError here seems out of place unless there's something I'm misunderstanding about the ListenStateMachine.
To Reproduce
Simply run the following:
import NIO
import AsyncKit
import PostgresNIO
import PostgresKit

let group = MultiThreadedEventLoopGroup(numberOfThreads: 1)

let configuration = SQLPostgresConfiguration(
    hostname: "localhost",
    port: 5432,
    username: "postgres",
    password: "postgres",
    database: "test",
    tls: .disable
)
for _ in 0..<1000 {
    let pool = EventLoopGroupConnectionPool(
        source: PostgresConnectionSource(sqlConfiguration: configuration),
        on:  group
    )

    Task {
        for _ in 0..<100 {
            _ = pool.withConnection({ conn in
                let channel = "default"
                _ = conn.addListener(channel: channel) { _, notifications in
                    assert(notifications.channel == channel, "Notification channel mismatch")
                }

                return group.future()
            })
        }
    }

    let promise: EventLoopPromise<Void> = group.next().makePromise()

    pool.shutdownGracefully({ _ in
        promise.succeed(())

    })
    try promise.futureResult.wait()
}

with the following dependencies:
        .package(url: "https://github.com/apple/swift-nio.git",  from: "2.54.0"),
        .package(url: "https://github.com/vapor/async-kit.git", exact: "1.17.0"),
        .package(url: "https://github.com/vapor/postgres-nio.git", exact: "1.20.0"),
        .package(url: "https://github.com/vapor/postgres-kit.git", exact: "2.11.3"),

and the following for the database setup:
docker run \
    --detach \
    --name timescaledb \
    --publish 5432:5432 \
    --env POSTGRES_HOST_AUTH_METHOD=trust \
    --env TIMESCALEDB_TELEMETRY=off \
    timescale/timescaledb:latest-pg16;

psql -h localhost -U postgres -w -c 'create database test'

the result will be:
PostgresNIO/ListenStateMachine.swift:182: Fatal error: Invalid state: initialized

Expected behavior
My assumption about the behavior for the code would simply be that once the EventLoopGroupConnectionPool gracefully shuts down, all outstanding listeners created with addListener would be cleaned up or closed since the connections they are listening on have been shutdown.
Environment

Xcode 15.0 w/ default swift toolchain
swift package init from the terminal

Vapor Framework version: N/A
Vapor Toolbox version: N/A
OS version: Sonoma 14.1.1

@CrownedPhoenix Thank you for the bug report. In order to prioritize this, what do you want to achieve? Above code very much looks like it is geared towards reproducing the issue.
Can you use the new async listen API? Maybe in connection with the new PostgresClient? Those APIs should be way more stable as well.

Yeah - the reproducer I provided was mainly to simulate an error that I'm running into in practice in my test setup.
I think it is concurrency related based on the reproducer and all - but I haven't been able to decide if it's something I need to change in my code or if it's a PGNIO thing.
It just depends on whether the fatalError described above is expected behavior or if it is a state that shouldn't even be reachable under any scenario.
All said, I'll give the code changes you suggest here a try with the reproducer and then see if replicating them in my own test cases will resolve the issue for me.

Totally understand that - if I can find a solution that doesn't require you to change anything that will be great for me as well.
I'm going to see if I can refine the reproducer a little more.
In the meantime, do you have any idea why PostgresClient wouldn't be able to be found? I see the @available but I'm on Sonoma so I wouldn't expect that to be an issue.
I'm getting Cannot find ``PostgresClient`` in scope while other types from PostgresNIO are found just fine.
I get the same issue building on linux.
I'm just adding it straight to the code in the reproducer so no other dependencies or environment things have changed.

Oh I need to do the @_spi thing I bet. Sorry i missed that.

Yeah. We intend to drop the requirement to use @_spi very soon.

Alright, it looks like using PostgresClient resolves this issue.
However, it's not something I'm going to be able to use in the near-term because of sweeping the scope of the change that would be required to swap it out for EventLoopGroupConnectionPool. The main reason being that I can't get an SQLDatabase from PostgresClient synchronously like I can from EventLoopGroupConnectionPool with:
extension EventLoopGroupConnectionPool where Source == PostgresConnectionSource {
    public func database(logger: Logger) -> any PostgresDatabase {
        _EventLoopGroupConnectionPoolPostgresDatabase(pool: self, logger: logger)
    }
}

Because of this there's a bunch of places where I depend on synchronous behavior that I'd have to change to async - which is doable just large scale in my case unfortunately.
I am interested in pursuing the use of PostgresClient though. So in the meantime I'm willing to consider this issue closed and leave a ticket in my backlog for swapping over to use PostgresClient.

I've been diving into this issue a bit recently as well, and I think the root of the issue is that @CrownedPhoenix is using let channel = "default", which is a Postgres reserved keyword. I've ensured quoting in this MR: https://github.com/vapor/postgres-nio/pull/466

@NeedleInAJayStack makes an important point. At any rate, I would expect different error behavior if PGNIO wasn't able to fulfill the listen request.

@CrownedPhoenix This is why I removed that #466, fixes this issue. But I have now an easy way to replicate it :)

Oh good call - thanks for un-fixing. I totally agree that we ought to better protect against the fatalError case too.


How to predict with dropout on? (to simulate ensemble for uncertainty)

Hi all,
I would like to make predictions with dropout turned on - this will allow repeat predictions to build up a conditional probability distribution (http://mlg.eng.cam.ac.uk/yarin/blog_3d801aa532c1ce.html), providing information on prediction uncertainty. This would be extremely useful!
It seems that dropout is turned off for prediction (predict.MXFeedForwardModel) by default. How can I turn it on for prediction? I'm using MXNet R package on windows.
Many thanks.

Try to turn the training flags on when forwarding, i.e, use forward(is_train=True)

Thanks sxjscience, I'm afraid I'm fairly new to this and can't work out where to use that statement. I see in the R executor code we have:

#' Peform an forward on the executors
#' This function will MUTATE the state of exec
#'
#' @export
mx.exec.forward <- function(exec, is.train=TRUE) {
exec$forward(is.train, list())
}

But I don't know which is my 'exec' to perform the function on. My simplified code is as follows:

Define the input data and network
data <- mx.symbol.Variable("data")
fc1 <- mx.symbol.FullyConnected(data, name="fc1", num_hidden=176)
act1 <- mx.symbol.Activation(fc1, name="ReLU1", act_type="relu")
drop1 <- mx.symbol.Dropout(act1, p=0.5)
fcfinal <- mx.symbol.FullyConnected(drop1, name="fcfinal", num_hidden=1)

Use linear regression for the output layer
lro <- mx.symbol.LinearRegressionOutput(fcfinal)

Train model
model <- mx.model.FeedForward.create(params)

Make predictions
preds <- predict(model, testdata)[1,]

Where should I add 'forward(is_train=TRUE)'?
Many thanks.

I'm not familiar with the R API and I suggest you look into the implementation of predict, which will call the mx.exec.forward. You can do that by chaning the is.train flag to true.

Your advice was exactly right, thank you very much. For anyone else using R, the procedure was as follows:
#See code behind predict:
getAnywhere(predict.MXFeedForwardModel)
#Create modified predict function:
predict.MXFeedForwardModelDropout <- function (identical to predict.MXFeedForwardModel but with 'mx.exec.forward(pexec, is.train = TRUE)'
#Add the modified function to the mxnet namespace:
environment(predict.MXFeedForwardModelDropout) <- asNamespace("mxnet")
#Then specify modified function when predicting:
preds <- predict.MXFeedForwardModelDropout(model, testdata)[1,]
Predictions are now different realisations each time.


DB migration from versions

Hi,
im using version 1.0.21, and I want to upgrade to newer version.
I saw that on version 1.0.22 there was a schema change.
I wonder if there is any script or ability to convert SQLITE DB to newer schema.
thanks.

As described in the documentation, the schema migrations are executed automatically for you. Don't forget to backup your database.
http://kanboard.net/documentation/update


[SPARK-42168][3.2][SQL][PYTHON] Fix required child distribution of FlatMapCoGroupsInPandas (as in CoGroup)

What changes were proposed in this pull request?
Make FlatMapCoGroupsInPandas (used by PySpark) report its required child distribution as HashClusteredDistribution, rather than ClusteredDistribution. That is the same distribution as reported by CoGroup (used by Scala).
Why are the changes needed?
This allows the EnsureRequirements rule to correctly recognizes that FlatMapCoGroupsInPandas requiring HashClusteredDistribution(id, day) is not compatible with HashPartitioning(day, id), while ClusteredDistribution(id, day) is compatible with HashPartitioning(day, id).
The following example returns an incorrect result in Spark 3.0, 3.1, and 3.2.
import org.apache.spark.sql.expressions.Window
import org.apache.spark.sql.functions.{col, lit, sum}

val ids = 1000
val days = 1000
val parts = 10

val id_df = spark.range(ids)
val day_df = spark.range(days).withColumnRenamed("id", "day")
val id_day_df = id_df.join(day_df)
// these redundant aliases are needed to workaround bug SPARK-42132
val left_df = id_day_df.select($"id".as("id"), $"day".as("day"), lit("left").as("side")).repartition(parts).cache()
val right_df = id_day_df.select($"id".as("id"), $"day".as("day"), lit("right").as("side")).repartition(parts).cache()  //.withColumnRenamed("id", "id2")

// note the column order is different to the groupBy("id", "day") column order below
val window = Window.partitionBy("day", "id")

case class Key(id: BigInt, day: BigInt)
case class Value(id: BigInt, day: BigInt, side: String)
case class Sum(id: BigInt, day: BigInt, side: String, day_sum: BigInt)

val left_grouped_df = left_df.groupBy("id", "day").as[Key, Value]
val right_grouped_df = right_df.withColumn("day_sum", sum(col("day")).over(window)).groupBy("id", "day").as[Key, Sum]

val df = left_grouped_df.cogroup(right_grouped_df)((key: Key, left: Iterator[Value], right: Iterator[Sum]) => left)

df.explain()
df.show(5)

Output was
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=false
+- FlatMapCoGroupsInPandas [id#8L, day#9L], [id#29L, day#30L], cogroup(id#8L, day#9L, side#10, id#29L, day#30L, side#31, day_sum#54L), [id#64L, day#65L, lefts#66, rights#67]
   :- Sort [id#8L ASC NULLS FIRST, day#9L ASC NULLS FIRST], false, 0
   :  +- Exchange hashpartitioning(id#8L, day#9L, 200), ENSURE_REQUIREMENTS, [plan_id=117]
   :     +- ...
   +- Sort [id#29L ASC NULLS FIRST, day#30L ASC NULLS FIRST], false, 0
      +- Project [id#29L, day#30L, id#29L, day#30L, side#31, day_sum#54L]
         +- Window [sum(day#30L) windowspecdefinition(day#30L, id#29L, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS day_sum#54L], [day#30L, id#29L]
            +- Sort [day#30L ASC NULLS FIRST, id#29L ASC NULLS FIRST], false, 0
               +- Exchange hashpartitioning(day#30L, id#29L, 200), ENSURE_REQUIREMENTS, [plan_id=112]
                  +- ...

+---+---+-----+------+
| id|day|lefts|rights|
+---+---+-----+------+
|  0|  3|    0|     1|
|  0|  4|    0|     1|
|  0| 13|    1|     0|
|  0| 27|    0|     1|
|  0| 31|    0|     1|
+---+---+-----+------+
only showing top 5 rows

Output now is
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=false
+- FlatMapCoGroupsInPandas [id#8L, day#9L], [id#29L, day#30L], cogroup(id#8L, day#9L, side#10, id#29L, day#30L, side#31, day_sum#54L), [id#64L, day#65L, lefts#66, rights#67]
   :- Sort [id#8L ASC NULLS FIRST, day#9L ASC NULLS FIRST], false, 0
   :  +- Exchange hashpartitioning(id#8L, day#9L, 200), ENSURE_REQUIREMENTS, [plan_id=117]
   :     +- ...
   +- Sort [id#29L ASC NULLS FIRST, day#30L ASC NULLS FIRST], false, 0
      +- Exchange hashpartitioning(id#29L, day#30L, 200), ENSURE_REQUIREMENTS, [plan_id=118]
         +- Project [id#29L, day#30L, id#29L, day#30L, side#31, day_sum#54L]
            +- Window [sum(day#30L) windowspecdefinition(day#30L, id#29L, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS day_sum#54L], [day#30L, id#29L]
               +- Sort [day#30L ASC NULLS FIRST, id#29L ASC NULLS FIRST], false, 0
                  +- Exchange hashpartitioning(day#30L, id#29L, 200), ENSURE_REQUIREMENTS, [plan_id=112]
                     +- ...

+---+---+-----+------+
| id|day|lefts|rights|
+---+---+-----+------+
|  0| 13|    1|     1|
|  0| 63|    1|     1|
|  0| 89|    1|     1|
|  0| 95|    1|     1|
|  0| 96|    1|     1|
+---+---+-----+------+
only showing top 5 rows

Spark 3.3 reworked HashClusteredDistribution, and is not sensitive to using ClusteredDistribution: #32875
Does this PR introduce any user-facing change?
This fixes correctness.
How was this patch tested?
A unit test in EnsureRequirementsSuite.

cc @sunchao

@sunchao good catch!
I have renamed the PR and added the Python example as a unit test.

Tests are green: https://github.com/G-Research/spark/actions/runs/4003829481

Merged to branch-3.2.

cc @kazuyukitanimura since this landed to branch-3.2.


Navbar and footer // both responsive

might need font change later on and some re-styling
Pulled and merged locally before pushing, latest code

Wrong branch, will re-pull to proper one
This one was pointed to main


HyperLogLog merge is not commutative

Hi,
I ran into an issue where issuing the same query without cache on the same dataset would yield slightly different hyperLogLog result.

It's true, but I don't see what we could or should do about it. The algorithm we use is noncommutative by design. One specifically noncommutative feature is the 'floating max', which isn't part of standard HLL but is part of Druid's implementation. Although, either way, you will still get results within a predictable accuracy.
I am not sure if the alternative HLL algorithm in #5712 is commutative or not, but maybe you could try that out too.

Fair enough, it was a long shot.
We have quite some data as HLLCV1 and I doubt those can be converted to sketches, but thanks for the suggestion.


Release v0.0.67

RN 0.46 (current release) uses Flow 0.47, which exposes the error that is addressed by #285. Any chance we could get a new release?

Closing since I released it few days ago.


Collapsing with comments

Often when I am separating my code, I have my function, and some testing code below it. When I productize, I eliminate the testing code, but during the building phase it is helpful to have. I often separate my functions with long strings of '#########' so that I can scroll fast and not pay attention to the names.
The feature request comes from something I can do in R studio. There, if there is a long enough string of comments, it will allow me to collapse my code below it until it hits another long string of comments.
eg:
######################## Some Function for something #################
def some_function(variables):
*Insert long code here *
######################## Some Function for something Testing ############
*Long bunch of test code here *
####################### another one ################################
def another_one(variables):
*code *
######################### another one testing ########################
*more testing code *
in the above example, it would be nice to be able to collapse the long comment strings so I can block out testing code or function code while I'm working on another area.

This is what cells are for. We plan to add support for folding cells in the future.


Laravel 5 support?

hi ~ @jenssegers, I found that laravel-mongodb is not support L5 yet. could you please support some time?

+1, I'd love to use this package on Laravel 5.

+1!

We're working on it.

Thanks @jenssegers for the fast response and interest, good luck :+1: !

awesome, thanks a lot and much luck!

Thanks for your effort @jenssegers . Is it available now or by when it will be available to use.

Use the 2.1.0 tag. I'm currently working on a 3.x version on the develop branch, but it is no way near stable :)

Thanks for that quick reply, what do you mean by 2.1.0 tag and 3.x version here, Is that mean I can use with Laravel 5.0.0 to Laravel 5.0.2, or will be better to use L 4.2.x and if we have docs to use that with L 5 versions. Thanks a lot @jenssegers


sample path fix

What does this PR do?:
fixes path to the kcp sample application
Which issue(s)/story(ies) does this PR fixes:
None

We've removed the KCP code from HAS, so closing...


There is no color formatting for .rhtml files

VSCode Version: Code 1.12.2 (19222cdc84ce72202478ba1cec5cb557b71163de, 2017-05-10T13:16:25.899Z)
OS Version: Linux x64 4.4.0-78-generic
Extensions:

Extension
Author
Version

material-icon-theme
PKief
2.0.8

vscode-icons
robertohuertasm
7.8.1

Steps to Reproduce:

I had created a pr. which should fix it.


babel-preset: transforms dont work with swc commonjs output

cross posting for better visibility
All the context: https://github.com/microsoft/fluentui/issues/26251

@Hotell indeed, Linaria v3 does not have the best handling of SWC output (which changes from version to version 🙄 ) and will not as it's EOL. wyw-in-js has better coverage there, https://github.com/Anber/wyw-in-js/pull/19. Anyway, compiling CJS code is not something that we are looking forward.
With a switch to wyw-in-js it should work in the scenario linked, https://github.com/microsoft/griffel/issues/513#issuecomment-2042819789.


Adds function call to use hooks

Odd thing, in debian & ubuntu this is not needed but in babun (cygwin) without this call to autoload an error gets printed and the plugin does not work. Calling the autoload function before the add-zsh-hook solves the problem.

Thanks.