Question: How can I abort a query already in progress?

Is there any way to abort a useQuery that is in progress?
I have a map that queries data based on the bounding coordinates of the current window. Data fetch times can be a few seconds but if the user rapidly moves around I want to be able to cancel the last request made in favor of the new query.
I do use throttling and checking the loading state of the useQuery however this just prevents new queries being executed concurrently. This is fine but it leaves me with waiting for data from the original useQuery to finish before I can do anything else.

@Dastari as I just moved away from this package, figured I'd give you a heads up that this package is now deprecated. You will probably get an answer on this sooner over at https://github.com/apollographql/react-apollo


BUG ? Poll reports Windows socket  as 'Readable' after reregister with mio 0.6.1, not with mio 0.6.0

Hello,
In scaproust each tcp stream goes through a handshake phase that look like this:

register writable level, when polled as writable, write handshake and move next
reregister readable level, when polled as readable, read handshake and move next
reregister all edge, when readiness change, notify observers.

And with mio 0.6.0, everything was working fine, on both Linux and Windows.
Starting with mio 0.6.1, the step 3 causes Poll to report the stream as both readable and writable, while the peer has no sent anything. With version 0.6.0 the stream was reported as writable only.
This caused the following issue blabaere/scaproust#1.
If this helps, here is an extract of the test logs in both version:
MIO 0.6.0
DEBUG:scaproust::reactor::dispatcher: process_io Token(4) Ready {Writable}
DEBUG:scaproust::reactor::dispatcher: process_io Token(4) Ready {Readable}
DEBUG:scaproust::reactor::dispatcher: process_io Token(4) Ready {Writable}

MIO 0.6.1
DEBUG:scaproust::reactor::dispatcher: process_io Token(4) Ready {Writable}
DEBUG:scaproust::reactor::dispatcher: process_io Token(4) Ready {Readable}
DEBUG:scaproust::reactor::dispatcher: process_io Token(4) Ready {Readable | Writable}

I don't have a simple isolated unit test to reproduce the problem but I will gladly accept a git revision and let appveyor run the whole scaproust test suite with it. Don't hesitate to get in touch with me for more details.
Regards,
Benoît

Note that the precise semantics of what's readable/writable when change across platforms and don't tend to have strong guarantees. In that sense it's not immediately clear to me that this is a bug, but an isolated test case would be very helpful!

I understand that it is not possible to get the same ordering, same timing, same number or even the same sequence of events on all OSes. I also think it is not a problem to have 'false negative' that get reported correctly later, 'false positive' on the other hand have more impact. In my use case (load balancing writes and fair queuing reads), the 'receiver' will choose a stream and start a receive operation based on that 'false positive'. Since that stream has not received anything from its peer the operation will timeout and report an error. That's why I think readable 'false positive' is a bug.
Anyway I will write an isolated test case for that and let you know.

The test is written, available here. It is in src/lib.rs, the test is named spurious_readable_with_mio_061_but_not_with_060.
There is a passing travis build.
And a failing appveyor build.

Hm unfortunately I don't have access to a windows machine right now, could you set mio=trace in the log environment variable?

Done

Ok, looking at that specific trace, I believe there is no bug (nor false positive). On Windows there's an emulation layer of epoll with IOCP, and it's not perfect, but mirrors the "lossy" and/or "unpredictable" semantics one might expect with epoll itself.
What happened here was:

A read of length 0 was scheduled, the socket is now blocked and not receiving events
The read finished, so the socket transitioned to readable
Data was read, but successfully. The socket is still readable

At that point you're expecting that because all the data was read the socket is no longer readable. On Linux this is discovered through the kernel which has knowledge of internally buffered data, but on Windows we discover "not readable" through an error returned from an attempted read, where in this case that's not happening. If you were to attempt to do a read (even 0-length) it'd return "would block" and the socket would transition to not readable.
This is unfortunately just how the semantics are different across platforms. In general you can't have particularly strict assertions about whether sockets are readable or writable and expect that to work everywhere.

@alexcrichton So in my case, to get a more accurate readiness of the socket, I can call TcpStream::read and TcpStream::write, passing a 0 length slice and then only call Poll::reregister. Is that correct ?
I can try that in Scaproust and let you know about the result.
But that still does not explain why the provided test is passing with mio 0.6.0 and failing with mio 0.6.1, I kept the mio trace activated in case you're interested.

@alexcrichton The proposed solution did fix the Scaproust tests, so for the moment I'll keep it like that but that still feel a bit hacky.
Thanks a lot for your help !

A quick search on msdn brought this polling function, supported starting Windows 8.1. So is there any hope that one day, when nobody remembers Windows XP, Windows 8.1 being so old that it is the minimum version supported by Rust, we could get mio Linux and Windows have a closer behavior regarding readiness ?

WSAPoll is not scalable. If there are a million sockets, an array w/o all million has to be passed in / scanned.

@blabaere the behavior of TCP reads changed on Windows between 0.6.0 and 0.6.1, so that's why the update ended up breaking your tests.
And yeah as @carllerche mentioned I don't think we'll be able to use a polling function here.
It looks like, though, that this has been solved, so closing.


Dracula for jGrasp

Hey, I made Dracula for jGrasp. I'm new to GitHub so sorry if I did a few things wrong (like I don't think I was supposed to fork the template).
Dracula for jGrasp

Hey @peter-er!
First of all, sorry for taking so long to reply.
Now about your theme... I'm glad to see a theme for jGrasp! You did a great job for a new GitHub user ;)
I just cloned your repo and pushed all the commits to a new one over here. I also sent you an invite to join the Dracula org. You'll continue maintaining this project, the only difference is that we can make it more visible and include it on the site.
We have the screenshot already, that's great! The next steps would be adding instructions on how to install it. You can leave them here and I can put them on the site.
You can also update some of the information on the README like the team section.
Anyways, thanks for the contribution!

Hey @peter-er, any news about this? I'm afraid we'll have to archive the repository in case we don't hear from you since there are no instructions for people to use the theme.

Hi @zenorocha, so sorry. I updated it awhile ago but I only did my personal repository and not the actual one. I think it should be good now.

Hey @peter-er 👋 Would you mind pushing the changes to the org repo?

@hacknug Does it look OK now?

Thank you so much! I also wanted to ask you if there's any reason to keep both repos active? Won't this make it harder for you to maintain?
(Closing the issue since the org repo is now 👌)


⚠ Processors has degraded performance

In ff0665c, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 41 ms

Resolved: Processors performance has improved in f887c27.


Removed unnecessary string.Format of WriteLine

I did a little adjustment removing unnecessary string.format from the WriteLine method because WriteLine has an overload like string.format.
Thanks

Thanks!


Highlighter shiki adds a trailing new line to code blocks

Describe the bug
In contrast to the highlighter prisma, shiki adds a trailing new line to code blocks.

How this new line can be removed?
To Reproduce
Steps to reproduce the behavior:

Use default template
Add a code block

Desktop (please complete the following information):

OS: Linux
Browser: Firefox
Slidev version: 0.47.4

I can't reproduce it 😭

Here is a reproduction:

Open up https://github.com/openscript-ch/slidev-theme/tree/repro/new-line
pnpm install
pnpm dev
Open slide 8

Same problem here, I'm using highlighter: shiki, and code block with triple quotes produces one extra empty line (I've replaced backticks with quotes):
'''matlab
temperature = 38
'''

This code produces two lines of code in slide, where last one is just empty line, instead of single one line of code. In early versions of slidev there was no such problem. Here's part of the package.json from previous version I've been using:
  "dependencies": {
    "@slidev/cli": "^0.42.9",
    "@slidev/theme-default": "latest",
    "@slidev/theme-seriph": "latest",
    "slidev-theme-academic": "^1.1.3"
  },

Thank you @KermanX and @antfu for fixing this so quickly!


config.PoolSpans(true) will panic

i create a new tracer with 128bit and pool span options, found that the PoolSpans(true) options may panic my app
cfg.NewTracer(
		config.Gen128Bit(true),
		config.PoolSpans(true),
	)

i walk through the code base, found that
func (t *Tracer) newSpan() *Span {
	if !t.options.poolSpans {
		return &Span{}
	}
	sp := t.spanPool.Get().(*Span)
	sp.context = emptyContext
	sp.tracer = nil
	sp.tags = nil
	sp.logs = nil
	return sp
}

and
func (t *Tracer) reportSpan(sp *Span) {
	t.metrics.SpansFinished.Inc(1)
	if sp.context.IsSampled() {
		t.reporter.Report(sp)
	}
	if t.options.poolSpans {
		t.spanPool.Put(sp)
	}
}

but t.reporter.Report(sp) may be an async operation by put that span into a buffered channel, and then you guys put the span back into the pool, the span is still alive in the channel, this will cause some race condition or nil pointer reference therefore panic the program

Is this with the standard Jaeger reporter or a custom one?
Is 128bit relevant?

128bit not relevant, its the standard jaeger remote reporter

func (r *remoteReporter) Report(span *Span) {
	select {
	case r.queue <- reporterQueueItem{itemType: reporterQueueItemSpan, span: span}:
		atomic.AddInt64(&r.queueLength, 1)
	default:
		r.metrics.ReporterDropped.Inc(1)
	}
}

r.queue is a buffered channel

@philchia You might be interested to see this

@guanw the mr not exists in the latest release

@yurishkuro @philchia I guess we need to push a new release for jaeger-client-go?

Fixed in 2.17


⏪ Revert line clamp (#1245)

🤖 Generated by Copilot at a83ca7d
Summary
📚🎨🛠

Reduced the lineClamp property of app names in AppTile.tsx to avoid overflow and improve dashboard appearance.

lineClamp reduced
App name fits in the tile
Clear autumn dashboard

Walkthrough

Reduce the lineClamp property of the Text component to prevent app name overflow (link)

FYI: This is a hotfix for 0.13.1. We'll properly fix this issue in an upcoming release


UI - Glossary assets modal improvemetns

[ ] instead, chips use filter dropdown as we have in explore page

[ ] If I click on "Topic" or anything else (not All) to search for assets, and scroll to the bottom, it automatically moves me to All

https://user-images.githubusercontent.com/12962843/228821079-e0aed0f1-4deb-43a1-8e75-3f9860c5fed1.mov

[ ] filter counts are inaccurate

cc: @pmbrull @harshach Feel free to add more here if you found any


Click on input to open file

I wish a click on the input field itself would open the file dialog as this is standard behaviour for the regular version.

Be clearer

When you click on the input field (where your filename appears), it would generally have the same behaviour as when you click on the button itself. So it would open the file dialog.
Should not matter if you click on A or B.

I understand your point of view. But the plugin has a text input field to display the file name. Logo HTML when the element is disabled it does not run events. If I remove the disabled attribute field will be editable and occur several problems. Then the plugin will work only on the button.


reshape issue

1. environment from your docker
2.openvino model to convert:
https://drive.google.com/drive/folders/1I0XLgaswb-4NV_RB1MN9PyjbO3nxbhbn?usp=sharing
3. Issue Details

Fix
https://github.com/PINTO0309/openvino2tensorflow#6-7-replace-weights-or-constant-values-in-const-op-and-add-transpose-or-reshape-or-cast-just-beforeafter-the-operation-specified-by-layer_id


[Bug]: Index creation on table fails if table is created via SELECT INTO command

What happened?
When creating an index on a temp table, errors occur.
Could a maintainer please let me know if this is fixed in versions later than 4.0 ?
Here's how to quickly reproduce the issue
create table #PROB(ID int)

select ID into #REPRO from #PROB
create index IDX_REPRODUCTION on #REPRO(ID)

select #PROB.ID into #REPRO2 from #PROB
create index IDX_REPRODUCTION2 on #REPRO2(ID)

This error is raised at each create index Msg 33557097, Level 16, State 1, Line 38, column "id" does not exist
Here are some notes I took to find the problem:
drop table if exists #UPPERCASE_TEST_PARENT
drop table if exists #UPPERCASE_TEST_ONE
drop table if exists #UPPERCASE_TEST_TWO

create table #UPPERCASE_TEST_PARENT (ID int, NAME varchar(50))
select * from #UPPERCASE_TEST_PARENT --Columns are created with upper case here.
create index INDEX_THAT_WORKS on #UPPERCASE_TEST_PARENT(ID,NAME) --Create index with upper case columns works.
create index INDEX_THAT_ALSO_WORKS on #UPPERCASE_TEST_PARENT(id,name) --Lower case columns also works.

select #UPPERCASE_TEST_PARENT.ID,#UPPERCASE_TEST_PARENT.NAME into #UPPERCASE_TEST_ONE from #UPPERCASE_TEST_PARENT

--BABEL3_3_0: Columns are created with lower case because of table alias.
--BABEL4_0_0: Upper case column names
select * from #UPPERCASE_TEST_ONE

select ID,NAME into #UPPERCASE_TEST_TWO from #UPPERCASE_TEST_PARENT
select * from #UPPERCASE_TEST_TWO --Columns create with upper case when no table alias is present.

--This works on tag BABEL 3_3_0 but not on BABEL 4_0_0
create index INDEX_THAT_FAILS_ON_BABEL4 on #UPPERCASE_TEST_ONE(ID,NAME) -- fails only on babel 4_0_0:     column "id" does not exist, Msg 33557097, Level 16, State 1, Line 52

--This does not work.
create index INDEX_THAT_FAILS_ON_BOTH on   #UPPERCASE_TEST_TWO(ID,NAME) -- fails on both 4_0_0 and 3_3_0: column "id" does not exist, Msg 33557097, Level 16, State 1, Line 52

Shown here is the error when creating an index.

Shown here is the behavior difference between version 3.3 and 4

Version
BABEL_3_X_DEV (Default)
Extension
babelfishpg_tsql (Default)
Which flavor of Linux are you using when you see the bug?
Ubuntu (Default)
Relevant log output
column "id" does not exist, Msg 33557097, Level 16, State 1, Line 52

Code of Conduct

[X] I agree to follow this project's Code of Conduct.

Thanks @andrevb-croesus for reporting this issue. Please note that this issue is also reproducing on normal table under same scenario.

Will be fixed in branches 3_X_DEV & 4_X_DEV with these pull requests.
https://github.com/babelfish-for-postgresql/babelfish_extensions/pull/2893
https://github.com/babelfish-for-postgresql/babelfish_extensions/pull/2781


feat: collect tracecontext through server-timing for nav timings

Why

What

Links

Checklist

[ ] Tests added
[ ] Changelog updated
[ ] Documentation updated

@cedricziel please add a proper PR description.
Also check the todo items if applicable


Can't Connect to LocalHost After DotNet Setup

Problem encountered on https://dotnet.microsoft.com/learn/web/aspnet-hello-world-tutorial/run
Operating System: windows
I followed the instructions to set up .NET core and came across this in the tutorial:
Once the command completes, browse to http://localhost:5000
In my attempt to browse to the link provided, I kept receiving the following message (screenshot attached):
Hmmm… can't reach this page localhost refused to connect.
Try:
Checking the connection
Checking the proxy and the firewall
ERR_CONNECTION_REFUSED
I am using Microsoft Windows Version 1903 (OS Build 18362.145)

How did you run the app? From command line? What is the output? Does it fail to bind to the port?
Did you check your firewall settings?
Here's similar troubleshooting: #2411

Closing due to lack of response.

I actually got it working just now. I don't believe that I have done anything different but it works.


Request for clarification: Has this project been abandoned?

We've been searching for a GitOps-y way to deploy monitoring at the same time as we deploy our Kubernetes manifests. This project seems promising, but it is lacking features and does not appear to be actively worked on. Can someone from New Relic confirm or deny whether this is abandonware now?
Thanks,
Reid

Hi @reidhoch
At the moment we don't have the bandwidth to develop this further. That doesn't mean it's completely abandoned, as we are open to any community contributions and will review these.
Our main focus at the moment is on Terraform where we recently added support for latest Synthetics and Alerting features: https://github.com/newrelic/terraform-provider-newrelic So I would advise Terraform as an alternative.
Once we have caught up with the Terraform work we want to start investing in this again.

@kidk Thanks for the update, I appreciate your responding so quickly. We're already using the New Relic Terraform provider, so I'll focus my effort there.

+1 my team would also prefer using Kubernetes for GitOps management of NewRelic entities.
We do use terraform extensively but it's more heavyweight in our setup for managing these sorts of things.  We'd like to be able to define the monitoring entities (alerts, dashboards, etc.)  in the same Helm chart as the application itself and have them deployed and versioned together.


[SUPPORT] Partition prunning broken with metadata disable

hudi 0.11.1
spark 3.2.1
I have a huge performance drop when disabling metadata table at read time.
Here is a reproductible example with 2k partitions. (spotted in production with 40k partitions)
basePath = "/tmp/test_table"
df = spark.range(1,2000).selectExpr("id", "id as part", "id as combine")
hudi_options = {
    "hoodie.table.name": "test_table",
    "hoodie.datasource.write.recordkey.field": "id",
    "hoodie.datasource.write.partitionpath.field": "part",
    "hoodie.datasource.write.table.name": "test_table",
    "hoodie.datasource.write.operation": "bulk_insert",
    "hoodie.datasource.write.precombine.field": "combine",
    "hoodie.datasource.write.keygenerator.class": "org.apache.hudi.keygen.ComplexKeyGenerator",
    "hoodie.datasource.write.hive_style_partitioning": "true",
    "hoodie.datasource.hive_sync.enable": "false",
    "hoodie.datasource.write.keygenerator.class": "org.apache.hudi.keygen.ComplexKeyGenerator",
    "hoodie.metadata.enable": "true",
}
(df.write.format("hudi").options(**hudi_options).mode("overwrite").save(basePath))

Then try both (restating spark between two tests)
spark.read.format("hudi").option("hoodie.metadata.enable","true").load(basePath).filter("part=1").show()
spark.read.format("hudi").option("hoodie.metadata.enable","false").load(basePath).filter("part=1").show()

the former is fast, while the later, is as slow as reading the whole table (no partition prunning)

If you check the sql tab of spark-ui or the explain plan for both statements you see that partition pruning is happening and we only read files from 1 partition to satisfy your query. It is the planning stage that differs where spark catalog first needs to know all the partitions that do exist so that it can filter or prune them. That is exactly the problem Hudi metadata will solve and does not have to list all partitions

 and hence the increased performance for you.

spark.read.format("hudi").option("hoodie.metadata.enable","true").load(basePath).filter("part=1").explain()
== Physical Plan ==
*(1) ColumnarToRow
+- FileScan parquet [_hoodie_commit_time#134,_hoodie_commit_seqno#135,_hoodie_record_key#136,_hoodie_partition_path#137,_hoodie_file_name#138,id#139L,combine#140L,part#141L] Batched: true, DataFilters: [], Format: Parquet, Location: HoodieFileIndex(1 paths)[/tmp/test_table], PartitionFilters: [(part#141L = 1)], PushedFilters: [], ReadSchema: struct<_hoodie_commit_time:string,_hoodie_commit_seqno:string,_hoodie_record_key:string,_hoodie_p...

scala> spark.read.format("hudi").option("hoodie.metadata.enable","false").load(basePath).filter("part=1").explain(false)
== Physical Plan ==
*(1) ColumnarToRow
+- FileScan parquet [_hoodie_commit_time#65,_hoodie_commit_seqno#66,_hoodie_record_key#67,_hoodie_partition_path#68,_hoodie_file_name#69,id#70L,combine#71L,part#72L] Batched: true, DataFilters: [], Format: Parquet, Location: HoodieFileIndex(1 paths)[/tmp/test_table], PartitionFilters: [(part#72L = 1)], PushedFilters: [], ReadSchema: struct<_hoodie_commit_time:string,_hoodie_commit_seqno:string,_hoodie_record_key:string,_hoodie_p...

"I have a huge performance drop when disabling metadata table at read time."
The above should read more like
"I have a huge performance improvement when enabling metadata table at read time."

spark.sql.sources.parallelPartitionDiscovery.threshold=32
may be you can bump up this config to a bigger number since you have 40K partitions to see some improvement  to help with the no metadata situation.

@rkkalluri this makes sense ! thanks for your explanations

I am able to force reading of just 1 partition with plain spark like
scala> spark.read.format("parquet").option("basePath",basePath).load("/tmp/test_table/1").show()
+-------------------+--------------------+------------------+----------------------+--------------------+---+----+-------+
|_hoodie_commit_time|_hoodie_commit_seqno|_hoodie_record_key|_hoodie_partition_path|   _hoodie_file_name| id|part|combine|
+-------------------+--------------------+------------------+----------------------+--------------------+---+----+-------+
|<PHONE_NUMBER>3553165|20220708143553165...|                 1|                     1|abb854c5-dbdc-4ec...|  1|   1|      1|
+-------------------+--------------------+------------------+----------------------+--------------------+---+----+-------+
The equivalent version for hudi is not working though...it is reading data from all partitions and doing catalog file metadata collection like below.
scala> spark.read.format("hudi").option("hoodie.metadata.enable","false").option("basePath",basePath).load("/tmp/test_table/1").show()
+-------------------+--------------------+------------------+----------------------+--------------------+---+-------+----+
|_hoodie_commit_time|_hoodie_commit_seqno|_hoodie_record_key|_hoodie_partition_path|   _hoodie_file_name| id|combine|part|
+-------------------+--------------------+------------------+----------------------+--------------------+---+-------+----+
|<PHONE_NUMBER>3553165|20220708143553165...|               196|                   196|abb854c5-dbdc-4ec...|196|    196| 196|
|<PHONE_NUMBER>3553165|20220708143553165...|               199|                   199|abb854c5-dbdc-4ec...|199|    199| 199|
|<PHONE_NUMBER>3553165|20220708143553165...|               190|                   190|abb854c5-dbdc-4ec...|190|    190| 190|
|<PHONE_NUMBER>3553165|20220708143553165...|               193|                   193|abb854c5-dbdc-4ec...|193|    193| 193|
|<PHONE_NUMBER>3553165|20220708143553165...|               194|                   194|abb854c5-dbdc-4ec...|194|    194| 194|
|<PHONE_NUMBER>3553165|20220708143553165...|               197|                   197|abb854c5-dbdc-4ec...|197|    197| 197|
|<PHONE_NUMBER>3553165|20220708143553165...|               195|                   195|abb854c5-dbdc-4ec...|195|    195| 195|
|<PHONE_NUMBER>3553165|20220708143553165...|               198|                   198|abb854c5-dbdc-4ec...|198|    198| 198|
|<PHONE_NUMBER>3553165|20220708143553165...|               192|                   192|abb854c5-dbdc-4ec...|192|    192| 192|


Add sub-layouts option (Enhancement)

Could you add a feature where you can have a base layout and then multiple derivative sub layouts that build upon the base, but only include those elements that are unique to the sub layout?
This way you could modify the base layout without needing to duplicate the changes

I investigated this a bit in the past and decided on a slightly different approach: element references
You can copy 1+ elements from one layout and paste them into another layout as references (manual covers a bit more on them).

I have done some brainstorming on how this feature could be implemented without a massive overhaul of any of the underlying systems.  I have recently been trying to streamline the process of having sub-layouts because I have a main card with multiple sub-pieces which are all very similar but unique, and the process can be automated in a manner of sub-layouts.  I currently have it set up such that one layout with transparent backgrounds exports an image based on the spreadsheet, and then a second layout loads these images into the correct places.  (I use custom naming schemes to find them in the output folder.)
The weakness of the above approach is that if I want customizations based on the main layout to work in the sub-layout, I have to create many variations of identical parts, just with different colors, to use in the primary scheme.  This ends up with hundreds to thousands of extra images that I'm referencing only for the purposes of inserting them in another layout.  Additionally, there is a headache of having all of these extra entries in the spreadsheet, instead of just a series of primary ones and then individual ones where they should be unique.
My proposed solution to this, to make it work, is a shortcut for many parts of this which I imagine would be easy to implement behind the scenes.  Basically, one designs a layout as normal to serve as a sub-layout.  It can reference its own rows in the spreadsheet in order to have the necessary duplication of interests.  Then, on another layout, there is a new element type, we can call it "Layout Reference" for simplicity.  The "description" field would be the name of the other Layout for duplications. What this layout does: for each row of it's own spreadsheet, it passes all of its own data dictionary as additional entries to the sub-layout.  This means that if the sub-layout is referencing "@[SpecialColorA]", and SpecialColorA is a field in the primary layout, then the secondary/sub layout would receive that as well, and be able to use that field. For EACH iteration of the layout which contains the "Layout Reference" field, it would invoke the SubLayout as well, passing in the additional fields from the primary spreadsheet row.  Then after that SubLayout has iterated, the rest of the main layout runs.  The outputs from the SubLayout could be stored in the typical output folder, using the custom names defined in its own properties, as normal.  Each invocation of the primary layout, which would trigger the sublayout, could use the outputs from the sublayout as inputs to its own fields.
The "Layout Reference" field wouldn't have a visual representation, just tell CardMaker which other Layouts to invoke for each iteration of the primary Layout, by passing the custom per-main-layout info along into the sub-layout's data definitions.
The use case that I have looks like this:
Main layout containing 11 sub-layouts which are definitions of different units available for the player.
Each of these sub-layouts is effectively the same, in terms of the content, except sometimes there are individual values which are unique to a specific unit. Additionally, each unit is supposed to be colored based on parameters unique to the invocation of the main layout.  Instead of creating 11 new lines in the sub-layout spreadsheet for each main layout, which is of course doable, it would be wonderful if the sub-layout only contained the special parameters for the unique variants, and then did lookups into the main layout for the coloration necessary to fulfill the main layout.  Sometimes the sub-layouts also have iconography which is associated with the main layout, but that would be the only differentiating element on the entire sublayout.  This icon could be defined in the primary layout, and then it would be automatically incorporated when invoked from the primary layout.
In summary, TLDR, the requested functionality is merely to have a functionality where one layout can trigger the execution of another as a pre-step, and that the fields in the first layout are appended to each of the rows of the spreadsheet for the referenced layout.  We already have the wiring for the rest of the functionality as necessary, because of the benefits of transparent backgrounds, custom output names, etc.
If you'd be open to it, I would be willing to try to implement this new field type, if you could tell me where you think these parts would go?  (New entry to the tab for tabControl, new layout on it to specify which layout to force to re-run in advance, and where to stick the code to append the data and force a pre-execution)

I'll probably have to review this a couple times to make sure I'm understanding it all. We'll want a clear set of requirements to make sure we cover it all. 👍
Some things to consider

What index does the SubLayout process at with its own references? Is it always the same index as the PrimaryLayout? What happens if the counts do not match up (default to the last entry in the SubLayout)?
I think making an Element that is purely a Layout Reference is probably the easiest (I hope 🤣) path. The only values that might impact how it is rendered would be the x/y position. This way a SubLayout could be re-positioned (maybe more could be nice).
The main thing I would need to investigate is reference loading and translation so the PrimaryLayout reference data can be used as potential overrides when rendering/processing the SubLayout with its own reference. It's another layer of override that doesn't exist now.

Clarification

The outputs from the SubLayout could be stored in the typical output folder, using the custom names defined in its own properties, as normal. Each invocation of the primary layout, which would trigger the sublayout, could use the outputs from the sublayout as inputs to its own fields.

I'm not quite sure I follow this. Is the idea that Rendering of the PrimaryLayout forces SubLayout(s) to export an image and the image is rendered?
Repeating the concept in my own words
(just to make sure I'm understanding!)

A Layout ("PrimaryLayout") can have SubLayouts
SubLayouts should be drawn in order relation to other SubLayouts/Elements
When rendering a given SubLayout it should have access to the Reference data of the PrimaryLayout. The data from the PrimaryLayout should override any data by the same name the SubLayout uses.
One possibility would be to have the SubLayout render to an image and then draw that image when producing the PrimaryLayout.

I'll review again later to make sure I'm not missing anything.

Your repeat of things in your own words is complete and correct.  The piece that is missing will solve the requested clarification and also answer your first question about which indexes happen.

My suggestion is that the Referenced Layout is executed in its entirety.  So if the SubLayout has 15 entries, they are all executed.  This means that if someone wants a SubLayout with only 1 instance, that's fine.  It will be executed in each Layout that needs it. If someone wants a SubLayout that has many variations so they can include them all, then they get all of them processed, for every entry of the Primary Layout.  This isn't a duplication of work, it's just moving where it's done.  In my example that I'm trying to solve, I already had all of the same fields in the Primary Layout, duplicated 11 times, so I had hundreds of fields.  This merely cuts those down to a single Reference Layout field, which tells which SubLayout to execute.  Next answer tells how it's USED.
The second and third answer is actually the same one: The SubLayout would create all of its assets AS NORMAL, the only change to how the SubLayout functions is that it has access to more/specific information coming from the Primary Layout.
This has the effect of creating an asset/output however the SubLayout is configured.  The part that comes up is that in order to USE the output, we create a Graphics entry like normal and put into it's description the filename(s) we expect to be in the output from the SubLayout.   Since we're guaranteed that the SubLayout is going to run before anything happens in the Primary Layout, all of the assets necessary would already be created. This also means that the assets can be created at a different resolution, and then stuck into the Primary Layout using all of the fancy features of the Graphics tag, including things like rotation, mirroring, scaling, shading, etc.
I'm suggesting that the job of placing one or more assets from the SubLayout isn't the job of CardMaker, it's the job of the user to decide how the Outputs are used.  It's not as easily user friendly, but it's significantly more straightforward to design.  Forcing a user into a specific usage pattern takes away their choice and agency, and simultaneously requires more programming time on your part.   The goal of my suggestion is to make the processing merely happen to make the Layouts (be benefit of having access to the Primary Layout's data definitions), leave it up to the user if they want N new SubLayout output(s) and also how they want to use them.  You've done a great job of making the system flexible, and as complicated as it already is, the concept of using a SubLayout to make new assets for each Primary Layout, and then being able to use those generated assets through self-defined Graphics, is not overly complex.  Writing up an example is "Primary Layout A needs a generated asset -> SubLayout B can generate that asset with parameters supplied by A. Then we just make a Layout Reference [A relies upon B], and a corresponding Graphics entry that looks for the outputs of B, and uses them accordingly."

I am a strong supporter of having the Primary Layout definitions override any SubLayout definitions, if feasible.  I'm also perfectly happy forcing them to be unique and identifying them the way that conflicts are already identified, between th ings like the _defines files and normal columns.
Hopefully this has clarified what questions remained, and elucidated that the amount of work necessary to complete this task isn't actually immense.  We don't need to worry about coordinate systems, placement, forwarding of information, hard links, etc. We just need to ensure that the Referenced Layouts are executed a priori and therefore have the generated assets available in time for the standard workflow.

It's important to mention that since the SubLayout is merely executing along it's normal output creation path, using its saved output parameters, the SubLayout's outputs can also be used without being in a SubLayout at all.  This would guarantee that the same asset being used in multiple places would be identical, especially if the SubLayout was being printed on something larger AND ALSO being printed as something individual like a companion card.

I would very much like the chance to pursue this as a test or partial implementation, if you could guide me in the direction of where you think the changes should be made.

I think I'm slowly understating what you're going for with this. 🤔
Another attempt at my interpretation 😃
Layouts

SubLayout (2 items)
PrimaryLayout (10 items)

Has a reference that looks kind of like the table below

count
title
color

1
Rock
Grey

1
Tree
Brown

Export of PrimaryLayout

Row 1 of the reference:

Export the SubLayout with the row data fromPrimaryLayout Row 1 (title Rock and color Grey are passed in)

Overwrites any existing SubLayout_1.png and SubLayout_2.png (uniquely generated with the title & color)

Rest of the elements on PrimaryLayout are rendered and exported to PrimaryLayout_1.png (some might use the SubLayout_? files)

Row 2 of the reference:

Export the SubLayoutX with the row data fromPrimaryLayout Row 2 (title Tree and color Brown are passed in)

Overwrites any existing SubLayout_1.png and SubLayout_2.png (uniquely generated with the title & color)

Rest of the elements on PrimaryLayout are rendered and exported to  PrimaryLayout_2.png (some might use the SubLayout_? files)

... continues for all 10 items in PrimaryLayout.
Additional Thoughts

One concern is that the references will reload every time you re-export the SubLayout (if you happen to be using Cached Google References you might be avoiding that).
I've hit some occasions where the operating system reports files as missing so we'd need to be sure the file is completely written to disk and available before the PrimaryLayout attempts to use it via a Graphic element.

Alternative (sort of) approach
I think offering parameterized layouts via a Graphic element could be an interesting approach.
::SubLayout::[Card Index]::[parameter name]::[parameter value]::[parameter name]::[parameter value]::(continue with more parameters)
This would force the SubLayout to load, pass along the specified parameters, generate exactly the card index specified and potentially do so completely in-memory.
Files To Investigate (specifically around exporting layouts)
Files to potentially look into for the export of the SubLayout:

CardMaker\Card\Export\FileCardExporter.cs
CardMaker\Card\Export\CardExportBase.cs
CardMaker\Card\Deck.cs

I think you will need to pass in an override set of reference data to the FileCardExporter and then merge/override the values on the Deck (ValidLines and the Defines).
It's also possible you could perform the merge/override in the CardMaker\Card\DeckReader.cs itself.
The biggest overall challenge is getting whatever parameters you need into the SubLayout export.

I really love the Alternative approach suggested, because that would solve having to make an entirely new element type.  My concern is that the card index might be strange, because it will change over time, and I would end up with #(if @[Card Override 15] != 3 then @[Card Override 15 else 3 )# in every field, and then if just one card gets inserted into the SubLayout's spreadsheet then every override and every default would also have to change for every field and every override field in the PrimaryLayout spreadsheet.
My plan for just adding the override fields from the PrimaryLayout and the following the rules for the SubLayout itself means that it can output Custom_@[Unique-But-Constant Card ID].png for each of it's cards, and then the PrimaryLayout just needs to have a Graphics field which specifies<EMAIL_ADDRESS>and part of the logic for the Graphics field would have an if-statement which would default to one of the base layouts if nothing is specified for the Primary Layout. Making it depend on specific indices in the SubLayout complicates these matters from an automation standpoint, we can no longer just "create cards", we have to track the card dependencies in our heads too.  Allowing for a range of cards to be accessed through the suggested alternative approach, so a range could be created, would somewhat fix this, but then we risk causing a re-interpretation for the entire sub-layout for every Graphics element which wants N potential imports from the SubLayout.  10 Graphics elements, each one needing the entire SubLayout, even though the outputs of the SubLayouts are all the same because they're fed from the same PrimaryLayout row, means that we would need better dependency tracking.
As far as ensuring that the files are written in SubLayout before grabbing them from PrimaryLayout:  I believe that closing the file handle causes a flush and in this case it would function as we want.  Adding a file-availability check to PrimaryLayout as a bug fix shouldn't be too difficult if this error crops up.
As for the concern of re-loading the references repeatedly, I wasn't aware that you're force reloading the references at the start of each layout running.  My suggestion would be to step which does a refresh at the start of the process, then keeps the entire layout's data of everything in memory to reuse for the rest of the layout.  Basically, allow a SubLayout to NOT re-check its data after the initial grab.  In my case I'm testing with Excel, so the testing phase can be low-impact for now.
In summary, yes your idea about how the flow works is correct now.  You do fully understand.  The key you were missing which unlocks simplified access is that we can have custom filenames for SubLayout, by using the "Name Format" field for the SubLayout.  So instead of SubLayout_N.png, we get Special_Card_Orange.png or Default_Card_Ally.png (from the same SubLayout invocation!).  Perhaps in the Layout Reference field we can have [SubLayoutName]::[SubLayout File Name Format] if we want to use that functionality to give us custom filenames on a per-layout basis.  That would be pretty effective.  This SubLayout File Name Format would be easy to understand as well, since the fields in it can use the full Incept translator and will have access to the SubLayout fields as well as the Primary Layout's fields.

Correction, I properly have one layout causing the execution of another layout, based entirely upon a new ElementType which I created called "SubLayout".  The element 'value' is the name of the new Layout.

To simplifying development, since I'm not sure how you want to handle it, I added a flag to disable ProgressReporting since it's no longer consistent.
I added a function to ProjectManager called GetLayoutIndex which takes the name of a Layout and returns it's index.
New ElementType::SubLayout
Function inside Deck that combines a new field ParentPrintLine with the CurrentPrintLine.  The ParentPrintLine is set as the value of this same function, to allow for recursive nesting of layouts.  All instances of CurrentPrintLine in the code use this new function. THIS IS WHERE ITS INCOMPLETE  I need help figuring out how to make this function properly combine the lines, and their corresponding column names (I guess they're only stored in Translator?)
All other changes are in FileCardExporter.  I created a new function which traverses the layout in advance and isolates each of the SubLayouts, in forward order.  It uses the Translator of the CurrentPrintLine+ParentPrintLine on the value, then does a search for the result inside ProjectManager.Instance.GetLayoutIndex() referenced earlier. Any layouts it finds get added to a list and returned.
Inside FileCardExporter::ExportThread, the only changes I needed to make were to figure out the SubLayout list before we loop through the cards, and then at the start of the card loop, create a fresh FileCardExporter of similar/default settings for each of the found SubLayouts, and then execute it in-thread.  After that loop is finished, it resets the current Layout through ChangeExportLayoutIndex(nIdx) to bring consistency back to the Parent Layout and continues with that particular card.

All of the code referenced above is relatively trivial to write.  I've pushed all of my changes to my fork at https://github.com/MarkVabulas/cardmaker if you'd like to see the diff for the entire change log.  I'm curious what you think about this stuff.  I'll create a PR if you would like, but I'm pretty sure none of this is up to the standards or methods you might want to use in the end.

I'm glad you were able fairly far along.
I'll add some comments to the submit you made on things I found when trying to get things somewhat working.
I'll take a more in-depth look hopefully in the next day or so.

Added a comment and patch with my in-progress hacking: https://github.com/MarkVabulas/cardmaker/commit/3b83c967605f9d0d1d808b5c2726825c2c84cf96#commitcomment-140690353

New seemingly-100% working commit.  Testing required, but I'm going to do that extensively, hopefully this weekend.  The commit changelog explains what I did.
commit 4bf9655
I incorporated your changes and made the necessary improvements.  It all works correctly now, including with the cache.  Each Primary Layout is triggering a rebuild of the different cards in the SubLayout and able to access the fields in the SubLayout.  I removed the GetFullPrintLine since it actually became useless once the Translator was incorporated.  The Primary Layout doesn't/shouldn't need access to the SubLayout, and the SubLayout only needs the fields for translation purposes.  I also saw your #warning at Deck.cs:264, but I don't think we actually need that info anywhere.  What were you referring to?
I did switch the order of checking for the Translator to use the PrimaryLayout info first, so it can function as an override.  I also reverted the not-found functionality back to as-before, since it shouldn't silently fail for a Layout just because it's also going to be a PrimaryLayout.  We should have the fields defined in both, if it's being used, AND overridden.
I'm really excited to try this on my problem set, but today is a day to relax :)

And sorry for the line-endings getting screwy, I tried to make sure they wouldn't get messed up this time and it still happened.

So this works great.  The sub-layouts are all originally defined as white, but the purple ships and red boxes around the numbers are all re-defined in the ParentLayout as overrides.  The issue we are having is that it's reusing the image cache from the first card on subsequent ones, even though it is still correctly generating the SubLayouts (I can see them in File Manager thumbnails getting updated, but then the output for the next card contains the cached images).

It should be stated that (I didn't time it) this card appears the take an essentially equal amount of time.  I kept my old layout, and it takes about 1 minute.  This new one using SubLayouts takes about a minute as well.
I included the ImageCache.ClearImageCaches() call inside the loop after resetting the Primary Layout CardPrintIndex, and it works perfectly now.  link to commit

The only difference between all of this mini-ship cards is what is defined in the PrimaryLayout.

It definitely seems like there is a bug that is requiring the call to ChangeExportLayoutIndex(nIdx); on every card export. I'll have to check out why that is necessary. The primarylayout/sublayout exporters and their CurrentDecks should be independent (but may not be!).
New Element Type vs. Layout Setting
For this initial implementation (not the advanced one that would use a Graphic element type): Do you need need the sublayout name to be translated as a completely independent element? I am considering just making a layout level variable like sublayoutsExportPerCard and it would just have a comma delimited list (shipbackground,gamelogo,base).

ChangeExportLayoutIndex is required because the call to CardRenderer and all all calls inside there use the (GLOBAL!!!!!) CurrentDeck and the deck changes in the start of every call to ExportThread.  (CurrentDeck is fully global, from what I can tell.)  Since we're calling ExportThread from inside ExportThread, the Sublayouts are overwriting the deck which was chosen by the PrimaryLayout.  You don't really have a full MVC style separation of data happening because your renderer is using whatever some current state is, which is shared elsewhere, instead of using only parameters passed in from elsewhere.  If you wanted this to function better, then CardRenderer, and even deeper the exporter, should only access data from things passed into them.  That has knock-on effects in places like the Translator because it's also using the global state.  Basically, everywhere in the software you've saved yourself the hassle of passing around the deck/layout because you only ever needed 1 and only 1 at any given time.  If we wanted this to be more seamless, it would all have to be thrown around as arguments instead then we should ship off commands like SubLayout renders in a multi-threaded way, because they wouldn't all simultaneously be using the CurrentDeck as the central repository of what's happening.  Just the mere existence of using a global CurrentPrintLine is an example of a design which would make this part of the project difficult.
For the part about New Element Type vs Layout Setting, I had the same thought, and the only conclusion I came to was that there are actually 3 ways to do it.

As we currently are, with a SubLayout Element (Which has the benefits of a full implementation of the incept translator for the value field and selectively choosing layouts programmatically)
Using a Layout-wide property which we can set as "we need ____"
As a PER-ELEMENT reference which defines which other layout this Element itself specifically depends upon

The first choice I feel is very flexible because of the incept translator, the ability to double-click to disable the SubLayout which would automatically disable the execution of it (if we don't need it every time), and it's also visually evident that it's there as a process/step because it's in the list of elements.  I'm fully expecting and hoping to rely on the existence of the incept translator for SubLayout selection.
Having a layout-wide property is great because it more closely represents the relationship between two different layouts, and it is technically a properly of the layout moreso than as an element itself.
Having a per-element reference, for Graphics AND textual fields, that declares that the contents depend directly upon a specific Layout for that element to render correct gives the flexibilty of disabling Layouts, but also means we can know more directly in the future which parts rely on what.  So if a specific element's value is changed, we could theoretically update the SubLayout to make sure it renders correctly, without having to do that same processing task Layout-Wide during any given edit.  The reason that even FormattedText fields might need a SubLayout to execute first is because of inline images or background images, etc.

The one thing which is currently missing from the SubLayout implementation is a more robust preview functionality, but in my view, the base SubLayout would need to exist as a pre-rendered item first anyway just to mock it up.  I feel it's not unreasonable, and would be understandable to the user, that before a SubLayout can be used in a Primary Layout that it needs to be rendered first at least once, independently.  While the SubLayout's exported preview-contents might not end up matching the final output, a quick run of a single-card export followed by a cache refresh gives immediate results for previewing the final task as-is, without forcing the user to sit through a full SubLayout re-export just to design their Primary Layout.
In the end, if we can keep the text translator working on the contents of whatever field is used for the SubLayout, I think that's optimal.  Personally, I'm not wed to any given solution, but I really like the idea of having an Element which serves as a visual reminder and controller for SubLayout References, it makes it feel more obvious that "yes, this is going to have to do something" and I'm worried that hiding that behind a settings screen might make it feel less impactful than it really is.  SubLayouts ARE "big-ask" types of card rendering.

ChangeExportLayoutIndex is required because the call to CardRenderer and all all calls inside there use the (GLOBAL!!!!!) CurrentDeck and the deck changes in the start of every call to ExportThread. (CurrentDeck is fully global, from what I can tell.)

CurrentDeck is not actually global (fortunately!). 😄 But, it's possible there are some other global-like aspects that may cause bugs with sublayout exports. The loaded project is definitely global, but the card rendering should be independent. You could have 20 layers of exporters and they shouldn't interfere with each other (at least that was the plan long ago).
To further clarify how CurrentDeck is constructed for exporters -- Each FileCardExporter extends CardExportBase. CardExportBase has this as part of construction:
            CardRenderer = new CardRenderer
            {
                CurrentDeck = new Deck()
            };

The intention was to allow the application UI to render and maintain its own CardRenderer/CurrentDeck (in CardCanvas and every exporter).
Something to try:
Can you let me know what happens with this change in FileCardExporter with your export of sublayouts? This is working for me so I want to narrow down where things aren't working for your use case. My test case is fairly trivial but I am able to export the primary with sublayout exported images that are customized by primary reference values.
if (vSubLayoutArray.Count() > 0)
{

//this call should not be necessary and is the cause of needing the CurrentDeck.CardPrintIndex assignment below
	//ChangeExportLayoutIndex(nIdx);

// this should be harmless but also not necessary
	//CurrentDeck.ResetDeckCache();

// as mentioned above this is caused by the ChangeExportLayoutIndex
	//CurrentDeck.CardPrintIndex = nCardId;

// definitely necessary as the exported file names of previously loaded files could be the same
	ImageCache.ClearImageCaches();
}

A potential TODO item would be for me to adjust the image cache to store the last updated time so we can avoid a complete cache wipe with ImageCache.ClearImageCaches();

So I removed those lines and it seems to function as before, with no discernable difference.
One issue that cropped up is that occasionally, randomly, and without logic, about 5% of the time ALL of the defintions inside the SubLayout fail.  The images still load, and they are controlled by the Incept translator so the values aren't completely nulled out, but in general every single SubLayout fails for that particular Primary Layout.  Then if I re-run that individual card solo using the Export button on the card, it works fine.  But it's not guaranteed that it fails and it's not consistent which one it happens to during the run.

I'm having trouble reproducing it to show you an example, but it has happened multiple times, and each time I confirmed it could be fixed, but I'm having trouble getting it to fail on command (it knows I'm watching!).

Here we can see what the SubLayout ends up producing.  Clearly the FactionIcon is coming from the ParentLayout, and showing up on the SubLayouts, as well as the background correctly, but the rest of the SubLayouts are broken.

My newest commit to my main branch includes FileCardSingleExporter working as well.

I've seen some odd cases over the last few months where a CSV file reference wouldn't be found on disk (the sample.csv which has existed on my PC for years). Definitely odd.
Are you seeing any log messages about files failing to load? With the ProgressReporter removed some issues with the sublayout export might be logging to nowhere. Sublayout exports might need a simple IProgressReporter implementation to at least allow for AddIssue to be called and tracked.
I might have to programmatically call the sublayout export a 100 or 1000 times in row and watch for issues.
Let me know once you feel like you're in a stable spot and I'll start planning the work on bringing things over to the main app repo. I'm probably going to stick with the element type approach and potentially do other things with it in the future.

I didn't notice any particular log messages or errors.  We'll have to investigate, but I can run it 10 times in a row with SubLayouts and not have it fail even once, so it's truly intermittent.
I am currently going through and doing more extensive testing of the SubLayouts, but I'm currently not foreseeing any more changes to the system than what we've already got in my HEAD commit on my fork. (The latest commits, I force-untabified my files before committing them back in)

Would you like me to submit a PR with my changes?

Concerning the IProgressReporter, if I had more familiarity with it, I would have implemented it as a 4th Progress Bar on the Export window, to show the progress when going through any sublayout(s).  That would allow for visualization, but we'd also have to keep track of the levels of ProgressBars deep that we are, in case there are nested SubLayouts.

Initial submission here: https://github.com/nhmkdev/cardmaker/commit/92c6eb507f8f561f1a8c6eea7be9980cad1e29df
I'm going to close out this ticket and migrate to https://github.com/nhmkdev/cardmaker/issues/141


how can i modify batch_size

error:cuda oom
It seems that there is no optional named batch_size in config.if i want to decrease it，what should i do.

I ve got several “batch_size”in the project， such  as “train_batch_size”，“ eval_batch_size” and so on. Which one should i decrease to let the project run without "cuda oom".


Retrieve tweeningValue from onUpdate callback in documentation

The tweeningValue no longer seems to be available in the tween object itself. Instead, the tweeningValue is available in the tweened object that passed as a parameter to the onUpdate callback.

With the current version of Tween.js, this patch is actually a regression. See #1574.


Remove Label from new Form

When you navigate to Shesha configuration -> Forms and then create a new form, the request is that we remove the label field as it is not important and not required.
https://starter-adminportal-test.shesha.dev/dynamic/shesha/forms

Hi @OmolemoBlessingLethuloe, also remove the label column from the table view in support for this item


P2 load sample timeout

change the sample service to create a sample summary straight away, kick off the upload in the background and return immediately
change the collection exercise service to create a sample link immediately but only activate it when it receives a message on the sample-outbound-exchange with routing key Sample.SampleUploadFinished.binding
change response-operations-ui to display the state of the upload and any successful completion or error

Codecov Report

Merging #8 into master will not change coverage.
The diff coverage is 0%.

@@         Coverage Diff         @@
##           master   #8   +/-   ##
===================================
  Coverage       0%   0%
===================================
  Files           8    8
  Lines          59   60    +1
  Branches        6    6
===================================
- Misses         59   60    +1

Impacted Files
Coverage Δ

...sponse/sample/representation/SampleSummaryDTO.java
0% <0%> (ø)
:arrow_up:

.../response/sample/representation/SampleUnitDTO.java
0% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a9cb3d3...e17f5aa. Read the comment docs.

 P2 Load sample timesout and doesn't generate samplelink record in collectionexercise


[Bug]: Wrong list of coordinate when a singleton coordinate exists

What happened?
Here is some simple code:
a = xr.DataArray(np.arange(5), dims='x', coords={'x':np.arange(5)})
a = a.assign_coords({'y':1})

Now calling a['x'] or a['x'].coords shows y as a coordinate of x, which is unexpected for me
What did you expect to happen?
I expect that a singleton coordinate of a dataset  not to be a coordinate of other coordinates present in the dataset
Minimal Complete Verifiable Example
import xarray as xr
import numpy as np
a = xr.DataArray(np.arange(5), dims='x', coords={'x':np.arange(5)})
a = a.assign_coords({'y':1})
print(a['x'].coords)

returns
Coordinates:
  * x        (x) int64 0 1 2 3 4
    y        int64 1

Relevant log output
No response
Anything else we need to know?
No response
Environment
INSTALLED VERSIONS
commit: None
python: 3.8.12 | packaged by conda-forge | (default, Oct 12 2021, 21:57:06)
[GCC 9.4.0]
python-bits: 64
OS: Linux
OS-release: 3.12.53-60.30-default
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.12.1
libnetcdf: 4.8.1
xarray: 0.19.0
pandas: 1.3.5
numpy: 1.20.3
scipy: 1.6.3
netCDF4: 1.5.8
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: <IP_ADDRESS>
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 2021.10.0
distributed: 2021.10.0
matplotlib: 3.2.2
cartopy: None
seaborn: None
numbagg: None
pint: None
setuptools: 60.5.0
pip: 21.3.1
conda: None
pytest: None
IPython: 7.31.0
sphinx: None

Thanks for raising this suggestion @lanougue .

I expect that a singleton coordinate of a dataset not to be a coordinate of other coordinates present in the dataset

What do you think it should return? When selecting a coordinate there seem to be only 3 options to me:

Coordinate variables retain all coordinates from the parent object.

This is what it currently does. It's a bit counter-intuitive, but it prioritises keeping coordinates available for use by users. It also means you can pull out a coordinate variable and then use other coordinates to index into it.

Coordinate variables retain no coordinates at all.

We could drop all of them, which makes some sense (why should a coordinate have coordinates?). We can try that out naively by adding a drop_vars call to DataArray._getitem_coord:
def _getitem_coord(self, key):
    from .dataset import _get_virtual_variable

    try:
        var = self._coords[key]
        coord_da = self._replace_maybe_drop_dims(var, name=key)
        return coord_da.drop_vars(coord for coord in coord_da.coords)

but this breaks a fair number of tests - 25 failed just in test_dataarray.py (although mostly parametrizations of one test):
FAILED xarray/tests/test_dataarray.py::TestDataArray::test_isel_fancy - AssertionError: Left and right DataArray objects are not identical
FAILED xarray/tests/test_dataarray.py::TestDataArray::test_sel_dataarray - ValueError: One or more of the specified variables cannot be found in this dataset
FAILED xarray/tests/test_dataarray.py::TestDataArray::test_coord_coords - AssertionError: Left and right DataArray objects are not identical
FAILED xarray/tests/test_dataarray.py::TestDataArray::test_coords_non_string - AssertionError: Left and right DataArray objects are not identical
FAILED xarray/tests/test_dataarray.py::TestDataArray::test_coordinate_diff - AssertionError: Left and right DataArray objects are not equal
FAILED xarray/tests/test_dataarray.py::TestReduce1D::test_idxmin[False-x0-5-2-None] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce1D::test_idxmin[False-x1-5-2-None] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce1D::test_idxmin[False-x2-5-2-1] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce1D::test_idxmin[False-x3-5-2-1] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce1D::test_idxmin[False-x4-nan-nan-0] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce1D::test_idxmin[False-x5-0-1-None] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce1D::test_idxmax[False-x0-5-2-None] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce1D::test_idxmax[False-x1-5-2-None] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce1D::test_idxmax[False-x2-5-2-1] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce1D::test_idxmax[False-x3-5-2-1] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce1D::test_idxmax[False-x4-nan-nan-0] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce1D::test_idxmax[False-x5-0-1-None] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce2D::test_idxmin[False-x0-minindex0-maxindex0-nanindex0] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce2D::test_idxmin[False-x1-minindex1-maxindex1-nanindex1] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce2D::test_idxmin[False-x2-minindex2-maxindex2-nanindex2] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce2D::test_idxmin[False-x3-minindex3-maxindex3-nanindex3] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce2D::test_idxmax[False-x0-minindex0-maxindex0-nanindex0] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce2D::test_idxmax[False-x1-minindex1-maxindex1-nanindex1] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce2D::test_idxmax[False-x2-minindex2-maxindex2-nanindex2] - KeyError: "'x' is not a coordinate variable."
FAILED xarray/tests/test_dataarray.py::TestReduce2D::test_idxmax[False-x3-minindex3-maxindex3-nanindex3] - KeyError: "'x' is not a coordinate variable."

*Coordinate variables retain only coordinates with which they share some dimensions

You might say "well in my example 'y' doesn't depend on dimension 'x', so why is it still there? We should keep the 'x' coord and drop the 'y' one." But I don't really think this option makes sense if you think of all coordinates as labels: the 'y' coord is a label for all values in the DataArray, that just happens to be independent of 'x'. Why drop one and not the other?
I agree this is a little counterintuitive, but it is actually causing you problems? Because I think there is a rationale for the current behaviour, and changing it would be a breaking change. In other words, it's not obvious to me that this is a bug rather than an implicit feature.
(One might also argue that if xr.Variable were first-class public API (as in our roadmap), then da.coords[var] should return a Variable, which has no coordinates. At the moment sel._coords[key] returns either a VariableorIndexVariable, which then gets wrapped back up into a new DataArray`.)
Interested to hear what you think though.

Thanks for the enlightening.
Actually, this coordinates dependency with singleton dimension caused me a problem when using the to_netcdf() function. No problem playing whith the xr.Dataset but I get some error when trying to write on disk using to_netcdf().
For now, I wasn't able to reproduce a minimalist example because the error disappears with minimalist example. I wasn't able to find the fundamental difference between the dataset causing the error and the minimalist one. Printing them are exactly the same. I have to do deeper inspection.
Concerning the philosophy of what a coordinate should be: For me the "label" idea is understandable at a dataset level. A singleton dimension become a (shared) "label' for the whole dataset. This is ok for me. However, I do not understand why it should also be a "label" of the other coordinates of the dataset. A singleton dimension should not be "more important" than the other (not singleton) dimensions. Why the singleton dimension should become a "label" of another dimension while the other dimensions are not. This do not seem logical to me.

For now, I wasn't able to reproduce a minimalist example because the error disappears with minimalist example. I wasn't able to find the fundamental difference between the dataset causing the error and the minimalist one. Printing them are exactly the same.

Can you perhaps use ncdump to show the differences?

However, I do not understand why it should also be a "label" of the other coordinates of the dataset. A singleton dimension should not be "more important" than the other (not singleton) dimensions. Why the singleton dimension should become a "label" of another dimension while the other dimensions are not. This do not seem logical to me.

I'm not quite sure I understand what you mean - currently aren't all coordinates reattached to any coordinate extracted from a DataArray? (so long as they have compatible dimensions)

Hello @TomNicholas ,
Reopening this issue 1 year later ! To answer your last question, singleton dimension seems to have, indeed, a unique behavior since they are reattached systematically to other coordinates (even if they naturally do not share any dimension with other coordinates).
These singleton dimensions introduce some strange behavior. This is another example:
a = xr.DataArray(np.random.rand(2,3,2), dims=('x','y','z'), coords={'x':[1,2], 'y':[3,4,5],'z':['0','1']}) b = xr.DataArray(np.random.rand(2,3,2), dims=('x','y','z'), coords={'x':[1,2], 'y':[3,4,5],'z':['0','1']}) res1 = a.sel(z='0')/b res2 = a.sel(z='0').expand_dims('z')/b
res1 and res2 do not have the same size on dimension "z". In res1, dimension "z" is not considered anymore as a dimension at all !


DNS challenge provider: External script?

Hi,
thank you all very much for your work on lego! I'd like to use lego (the CLI) for a project which requires using the DNS challenge and running custom code. So while I could add a custom DNS challenge provider and implement the code needed to setup the DNS record in Go, I'm wondering: Would you be willing to merge a new DNS challenge provider which calls out to an external program or script?
The background is that for this project, we're using infrastructure already in place, and all Go code that I would be writing to use it with lego would be highly specific for our setup and of no use to anybody else. Encapsulating this code in a small script (or separate Go program) feels like a much better idea. I can imagine that this is something which may be helpful to other users as well.
It could be named external or script, the name of the program/script to call could be configured via an environment variable. The program/script would then be run by lego with arguments, e.g. present fqdn value and cleanup fqdn value or so.
Would that be something you'd be willing to merge? If so, I'd be happy to do a PR.
Thanks again!

Hey @fd0! Nice to see you here.
I can only guess what xenolf would say, but I don't really see a problem with this. In fact, PR #417 might be what you want, although I think it needs more review before it is merged. I suggest getting involved on that PR and if necessary, make a new one.

Ah, interesting, I haven't found that PR. I'll have a look!

Ok, the PR does much more than I need, it also adds a second challenge provider which calls out to a named pipe. This could also be solved with a script, e.g. by using socat. I'll wait what the original author of the PR says, I'm happy to open a new PR and code this myself, but I'd like to avoid stepping on anybodies toes :)

Hm, it seems I misread the activity dates for the PR, the last activity by the submitter was in August last year. Do you think it's likely they will pick it up again?

I've tried to implement this and submitted PR #508, it was extremely simple.


Bump required videojs-wavesurfer version to 2.6.0

for microphone support in the MS Edge browser

Coverage remained the same at 83.807% when pulling 719c905af757746ba075293e355cce5ab165a6f2 on wavesurfer-210 into b237b26d56629fe60f4363fbaa2bfaa8185547b1 on master.


Not properly centered in 1440p

While in 1440p the search box and results appear on the left rather properly in the center.

Thanks for trying out this theme!
Yes, this is the current expected behavior.

results appear on the left rather properly in the center

Stylus (the theme injector) only has the capability to inject CSS style, so it's not possible to do what you're suggesting as that would require some javascript conditions to determine if there are any content on the right. (Which can be solved if i deploy a dedicated browser extension)
That being said, the intention of this theme is to position the google search engine to the middle of the screen without changing the default google experience.

you may say that but there are legitimate userstyles that do this, like this one for example: https://userstyles.org/styles/149976/center-google-search-results-improved

As my mention about not being able to detect right side content through css only.
You can see this introduces the problem of nonsymetrical layout as you can see below.

Hope that clears up the reason for not doing wanting to keep the current behaviour :)


Must introduce a copy button and a better documentation ...

Introducing a copy button in the docs would be better fitted as a suggestion over at issue #91 although I suspect copying a few lines would be more useful overall than copying the whole example.
I'm afraid you will have to clarify what you meant by better documentation, it would be more useful for us to know which specific areas that can be improved on and how, in terms of performance, design or anything else.

Thanks for comment. Actually I want to contribute in p5 because it changes my view about javascript. Documentation is good. I need more time to analyze documentation.  Thanks for your time

@glearner Glad to help. It would be great if you can close this issue if you have no further problem. Thanks.


Remove (or turn off?) all references to Listing.applicationDueDate

Was this originally to track all references in sites/public or sites/partners or both? I don't think we really have any references in the sites/public pages right now and we're displaying all listings instead of just those that are "open".

I think the original idea was just to remove it from sites/public, but it probably also makes sense to remove from sites/partners? It's displayed prominently on the partner dashboard:

I couldn't really find any references on the public site, so I think we've already done a good job removing it from there (and let me know if you see any). I'll focus in on removing it from the partner site - both displaying and editing.


Consider making a category for checking for iPhone or iPad devices

I see this type of code in three different classes:
if ([[UIDevice currentDevice] userInterfaceIdiom] == UIUserInterfaceIdiomPhone)
This works, but a better pattern might be to add two UIDevice category methods:
[[UIDevice currentDevice] isPad]
[[UIDevice currentDevice] isPhone]
These will keep the code a bit cleaner and simpler!

The methods should use a prefix, to avoid naming conflicts if apps integrating the library choose to implement the same method name in a category for the same class.
E.g. de_isPad


Add tests for #130: content + nocase issue

Closes redmine ticket #3057.

Requires 5.0. Will fix.

Merged with #151


Add possibility to define custom border color for disabled form controls

@MartijnCuppens: I assume we don't want this backported?

Nope

I'd be happy to make another PR for 4-dev if you'll have it. @MartijnCuppens


Banlist is deleted

The question is in the header itself. When you restart the containers using (docker compose down && docker compose up -d), the file with bans is deleted. It must not be deleted.

I think you expect to much from docker. When you run docker compose down it will delete all files inside the container. If you did not mount /home/steam/.local/share/Euro Truck Simulator 2 on your local file system all files will be deleted.
You need to mount /home/steam/.local/share/Euro Truck Simulator 2 using volume mounts to keep the banlist when using docker compose down like this:
[...]
    volumes:
      - "/opt/ets2-server/:/home/steam/.local/share/Euro Truck Simulator 2"
[...]

After you make this change your banlist might be deleted once more but after that it should stay in place.
If you want to view the current banlist before overwriting it again you can exec into the container and view it that way:
docker compose exec ets2-server bash
There you can navigate to /home/steam/.local/share/Euro Truck Simulator 2 and view all server settings files. Please note that server_settings.sii will be overwritten by the startup script whatever you do with the file. You can disable this behaviour using env variables.

Yes you can run multiple servers.
Just paste the same service config in your docker compose again. You will have to rename the service and change the port using ETS_SERVER_PORT environment variable.
Then it should work fine.
So it could look like this:
services:
  ets2-server1: # <- change for each server
    image: lshallo/ets2-convoy-server
    container_name: ets2-server1 # <- change for each server
    restart: unless-stopped
    tty: true
    network_mode: host
    volumes:
     - "/opt/ets2-server/save-data:/home/steam/.local/share/Euro Truck Simulator 2"
    environment:
      - "ETS_SERVER_UPDATE_ON_START=true"
      - "ETS_SERVER_NAME=My Server 1" # <- change for each server
      - "ETS_SERVER_DESCRIPTION=My server description 1"
      - "ETS_SERVER_WELCOME_MESSAGE=Welcome to my server!"
      - "ETS_SERVER_PASSWORD=abc123"
      - "ETS_SERVER_MAX_PLAYERS=8"
      - "ETS_SERVER_PORT=27015" # <- change for each server
      - "ETS_SERVER_QUERY_PORT=27016" # <- change for each server
      # more variables...

  ets2-server2:
    image: lshallo/ets2-convoy-server
    container_name: ets2-server2
    restart: unless-stopped
    tty: true
    network_mode: host
    volumes:
     - "/opt/ets2-server/save-data:/home/steam/.local/share/Euro Truck Simulator 2"
    environment:
      - "ETS_SERVER_UPDATE_ON_START=true"
      - "ETS_SERVER_NAME=My Server 2"
      - "ETS_SERVER_DESCRIPTION=My server description 2"
      - "ETS_SERVER_WELCOME_MESSAGE=Welcome to my server!"
      - "ETS_SERVER_PASSWORD=abc123"
      - "ETS_SERVER_MAX_PLAYERS=8"
      - "ETS_SERVER_PORT=27017" # <- changed here
      - "ETS_SERVER_QUERY_PORT=27018" # <- changed
      # more variables...


Cannot run serf_test.Test_Membership()

The test Test_Membership in serf_test.go keeps failing on me with error:
--- FAIL: Test_Membership (0.01s)
        Error Trace:    serf_test.go:37
	Error:		Received unexpected error Failed to create memberlist: Could not set up network transport: Failed to start UDP listener on "<IP_ADDRESS>" port 7948: listen udp <IP_ADDRESS>:7948: bind: address already in use

I cannot figure out why I'm getting this error. I trace the bind down to the point where it calls net.ListenUDP on UDP port 7948, set a breakpoint at this line and don't see any listening UDP connections on port 7948 at this point:
basph@MBP:~$ lsof -i -n -P | grep 794
___Test_M 6478 bas    8u  IPv6 0x21aa5a91cbda343b      0t0  TCP *:7947 (LISTEN)
___Test_M 6478 bas    9u  IPv6 0x21aa5a91c9953d5b      0t0  UDP *:7947
___Test_M 6478 bas   10u  IPv6 0x21aa5a91cbda2efb      0t0  TCP *:7948 (LISTEN)

Any ideas?

You might have better luck using netstat:
netstat -tulnvp udp | grep 7948

I'm gonna close since this isn't a jocko issue, it's something else running on your computer.


fix: port throw meaningful exception

Fixes #3158

Pull Request Test Coverage Report for Build 449241023

0 of 0   changed or added relevant lines in 0 files are covered.
20 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.05%) to 84.319%

Files with Coverage Reduction
New Missed Lines
%

libraries/botbuilder-lg/src/templateExtensions.ts
1
85.0%

libraries/botbuilder-dialogs-adaptive/src/input/inputDialog.ts
9
83.04%

libraries/adaptive-expressions/src/triggerTrees/node.ts
10
90.51%

Totals

Change from base Build 443119412:
-0.05%

Covered Lines:
17631

Relevant Lines:
19978

💛  - Coveralls


navController remove method doesn't work

Ionic version:  (check one with "x")
[ ] 1.x
[x] 2.x
[x] 3.x
I'm submitting a ...  (check one with "x")
[x] bug report
[ ] feature request
[ ] support request => Please do not submit support requests here, use one of these channels: https://forum.ionicframework.com/ or http://ionicworldwide.herokuapp.com/
Current behavior:
The navController remove method doesn't work as expected.
When the number of pages to remove are superior to 1 it's do nothing (and no errors are thrown).
Expected behavior:
In normal case it remove the number of pages.
Steps to reproduce:
I made a Plunker : https://plnkr.co/edit/lCPWvh?p=preview
You need to go to the last page (ionic go to home page) and click on the "Go to home page (bug)" button.
Related code:
this.navCtrl.remove(this.navCtrl.getActive().index, 2);

Other information:
I tried with index-1 (first parameter) but it not work. Same with -2 (second parameter) instead of 2.
In my project i use the following code in order to bypass this bug.
this.navCtrl.popTo(<any>(this.navCtrl.getActive().index-2));

Hello, thanks for using Ionic! We will look into this.

Possibly related to https://github.com/ionic-team/ionic/issues/12238

This issue is still at<EMAIL_ADDRESS>

As @JeongJun-Lee I can confirm this is still an issue
ionic-angular@3.9.2


The words can't start with the forbidden characters

Closes: #1
Updated validations.py python script.
Fixed the behavior of validate_user function in validations.py.
Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

[ ] Tests pass
[ ] Appropriate changes to README are included in PR

@googlebot I signed it!


Extra help?

Hi Tom,
First off this is a great package! Thanks so much for creating it.
Wondering if you'd be open to some help or extra maintainers for the project? I'd like to help out if you were open to it in terms of PRs, code reviews, or adding additional features. No worries either way, just hoping to help if possible and keep this great project going.
Thanks and Happy holidays!

Thanks for the offer. I'll look into making an engarde-dev organization and moving things over there.
cc @topper-123 if you'd be interested as well.

@topper-123 do you have any open source examples of the things you are doing?
Always curious to see how folks test in the data science / data engineering space

Please setup an engarde-dev organizational.  I am willing to help and want to make several improvements.  I have    emailed you and would prefer to enhance engarde instead of forking it.

Moved to https://github.com/engarde-dev/engarde and invited. LMK if you run into other blockers.

Possibly CI things like travis / readthedocs. Not sure if they inherit permissions from GitHub.

Thanks.  My changes will include adding the has_columns, added the ability to check input parameters or index of an function return argument. Also made it disable-able, and added Eval checks. I need to work with my company to make the changes.

Thanks @TomAugspurger for setting this up.
In terms of ongoing work do you have a certain direction you want the package to go? Just want to make sure we don't deviate too much from your vision.

Thanks @TomAugspurger for all the effort. When I didn't hear back I ended up creating a new library called Bulwark (and crediting your influence :) ). It has some changes to the backend, some new tests, and slightly different check names, but largely a continuation of all the work you did. Given that Bulwark has some active users/contributors at this point, I think I will continue to work on that project, but sounds like you have a couple of other great volunteers here. If you're interested in checking out the project:
GitHub: https://github.com/ZaxR/bulwark
PyPi: https://pypi.org/project/bulwark/
Docs: https://bulwark.readthedocs.io/en/latest/

@TomAugspurger Seems like I might only have read access. Was trying to setup Travis and push to a branch in engarde-dev but can't. No biggie. Will just work off a fork, but suspect that tests won't run?

When I didn't hear back I ended up creating a new library called Bulwark

Sorry about that (and nice name 😀). Is it worth trying to move people to bulwark?

No worries at all. The only blocker to moving people to Bulwark would be that I dropped python 2 support (in line with the direction of pandas itself). If it’s important to the engarde user base, I’m open to adding that back, though. Should only take a day to do.
Migration would be as simple as changing engarde to bulwark in many cases, with a few that would just need a tweak to the function name (old names could be added with a deprecation warning to help people out).
Let me know what you think.

I'm happy to leave that decision to others in this thread. I don't really
have a horse in this race.
On Fri, Jul 26, 2019 at 8:06 AM Zax<EMAIL_ADDRESS>wrote:

No worries at all. The only blocker to moving people to Bulwark would be
that I dropped python 2 support (in line with the direction of pandas
itself). If it’s important to the engarde user base, I’m open to adding
that back, though. Should only take a day to do.
Migration would be as simple as changing engarde to bulwark in many cases,
with a few that would just need a tweak to the function name (old names
could be added with a deprecation warning to help people out).
Let me know what you think.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/engarde-dev/engarde/issues/56?email_source=notifications&email_token=AAKAOIR3I2IHNUVYK5FTW3LQBLZFPA5CNFSM4GL6BDA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD24QXTA#issuecomment-515443660,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAKAOIVHKY3POMWOQOVOEILQBLZFPANCNFSM4GL6BDAQ
.

@alysivji @fdosani thoughts on migrating / directing people to bulwark?

I know Zax personally; have seen him spend a lot of time on bulwark. He's been pretty active in promoting the project and recruiting new contributors; had a lightning talk at PyOhio and led a sprint session.
I'm not in the data space anymore, but feel it's best to point folks towards tools that are actively maintained. Adding names with a deprecation warning makes sense for current engarde users to ease the path of migration.

@alysivji @fdosani thoughts on migrating / directing people to bulwark?

I think this makes sense. Happy to help @ZaxR if you are looking for more maintainers

Always looking for more ideas and help! I have a PR that adds support for the old engarde check and decorator names w/ a deprecation warning, which will either get merged tonight or when I get back from vacation in a few days. @fdosani and @drosengarten - do either of you need python 2 support? @drosengarten - a couple of the new features you wanted exist in bulwark already, and open to PRs for the others!

Hi,
I don't need python 2 support.
So should I be contributing to engarde (which I have made the changes
locally and just await corporate approval), or to Bulwark?
The changes include the ability to specify either a input variable, output,
or output in a list/tuple, the ability to disable the checks, and a check
based on df.eval.
On Fri, Aug 2, 2019 at 3:24 AM Zax<EMAIL_ADDRESS>wrote:

Always looking for more ideas and help! I have a PR that adds support for
the old engarde check and decorator names w/ a deprecation warning, which
will either get merged tonight or when I get back from vacation in a few
days. @fdosani https://github.com/fdosani and @drosengarten
https://github.com/drosengarten - do either of you need python 2
support? @drosengarten https://github.com/drosengarten - a couple of
the new features you wanted exist in bulwark already, and open to PRs for
the others!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/engarde-dev/engarde/issues/56?email_source=notifications&email_token=AANC4M6SFTJC6PMO5W7ZP4LQCOLGXA5CNFSM4GL6BDA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3MMMTA#issuecomment-517523020,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AANC4M7JWGEJS75T5TAIEG3QCOLGXANCNFSM4GL6BDAQ
.

--
Daniel J. Rosengarten, CFA
<EMAIL_ADDRESS>https://www.linkedin.com/in/fintechquantai/
http://www.linkedin.com/in/structuredcredit

@ZaxR Don’t need Python 2 support. 👍

The ability to disable checks already exists in Bulwark, so you’re good on that front. I’m open to a df.eval check, but that would require bumping up the minimum pandas version substantially I believe - I know of at least one eval bug that existed through pandas 0.22. Not exactly sure what you mean by the first feature, but again open to adding any helpful functionality. I’d suggest making issues/PRs to Bulwark at this point unless anyone has any objections. @tomaugspurger - does that sound good to you? If so, we can work on the rest of the details of the migration plan.

Yep, focusing on Bulwark seems like the best option. Thanks @Zax.
On Fri, Aug 2, 2019 at 8:13 AM Zax<EMAIL_ADDRESS>wrote:

The ability to disable checks already exists in Bulwark, so you’re good on
that front. I’m open to a df.eval check, but that would require bumping up
the minimum pandas version substantially I believe - I know of at least one
eval bug that existed through pandas 0.22. Not exactly sure what you mean
by the first feature, but again open to adding any helpful functionality.
I’d suggest making issues/PRs to Bulwark at this point unless anyone has
any objections. @TomAugspurger https://github.com/TomAugspurger - does
that sound good to you? If so, we can work on the rest of the details of
the migration plan.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/engarde-dev/engarde/issues/56?email_source=notifications&email_token=AAKAOISFZQ7VZH5RMFI5G5DQCQXHHA5CNFSM4GL6BDA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3NWRUA#issuecomment-517695696,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAKAOIRTGUWPH2HYGUWS7HLQCQXHHANCNFSM4GL6BDAQ
.

Just merged support for old engarde function names, which will be available in 0.5.0: https://github.com/ZaxR/bulwark/issues/34

I'd like to point people to Bulwark in the docs and archive this repo. Any objections? Any additional steps?

I just released v0.5.0 of Bulwark, which has support for Engarde function/decorator names where they were renamed in Bulwark. The Engarde version of the names will be removed in 0.7.0 (deprecation warnings notify users).

I'd like to point people to Bulwark in the docs and archive this repo. Any objections? Any additional steps?

Sounds good to me, and with v0.5.0 should be a pretty each migration for folks.

Sounds Great.  I'm glad the changes so far are being released. I've
received approval from my firm to be able to contribute.
On Sun, Aug 18, 2019 at 8:52 PM Zax<EMAIL_ADDRESS>wrote:

I just released v0.5.0 of Bulwark, which has support for Engarde
function/decorator names where they were renamed in Bulwark. The Engarde
version of the names will be removed in 0.7.0 (deprecation warnings notify
users).
I'd like to point people to Bulwark in the docs and archive this repo. Any
objections? Any additional steps?
Sounds good to me, and with v0.5.0 should be a pretty each migration for
folks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/engarde-dev/engarde/issues/56?email_source=notifications&email_token=AANC4M2WEYYWHZDH3FFIQW3QFGSAPA5CNFSM4GL6BDA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4RG4LI#issuecomment-522350125,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AANC4MYXGUHV4NXKSLMU73DQFGSAPANCNFSM4GL6BDAQ
.

--
Daniel J. Rosengarten, CFA
<EMAIL_ADDRESS>https://www.linkedin.com/in/fintechquantai/
http://www.linkedin.com/in/structuredcredit


dump at keys should output enrollmentId and apkam symmetric key

Is your feature request related to a problem? Please describe.
currently, dump_atkeys doesn't output enrollmentId and apkam symmetric key .
Describe the solution you'd like
Output enrollmentID and apkam_symmetric key if present
Describe alternatives you've considered
No response
Additional context
No response

PR Raised PR


S

Describe the bug
A clear and concise description of what the bug is and what you expected to happen.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Screenshots
If applicable, add screenshots to help explain your problem.
Device (please complete the following information):

iOS Version:
iOS Device:
unc0ver Version:

Place an "x" between the brackets if true:

[ ] this is a bug others will be able to reproduce
[ ] this issue is present with all tweaks uninstalled(except for default packages) or disabled
[ ] this issue is present after a rootfs restore
[ ] this issue is present on the latest version of unc0ver

Logs
If applicable, add logs or error messages here.

No.  Is there anything else you need help with today?  Ok thanks for coming by.


Illegal instruction when using floats

Pull docker image
docker run --rm -ti fedora:24 /bin/bash

Checkout and build
dnf groupinstall -y 'Development Tools'
# not everywhere 'which' is installed, but Makefile assumes it
dnf install -y llvm-devel gcc-c++ which zlib-devel ncurses-devel
git clone git://github.com/ponylang/ponyc.git
cd ponyc
make

The following example is using ponyc -d because of the issue #1225
mkdir minimal-cases/float-issue

Create the minimal-cases/float-issue/main.pony with the following content:
actor Main
  new create(env: Env) =>
    var f: Float = F64(0.0)
    env.out.print("f = " + f.string())

Build it and try to run:
build/release/ponyc -d minimal-cases/float-issue
./float-issue

It will aborts with the following message:
Illegal instruction

Backtrace:
#0  Main_tag_create_oo (this=0x7ffff6eca400, env=0x7ffff6ec9c00) at main.pony:3
#1  0x0000000000402848 in Main_Dispatch ()
#2  0x000000000040c32a in handle_message (ctx=0x7ffff6ecb848, actor=0x7ffff6eca400, msg=0x7ffff6ecb720) at src/libponyrt/actor/actor.c:95
#3  0x000000000040c4cd in ponyint_actor_run (ctx=0x7ffff6ecb848, actor=0x7ffff6eca400, batch=0x64) at src/libponyrt/actor/actor.c:154
#4  0x000000000040d33d in run (sched=0x7ffff6ecb800) at src/libponyrt/sched/scheduler.c:265
#5  0x000000000040d42f in run_thread (arg=0x7ffff6ecb800) at src/libponyrt/sched/scheduler.c:307
#6  0x00007ffff7bc45ca in start_thread (arg=0x7fffee6ca700) at pthread_create.c:333
#7  0x00007ffff71d9f6d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Disassemble:
Dump of assembler code for function Main_tag_create_oo:
   0x0000000000403e00 <+0>:         sub    $0x38,%rsp
   0x0000000000403e04 <+4>:         vxorps %xmm0,%xmm0,%xmm0
   0x0000000000403e08 <+8>:         mov    %rdi,0x28(%rsp)
   0x0000000000403e0d <+13>:        mov    %rsi,0x20(%rsp)
=> 0x0000000000403e12 <+18>:        vmovsd %xmm0,0x18(%rsp)
   0x0000000000403e1a <+26>:        vmovsd 0x18(%rsp),%xmm1
   0x0000000000403e22 <+34>:        mov    %rsi,0x10(%rsp)
   0x0000000000403e27 <+39>:        callq  0x4084a0 <F64_val_create_dd>
   0x0000000000403e2c <+44>:        movabs $0x41bd08,%rdi
   0x0000000000403e36 <+54>:        vmovsd %xmm0,0x30(%rsp)
   0x0000000000403e3c <+60>:        callq  0x4087a0 <FormatDefaultNumber_val_create_o>
   0x0000000000403e41 <+65>:        vmovsd 0x30(%rsp),%xmm0
   0x0000000000403e47 <+71>:        mov    %rax,%rdi
   0x0000000000403e4a <+74>:        callq  0x408450 <F64_val_string_oo>
   0x0000000000403e4f <+79>:        movabs $0x41bde0,%rdi
   0x0000000000403e59 <+89>:        mov    %rax,%rsi
   0x0000000000403e5c <+92>:        callq  0x4058f0 <String_val_add_oo>
   0x0000000000403e61 <+97>:        mov    0x10(%rsp),%rsi
   0x0000000000403e66 <+102>:       mov    0x18(%rsi),%rdi
   0x0000000000403e6a <+106>:       mov    %rax,%rsi
   0x0000000000403e6d <+109>:       callq  0x403630 <StdStream_tag_print_oo__send>
   0x0000000000403e72 <+114>:       mov    %rax,0x8(%rsp)
   0x0000000000403e77 <+119>:       add    $0x38,%rsp
   0x0000000000403e7b <+123>:       retq

The issue is reproducing under docker, as well as native x86_64 without any virtualization.

not repro on darwin x64_86, with latest release version.
0.3.2-722dc89 [release] (llvm 3.8.1)
otool -qtv -p _Main_tag_create_oo float-issue
_Main_tag_create_oo:
0000000100006510        subq    $0x48, %rsp
0000000100006514        movq    %rsi, 0x28(%rsp)
0000000100006519        movq    %rdi, 0x20(%rsp)
000000010000651e        callq   0x100013680
0000000100006523        vxorps  %xmm0, %xmm0, %xmm0
0000000100006527        movq    0x20(%rsp), %rsi
000000010000652c        movq    %rsi, 0x38(%rsp)
0000000100006531        movq    0x28(%rsp), %rdi
0000000100006536        movq    %rdi, 0x30(%rsp)
000000010000653b        vmovsd  %xmm0, 0x18(%rsp)
0000000100006541        vmovsd  0x18(%rsp), %xmm1       ## xmm1 = mem[0],zero
...

I've noticed that ponyc is using /usr/bin/cc, which is gcc 6.1.1 on Fedora 24. Is pony sensitive to the compiler?

Is your processor an Intel Skylake?

Yes, it is Skylake: i7-6820HQ

This is a bug in LLVM 3.8, where all Skylake processors are incorrectly assumed to support AVX-512 instructions. You can find a more detailed explanation in this issue. Until we implement a workaround in the compiler, you can fix the problem by passing --features=-avx512 to ponyc.

Passing --features=-avx512 does not help
$ ponyc -d --features=-avx512
Building builtin -> /home/avsej/.local/lib/pony/0.3.2-722dc89/packages/builtin
Building . -> /home/avsej/code/pony/ponyc/minimal-cases/float-issue
Generating
'-avx512' is not a recognized feature for this target (ignoring feature)
'-avx512' is not a recognized feature for this target (ignoring feature)
'-avx512' is not a recognized feature for this target (ignoring feature)
 Reachability
 Selector painting
 Data prototypes
 Data types
 Function prototypes
 Functions
 Descriptors
Writing ./float-issue.o
'-avx512' is not a recognized feature for this target (ignoring feature)
'-avx512' is not a recognized feature for this target (ignoring feature)
Linking ./float-issue
$ ./float-issue
Illegal instruction (core dumped)

Okay, llvm-3.9 works, and fixes #1225 as well.