Implement blue/green deployment support in REPL mode

Description
Features:

Add mutual exclusivity in session-based emr-s jobs to ensure only one job runs at a time, enhancing system stability during blue/green deployments. This allows active job exclusion and seamless task pickup by new jobs. Details in #94.

Fixes:

Resolve a bug where long-running queries failed to cancel post-index mapping verification, by introducing timely query cancellation checks within the REPL loop.
Address issue #138 with the proposed short-term fix, improving reliability.

Tests:

Conducted manual testing to validate blue/green deployment support and query cancellation.
Extended unit tests to cover new features and bug fixes.

Issues Resolved
https://github.com/opensearch-project/opensearch-spark/issues/138
https://github.com/opensearch-project/opensearch-spark/issues/94
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

could we make following value configurable. i found with 2 executors, some query can not finish by 10mins, but just in case user want to override it.
private val INACTIVITY_LIMIT_MILLIS = 30 * 60 * 1000 private val QUERY_EXECUTION_TIMEOUT = Duration(10, MINUTES) private val QUERY_WAIT_TIMEOUT_MILLIS = 10 * 60 * 1000

changed.


update discordjs permissions

one more ! SendVoiceMessages
14.11

oops! looks like you covered this base.


loadCells fail if ColumnStart and EndIndex are same

I want to load a column from a sheet. Following code fails with
TypeError: Cannot read property 'length' of undefined
 await masterSheet.loadCells({startRowIndex:1, endRowIndex:100, startColumnIndex:21, endColumnIndex:21})
If I use a bigger number for endColumnIndex, it works.
Version: "google-spreadsheet": "^3.1.15",
Detailed Error:
  at node_modules/google-spreadsheet/lib/GoogleSpreadsheetWorksheet.js:62:47
  at arrayEach (node_modules/lodash/lodash.js:530:11)
  at Function.forEach (node_modules/lodash/lodash.js:9410:14)
  at GoogleSpreadsheetWorksheet._fillCellData (node_modules/google-spreadsheet/lib/GoogleSpreadsheetWorksheet.js:58:7)
  at GoogleSpreadsheet._updateOrCreateSheet (node_modules/google-spreadsheet/lib/GoogleSpreadsheet.js:212:32)
  at node_modules/google-spreadsheet/lib/GoogleSpreadsheet.js:382:38
  at arrayEach (node_modules/lodash/lodash.js:530:11)
  at Function.forEach (node_modules/lodash/lodash.js:9410:14)
  at GoogleSpreadsheet.loadCells (node_modules/google-spreadsheet/lib/GoogleSpreadsheet.js:382:7)

These params are being passed through to google's API and that is how it works - see https://developers.google.com/sheets/api/reference/rest/v4/spreadsheets/other#GridRange


Dropout rate and other parameters are all set to 0

I think , these highlighted parameters must be set to certain values. For example , why the default value of maxDropPercentage is 0 ?   I know Resnet and Densenet has dropout layers and these layers should be set to a certain value but when i run analytics zoo only learning rate is set to 0,0005 because I already initialize it . Please can you explain that
learningRate: 5.0E-4
computeThresholdbatchSize: 100
maxDropPercentage: 0.0
learningRateDecay: 0.0
warmupIterationNum: 200
isLayerwiseScaled: false
dropPercentage: 0.0

No need to set these values for transfer learing.


Asynchronous path for document cleaning

For symmetry reasons introduced the async methods on the document cleaner API as well.

@jeremydmiller This is done from my point of view

@danielmarbach I should have shut this one down earlier. I think this is just unnecessary complexity and clutter in the codebase. I'd rather we didn't take this in. Document cleaner is strictly for testing.

Well then the API belongs into a different package. But even then, for example in our acceptance testing solutions we are purely async. The complexity argument: Well that is driven by the design decision to support sync and async overloads. So you either say: production lib is sync and async and then all APIs follow that guideline, which means the cleaner has to follow that rule as well OR you move it into separate package and come up with the rule where you say: testing code will not be async but sync only because of reasons.

Now that the API churn has settled down a bit, rebase off the latest master if you're still interested in getting this in. Sorry if this causes some inconvenience.


Issue with the installation of the whl data

Hello dear sir ,
I have a issue with the installation of the wheel data and it is really annoying. I saw on the internet page that there are 2 methods on how to install; either with the binary files or with the wheel data. Since I want to use it with python , I ran the command pip install whl_data. And the installation with pip was successful
But now when I want to use the very simple codeline for registration of the documentation, i get that error
'return_load(spec)'
'ImportError: DLL load failed: The specified module was not found'
This the code that i used
import SimpleITK as sitk
resultImage = sitk.Elastix(sitk.ReadImage("fixedImage.nii"), sitk.ReadImage("movingImage.nii"))
. Could you plese help me to solve that error?
Thanks!

Sorry for the no reply. Feel free to reopen if this is still an issue.


v1.3 bild extension

Please make a build to PHP7.3 nts x64 windows dll
Thanks!

Sorry, I do not have access to a Windows box. Building yourself should not be hard.

Sorry, but it is... Someone else who do it to version 1.2.1?

@gkralik Attached you can find the builds for Windows PHP version 7.3.8 and library version 1.3.0. You can include them in the release if you want. cc @stuli3
php_sapnwrfc-1.3.0-7.3.8-vc-15.zip

@avadaneidanut thanks, I'll add them in the next few days.

https://github.com/gkralik/php7-sapnwrfc/releases/tag/1.3.0


Zendesk test cannot be compiled to native with Camel 4 and Quarkus 3

fixes https://github.com/apache/camel-quarkus/issues/4514
Async-httpclient us upgraded to 3.0.0.Beta1 to support jakarta.
Synchronization between com.cloudbees.thirdparty:zendesk-java-client and ahc.version is disabled until a new client exist. Ticket to enable it back was created https://github.com/apache/camel-quarkus/issues/4591.

Nice @JiriOndrusek !


Enable cargo doc --open tests on macos.

Now that #7576 is merged, these tests should work fine on macos.

r? @Eh2406
(rust_highfive has picked a reviewer for you, use r? to override)

@bors: r+

:pushpin: Commit 517b59dbb4f19246da2fa6ab995a132537a1d282 has been approved by alexcrichton

:hourglass: Testing commit 517b59dbb4f19246da2fa6ab995a132537a1d282 with merge 70af90444b8cd2c3ef06f34dd193769cddf916cf...

:sunny: Test successful - checks-azure
Approved by: alexcrichton
Pushing 70af90444b8cd2c3ef06f34dd193769cddf916cf to master...


Add CMM Safe app

This is a placeholder issue.
The CMM team plans to have this ready until EOD Dec 3.
They expect a deploy then a few days after.
The CMM team does QA on this. So for us only basic review and smoke testing.
The CMM app should only be added to the xDai version.

Dani added the CMM app today


WIP: Test Plantuml diagram generation

This PR is for now just a playground to get PlantUML/graphviz diagram generation tested on our CI systems.

Woot!
Didn't think that it would run that smooth.
PlantUML tests now seem to run fine on Win and Linux.

Didn't think that it would run that smooth.

I hope that was more a matter of mistrust in the tools than in your skills :wink:

Hehe... Thank you!
Was more wondering if the binaries are available on the PATH and that it is available to the tests.

:+1:


Slow clearing of existing features for large feature spaces using PostgreSQL

Currently, clearing large feature sets can be very slow. For example, say I have a database with >375K feature_key entries built over a candidate set. I then want to then wipe that set and create features over a different set of candidates. Running the standard feature extraction code via the apply function hangs for hours on Clearing existing... Running DELETE FROM feature_key; directly similarly takes forever.
So are there some implicit CASCADE DELETE operations going on here?
In general, DB operations involving inserting feature key/value pairs are very slow in a default install of PostgreSQL.
Is there a better way to address with this?

Some more info would help narrow down whether it's a Postgres thing or a Snorkel thing.
Can you run EXPLAIN ANALYZE VERBOSE DELETE FROM feature_key (when the database is full again) and post the results?

Hopefully, I can add some insight to this problem. I actually have a similar issue except the database I'm working with has ~4M rows. Tried to run the above statement but that was hanging for a while, so instead I deleted only a 1000 rows. Here is the output:

Delete on public.feature_key  (cost=34.93..8290.00 rows=1000 width=34) (actual time=10.308..10.308 rows=0 loops=1)
->  Nested Loop  (cost=34.93..8290.00 rows=1000 width=34) (actual time=1.786..8.168 rows=1000 loops=1)
Output: feature_key.ctid, "ANY_subquery".*
->  HashAggregate  (cost=34.50..44.50 rows=1000 width=32) (actual time=1.762..2.272 rows=1000 loops=1)
Output: "ANY_subquery"., "ANY_subquery".id
Group Key: "ANY_subquery".id
->  Subquery Scan on "ANY_subquery"  (cost=0.00..32.00 rows=1000 width=32) (actual time=0.026..1.228 rows=1000 loops=1)
Output: "ANY_subquery"., "ANY_subquery".id
->  Limit  (cost=0.00..22.00 rows=1000 width=4) (actual time=0.020..0.727 rows=1000 loops=1)
Output: feature_key_1.id
->  Seq Scan on public.feature_key feature_key_1  (cost=0.00..89091.21 rows=4050121 width=4) (actual time=0.017..0.530 rows=1000 loops=1)
Output: feature_key_1.id
->  Index Scan using feature_key_pkey on public.feature_key  (cost=0.43..8.24 rows=1 width=10) (actual time=0.005..0.005 rows=1 loops=1000)
Output: feature_key.ctid, feature_key.id
Index Cond: (feature_key.id = "ANY_subquery".id)
Planning time: 0.530 ms
Trigger RI_ConstraintTrigger_a_6869322 for constraint feature_key_id_fkey: time=156648.600 calls=1000
Execution time: 156659.353 ms
(18 rows)

There might be useful info in #976, which adds indices to dramatically speed up clearing documents and candidates. We didn't actually touch the feature or feature key classes though because the annotation and annotation key mixin classes use all their foreign keys as primary keys.
I'm still not sure why that sequential scan is in the middle of the query plan...

Closing this for now, feel free to re-open!


Arbitrary load for transport, root view controller fix

Hi Michael,
builds with recent SDKs complain about missing root view controller
and I've added the arbitrary loads for transport security on iOS 9 , which make sense for an rss parser IMHO
( + leftover testing feed )

seems like a bad call, PRs are not being merged for some time :-(


Post Script to remove certain entries from extra field

I was struggling to remove the ZSCC key: value pairs (left over) in the extra field of some of my entries preserving possible other extras. The filter that BBT offers does not work for contents of the extra field - one can just prohibit the export of the whole field (note) there. Thus I created my first post script... which was also not trivial in the beginning

there is no example using the reference.remove function
there is no explanation for the duplicate parameter of the add function and I could not figure it out form the source code either (not a big fan of JS anyway but I really tried).

In case anybody wonders, the following two scripts seem to be equivalent.
if (Translator.BetterTeX && item.extra) {
    reference.add({ name: 'note', duplicate: true, value: item.extra.replace(/(^ZSCC: .*$)/gm, '') });
}

if (Translator.BetterTeX && item.extra) {
    reference.remove('note');
    reference.add({ name: 'note', value: item.extra.replace(/(^ZSCC: .*$)/gm, '') });
}

Feel free to copy them to the example page.

Wouldn't it make sense to filter ZSCC extras out by default?
I have seen there exists some code that parses extraFields already (e.g, csl)... What do you think about enabling the filtering of such "subfields" within the extra field? I mean by adding some code or even another text box to the export filter settings?

Postscripts are not really well documented, and users are not necessarily expected to become javascript gurus -- most samples on that page came about because someone requested a postscript and I wrote it. It's always OK to ask.
reference.remove removes fields from the bibtex entry. Since extra is not a bibtex entry, reference.remove won't do much useful.
The duplicate: parameter was a holdover from the early days of postscripts, it doesn't do anything now (but neither will it hurt to have it there). I've updated the sample in the docs, but that will take anywhere between 5 and 30 minutes to show up.
The two scripts are equivalent (and you don't need the replace marker anymore -- replace in postscripts is implied now. ^ZSCC will only work if that appears at the start of the extra field, I'd suggest
if (Translator.BetterTeX && item.extra) {
    reference.add({ name: 'note', value: item.extra.split('\n').filter(line => !line.startsWith('ZSCC:').join('\n') })
}

instead.
WRT filtering out -- I currently remove known Zotero fields, known CSL variables, and anything that starts with tex./bibtex./biblatex., but I don't think ZSCC falls under one of these categories. People do also use the extra field as a note field so I want to be conservative about what I remove. What does ZSCC stand for?
the code that parses extraFields is special because it knows about the type of the fields it parses (such as creator-type and date-type CSL variables), I wouldn't know any special type handling for generic fields, and then extraFields parsing is literally just what you see above -- split into lines, look for line starts, and filter them out.

Hm, right... it works without the duplicate parameter and without the remove call - I did not test it well enough in that respect I guess. But I think you missed the m (multiline) flag I was using in the replace regex(?) With that it works fine no matter in which line the ZSCC is written (as long as it starts at the beginning of a line). Do I miss something else?
The ZSCC entries come from the ZSC plugin: https://github.com/MaxKuehn/zotero-scholar-citations
I am currently not using it anymore because it has too many problems for the benefits I perceive.

You are right, I missed that. But I can't actively track what all Zotero plugins put there -- the tex fields are my own, and Zotero already parses out it's own and CSL fields, and I follow those.

I understand... chasing after such things is almost always a lost race. That's why I was thinking that allowing to filter them manually in the settings like already possible for whole fields would make sense. I am happy with the script solution though - feel free to close this issue. Thanks!

Welcome.

After some more testing, I think the remove is actually required in case the extra field would become empty. I presume this is because the add call to change it would silently fail with an empty value(?) but I havent checked the code or tried to really understand it (or why I thought it worked before...). Eventually, I ended up with, which also removes the trailing line break itself if need be:
if (Translator.BetterTeX && item.extra) {
    reference.remove('note');
    reference.add({ name: 'note', value: item.extra.replace(/^ZSCC:\s+\S+[\s]*/mg, '') });
}

:robot: this is your friendly neighborhood build bot announcing test build <IP_ADDRESS>88 ("clear field in postscript when adding an empty value")
Install in Zotero by downloading test build <IP_ADDRESS>88, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

Good point. I'm of two minds on this; usually I prefer to have .add just do nothing if the value is empty, but maybe in the postscript an unconditional overwrite is more appropriate. 5888 adds this behavior, but I'm still not sure.


Get child nodes from NCBI taxonomy

Use case is fairly simple:
Given a group (e.g. vertebrates) return all child taxids.
There are a couple biostars posts.
One suggests making a sqlite db sim to @jpb initial suggestion
Second points to this set of tools:
http://etetoolkit.org/docs/2.3/tutorial/tutorial_ncbitaxonomy.html

This is a start:  https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=taxonomy&term=vertebrata[orgn]&retmax=1000000

Fleshed out...
esearch -db taxonomy -query "vertebrata[orgn]" | efetch -db taxonomy -format docsum | xtract -pattern DocumentSummary -if Rank -equals family -element Id,Division,ScientificName,CommonName | more
there are a couple other options, which I will add tomorrow

Theres also a script on github which should work.  Instructions:
https://github.com/aaschaffer/vecscreen_plus_taxonomy
Notes:
The relevant code is in scripts/assign_levels_to_taxonomy.pl
The intended usage of assign_levels_to_taxonomy.pl is described in README
Part of the purpose of the full script is idiosyncratic to the way in which the taxonomy handles uncultured bacteria, but the code in assign_levels_to_taxonomy.pl is modular and likely to have procedures you want
The work is published, currently as a preprint: PMID 29069347, please cite it if you use the code in a publication.

From an anonymous source:
esearch -db taxonomy -query "vertebrata[orgn]"
will return extra undesirable nodes. The problem is that there is a ‘Vertebrata’ genus in Algae. The correct query would be
esearch -db taxonomy -query "vertebrates[next level]"


no-tree shake compiler mode

To make sure to compile all code at build/release time.

The option was already there, -F allFunctions -F allPrototypes
I added allcompile.ts that imports everything; it is now compiled with the right options on yarn test


Update main.js to fix undefined characterLimit variable

Summary of PR changes
Replaced the characterLimit variable with the variable commentCharacterLimit, which was previously defined in code.
This is a potential fix for this issue: https://github.com/im-open/process-dotnet-test-results/issues/28
PR Requirements

[x] For major, minor, or breaking changes, at least one of the commit messages contains the appropriate +semver: keywords.

See the Incrementing the Version section of the repository's README.md for more details.

[x] The action code does not contain sensitive information.

NOTE: If the repo's workflow could not automatically update the README.md, it should be updated manually with the next version.  For javascript actions, if the repo's workflow could not automatically recompile the action it should also be updated manually as part of the PR.

Thank you for this fix.  I'll merge it into a secondary branch then merge it to main shortly.

The new version is v2.3.2


⚠ Bofrost.at has degraded performance

In 3df41ee, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1425 ms

Resolved: Bofrost.at performance has improved in d9edc5d.


Run the Employee Onboarding Sample Scenario

Tutorial URL: https://developers.sap.com/tutorials/cp-starter-ibpm-employeeonboarding-5-run.html
Please specify the step you are referring to
I get the following error @ step 5 ->
Go back to the SAP Fiori launchpad homepage. The My Inbox tile shows one open task.
The OAuth token request to URL 'https://i034941trial.authentication.eu10.hana.ondemand.com?grant_type=client_credentials' failed with HTTP status code 403 (Forbidden) and response body '{"error":"Could not verify the provided CSRF token because your session was not found."}'
Log ID: ea7955a2-8455-455e-98c2-888af27bca9d
Determine EquipmentDec 13, 2019, 9:35:24 AM

your session would have expired. Please login into incognito mode and try again.

I had already tried this. I get the same error in incognito mode too.

(a) Can you open the Monitor Instances and send the error from execution log. (b) Hope you have given all the needed roles to your users from the setup section and also deployed the business rules successfully in your tenant.

I am not able to upload an image here....Will copy paste the execution log here :
"Determine Equipment" failed
Dec 18, 2019, 11:01:53 AM
Service endpoint:
BUSINESS_RULES
Error message:
The OAuth token request to URL 'https://i034941trial.authentication.eu10.hana.ondemand.com?grant_type=client_credentials' failed with HTTP status code 403 (Forbidden) and response body '{"error":"Could not verify the provided CSRF token because your session was not found."}'
"Determine Equipment" started
Dec 18, 2019, 11:01:14 AM
"Prepare Rules Input" completed successfully
Dec 18, 2019, 11:01:14 AM
"Prepare Rules Input" started
Dec 18, 2019, 11:01:14 AM
"Process Buddy List" completed successfully
Dec 18, 2019, 11:01:12 AM
"Process Buddy List" started
Dec 18, 2019, 11:01:12 AM
<EMAIL_ADDRESS>started the workflow
Dec 18, 2019, 11:01:11 AM

Were you able to go through this step?

I still get the same error
"Determine Equipment" failed
Jan 3, 2020, 2:27:26 PM
Service endpoint:BUSINESS_RULES
Error message:The OAuth token request to URL 'https://i034941trial.authentication.eu10.hana.ondemand.com?grant_type=client_credentials' failed with HTTP status code 403 (Forbidden) and response body '{"error":"Could not verify the provided CSRF token because your session was not found."}'

Please close the issue, as this seem to have resolved.
Reason: The token URL was not entered correctly by while modelling the destination


 I've tried restarting the test, but same result.

@khaeru it looks strange, based on exception we can conclude that scenario was not saved to DB (run id equals -1) when it calls checkIn method (as final step of commit call, when scenario should be already committed). In principle that is only possible when something went wrong committing scenario and failed in exception block, but then we should see message There was a problem writing data to the database with stack trace in the output. Could it be that pytest somehow hides/truncates the output?

Update jar. Let's see what the output will be now

Closing in favour of #289.


Use "native" types when possible

Problem
Currently when passing arrays between Python and Julia, then the code always uses a wrapper object (PyArray, ArrayValue, etc.), even if the data happens to be contiguous in memory. This is a problem because not all code works well (or at all) with these wrappers. In an ideal world with proper interfaces, this wouldn't have been a problem, but both Python and Julia are somewhat lax about strict interfaces for arrays, so in practice, things sometimes break (or just run slowly).
As just one trivial example, this will crash:
# Julia

function give_vector()::Vector
    return vec([1.0 2.0 3.0])
end

# Python

vector = in_julia.give_vector()
assert vector.__class__.__name__ == "VectorValue"
assert str(vector) == "[1.0, 2.0, 3.0]"
vector += 1

Because TypeError: unsupported operand type(s) for +=: 'VectorValue' and 'int'.
The point isn't about this specific missing operation (though fixing it would be nice); the point is that try as we may, we'll never make VectorValue be a 100% drop-in replacement to ndarray.
Solution
When converting arrays between Julia and Python, if the data is contiguous, then use frombuffer to wrap the memory with an ndarray for Python, or use jl_ptr_to_array to wrap the memory with an Array for Julia to use. If, however, the data is not contiguous in memory, keep the current behavior of returning a wrapper type.
Alternatives
This can be done manually of course, which is probably what I will be doing in my code for now. That said, even if this wasn't the default behavior, it would be nice to provide some convenience functions to make it easier to achieve.
Additional context
Here is some example code which worked for me to demonstrate the feasibility of the solution:
# Julia

function as_ndarray(array::Array)::PyArray
    np = pyimport("numpy")
    return PyArray(np.frombuffer(array))
end

function from_ndarray(ndarray::PyArray)::Array
    return unsafe_wrap(Array, pointer(ndarray), size(ndarray))
end

function give_numpy_vector()::PyVector
    return as_ndarray(give_vector())
end

function modify_vector(vector::AbstractVector)::Nothing
    vector[1] += 10
    return nothing
end

function modify_vector(vector::PyVector)::Nothing
    modify_vector(from_ndarray(vector))
    return nothing
end

# Python

vector = in_julia.give_vector()
assert vector.__class__.__name__ == "VectorValue"
assert str(vector) == "[1.0, 2.0, 3.0]"
in_julia.modify_vector(vector)
assert str(vector) == "[11.0, 2.0, 3.0]"

vector = in_julia.give_numpy_vector()
assert vector.__class__.__name__ == "ndarray"
assert str(vector) == "[1. 2. 3.]"
in_julia.modify_vector(vector)
assert str(vector) == "[11.  2.   3.]"

Naturally this is just a proof of concept and doesn't deal with issues such as proper mapping of dtype, testing whether the data is actually contiguous, etc.
One point woth noting is that the fact that Array "likes" to be column-major and ndarray "likes" to be row-major is not a show stopper here. I have plenty of Python code which explicitly works with column-major arrays (because that's what needed for efficiency), and Julia has PermutedDimsArray  which can likewise deal with row-major order. It is always the developer's responsibility to deal with this issue (summing rows of column-major data will be much slower than summing rows of the same data in rows-major order).
As a side note, the built-in implementation in both numpy and Julia for converting data between these layouts is slow as molasses for no good reason, so I had to provide my own implementation to get reasonable performance. But that's mostly irrelevant to the issue I raised here.

When passing an array from Julia to Python, if you want it to be a numpy array, you can simply do numpy.asarray(the_array). This will be a non-copying view of the original array where possible (i.e. the underlying array supports pointer and strides and the element type corresponds to a numpy dtype).
PythonCall/JuliaCall aims to be agnostic to particular Python packages like numpy, so automatically converting to ndarray is not preferred. Instead, we rely on interfaces. In particular, a wrapped Julia array will satisfy both the Python buffer protocol and the Numpy array interface - this is what enables the no-copy conversion above with numpy.asarray. Furthermore a wrapped Julia vector will satisfy the Python sequence interface, so it behaves a lot like a list. It intentionally does not provide anything else, because there is no standard for what else an array should support - so if you want the ndarray interface then you wrap the object with numpy.asarray yourself.
Conversely, Python arrays become PyArray which is not much more than a pointer to the data. It should be as fast as Array - if you find any cases where it is slower then please raise an issue. PythonCall does not use unsafe_wrap(Array, ...) because there is no way to ensure the data outlives the resulting Array - i.e. the numpy array could get garbage-collected, freeing the underlying memory, leading to a segfault when accessing the Array. But you are free to use unsafe_wrap(Array, ...) yourself if you are careful.

Thanks for the clarifications!
I assume the numpy.asarray could be on either the Julia or Python side?
And I take it that PyArray is therefore a DenseArray and a strided array, and would work well with Strided.jl and similar packages as well as Array does?
It would be nice if these points were mentioned in https://cjdoris.github.io/PythonCall.jl/stable/compat/

I get your point about not depending on specific Python packages, and I see how calling nump.asarray is a good solution when passing Julia arrays to Python - one gets a "real" ndarray and gets all the optimizations that follow.
But it seems there's no equivalent in the other direction. Getting data from Python wraps things in a PyArray which is not a "real" Array. Since PyArray is a general-case wrapper it isn't even a DenseArray even when it could be, which means losing performance when using it.
Here's a concrete example of the issue:
using BenchmarkTools
using Strided
using PythonCall
a = rand(4000, 4000);
pa = PyArray(a);
b = similar(a);
pb = PyArray(b);
@btime b .= (a .+ a) ./ 2;
@btime pb .= (a .+ a) ./ 2;
@btime @strided b .= (a .+ a) ./ 2;
@btime @strided pb .= (pa .+ pa) ./ 2;

Gives  me the results:
julia> using BenchmarkTools

julia> using Strided

julia> using PythonCall

julia> a = rand(4000, 4000);

julia> pa = PyArray(a);

julia> b = similar(a);

julia> pb = PyArray(b);

julia> @btime b .= (a .+ a) ./ 2;
  18.413 ms (4 allocations: 128 bytes)

julia> @btime pb .= (a .+ a) ./ 2;
  18.694 ms (4 allocations: 128 bytes)

julia> @btime @strided b .= (a .+ a) ./ 2;
  8.684 ms (17 allocations: 1.06 KiB)

julia> @btime @strided pb .= (pa .+ pa) ./ 2;
ERROR: MethodError: no method matching StridedView(::PyArray{Float64, 2, true, true, Float64})

One way to solve it is to keep PyArray for the general case, and create a new PyDenseArray to use only in the common case that the memory layout allows for it. Since PyDenseArray will be a subtype of  DenseArray, any Julia optimizations that kick in for DenseArray will be usable for it (e.g. Strided.jl).

A PyArray satisfies the Julia Base notion of being a strided array, namely it supports strides and unsafe_convert(Ptr{T}, ::PyArray{T}):
julia> using PythonCall

julia> @py import numpy as np

julia> x = np.random.randn(3,4)
Python:
array([[-0.65880647,  0.99940984, -0.45758152,  1.60004262],
       [ 0.82161821, -1.41043098,  0.62529857,  0.37647898],
       [ 0.10345882,  0.37506426,  0.12554197, -1.03708004]])

julia> y = PyArray(x)
3×4 PyArray{Float64, 2}:
 -0.658806   0.99941   -0.457582   1.60004
  0.821618  -1.41043    0.625299   0.376479
  0.103459   0.375064   0.125542  -1.03708

julia> pointer(y)
Ptr{Float64} @0x000002817d41eac0

julia> Base.unsafe_convert(Ptr{Float64}, y)
Ptr{Float64} @0x000002817d41eac0

julia> strides(y)
(4, 1)

I'm not familiar with Strided.jl, but maybe there is some integration work to do to make PyArray work with it.
PyArray is not a dense array in general - the data is strided but not necessarily contiguous.

One way to solve it is to keep PyArray for the general case, and create a new PyDenseArray to use only in the common case that the memory layout allows for it.

That would increase complexity quite a bit! Presumably Strides.jl relies on some trait somewhere that we could overload. Maybe in ArrayInterface.jl?

I assume the numpy.asarray could be on either the Julia or Python side?

Yep.

Yes, they do have a trait they call "StridedView", which in theory could be implemented for PyArray. That would be nice but is is just one example of one library. If, on the other hand, there was a PyDenseArray in the (extremely common case) that the data is actually dense (the strides work out), then this would work for "everything" that takes advantage of the standard DenseArray trait (restricted though this may be).
As far as complexity goes, both implementation of PyArray and PyDenseArray would be "mostly identical" so perhaps some creative use of include or something would keep the code DRY?

Another implementation strategy, hopefully simpler: since DenseArray is an abstract type, it should be possible to write DenseView which has an Array field (using jl_ptr_to_array), as well as a second field of type Any to keep whatever-is-holding-the-data-for-gc. Declare DenseView to be a DenseArray, and forward the relevant methods. Then it would just be a matter of wrapping PyArray with a DenseView (obviously, only if the strides work out). "In theory", this should allow anything that expects a DenseArray to "just work"...?

I'm getting good results with the following. I can use it to wrap a PyArray and get a DenseArray  for which all sort of optimizations kick in. Since the general problem is not specific to PythonCall, I'm closing this issue. Right now this is buried inside my (as yet unpublished) package - I'm not sure this deserves its own package...?
module AsDense

using LinearAlgebra
using Base: @propagate_inbounds

export as_dense_if_possible
export as_dense_or_copy
export as_dense_or_fail

struct DenseView{T, N} <: DenseArray{T, N}
    parent::AbstractArray{T, N}
end

@inline Base.IndexStyle(::Type{<:DenseView}) = Base.IndexLinear()
@inline Base.length(dense_view::DenseView) = length(dense_view.parent)
@inline Base.size(dense_view::DenseView) = size(dense_view.parent)
@inline Base.strides(dense_view::DenseView) = strides(dense_view.parent)
@inline function Base.stride(dense_view::DenseView, axis::Integer) = stride(dense_view.parent, axis)
@inline function Base.elsize(dense_view::Type{<:DenseView{T}})::Any where {T} = Base.isbitstype(T) ? sizeof(T) : (Base.isbitsunion(T) ? Base.bitsunionsize(T) : sizeof(Ptr))
@inline function Base.parent(dense_view::DenseView) = dense_view.parent
@propagate_inbounds @inline function Base.getindex(dense_view::DenseView, indices::Integer...) = getindex(dense_view.parent, indices...)
@propagate_inbounds @inline function Base.setindex!(dense_view::DenseView, value::Any, indices::Integer...) = setindex!(dense_view.parent, value, indices...)
@propagate_inbounds @inline function Base.view(dense_view::DenseView, indices::Integer...) = as_dense_if_possible(view(dense_view.parent, indices...))
@inline function Base.similar(dense_view::DenseView, dimensions::Integer...) = as_dense_if_possible(similar(dense_view.parent, dimensions...))
@inline function Base.similar(dense_view::DenseView, type::Type, dimensions::Integer...) = as_dense_if_possible(similar(dense_view.parent, type, dimensions...))
@inline function Base.reshape(dense_view::DenseView, dimensions::Integer...) = as_dense_if_possible(reshape(dense_view.parent, dimensions...))
@inline function LinearAlgebra.transpose(dense_view::DenseView) = DenseView(transpose(dense_view.parent))

"""
    function as_dense_if_possible(array::AbstractArray{T, N})::AbstractArray{T, N} where {T, N}

Given any `array`, return a `DenseArray` sharing the same memory if possible, or the original `array` otherwise.

This will return any `DenseArray` as-is without creating a wrapper.
"""
function as_dense_if_possible(array::AbstractArray{T, N})::AbstractArray{T, N} where {T, N}
    if array isa DenseArray
        return array
    end
    try
        array_strides = strides(array)
        array_sizes = size(array)
        if (array_strides[1] == 1 && array_strides[2] == array_sizes[1]) ||
           (array_strides[1] == array_sizes[2] && array_strides[2] == 1)  # only seems untested
            return DenseView(array)
        else
            return array  # untested
        end
    catch MethodError  # only seems untested
        return array
    end
end

"""
    function as_dense_or_copy(array::AbstractArray{T, N})::DenseArray{T, N} where {T, N}

Given any `array`, return a `DenseArray` sharing the same memory if possible, or a dense `Array` copy of the data.

This will return any `DenseArray` as-is without creating a wrapper or doing any copying.
"""
function as_dense_or_copy(array::AbstractArray{T, N})::DenseArray{T, N} where {T, N}
    as_dense = as_dense_if_possible(array)
    if as_dense isa DenseArray
        return as_dense
    else
        return Array(as_dense)
    end
end

"""
    function as_dense_or_fail(array::AbstractArray{T, N})::DenseArray{T, N} where {T, N}

Given any `array`, return a `DenseArray` sharing the same memory if possible, or fail with an error message.

This will return any `DenseArray` as-is without creating a wrapper or doing any copying.
"""
function as_dense_or_fail(array::AbstractArray{T, N})::DenseArray{T, N} where {T, N}
    as_dense = as_dense_if_possible(array)
    if as_dense isa DenseArray
        return as_dense
    else
        error("the array: $(typeof(array)) is not dense")
    end
end

end # module

That DenseView would be a nice think to package up.


Meeting: Discuss consistent codes for client acronyms

Description
Originated: During Sprint demo prep when Divya was demoing the review and edit request functionality. Ministry issue is related to the code which is there in FOI-Requests(Angular App) and FOI-FLOW. we used this file shared by Matt (IAO Client Acronyms.docx) for our master data. Here bc gov acronym is AFF and FOI-Requests(Angular App) has the code AGRI. So there is no matching code between FOI-Requests and FOI-Flow. That is why it was not selected in our Review Request page. Let's have a meeting to discuss
DOD

[x] Meeting to determine what acronyms we will be using
[ ]
[ ]
[ ]
[ ]

Plans has been made to use BCGOV code for IAO for the other organizations like Liquor Distribution Board etc. Also diff. in the code will be fixed in  FOI request app(Angular) till test env. and make it similar with all environments.  The change will go together into prod for all apps(FOI Flow, FOI requests etc.)
@divyav-aot @lmullane @m-prodan @acoard-aot @arielleandrews @JHarrietha-AOT @sumathi-thirumani-aot @mpilchar


Remaining things for 1.10 port

Because my memory isn't good enough to just remember. I just went through the readme.
TODO

[x] Bioplastic
[x] Burlap
[x] Composting

[x] Composter
[x] Rotten Plants
[x] Compost: Hard to tell whether it actually increases the growth rate. @squeek502 how did you initially test this?
[x] Fertilizer

[x] Doll's Eye: Double check mining drop
[x] Egg Replacers: Issue: Apple Sauce and Starch not used in recipes

[x] Apple Suace
[x] Starch

[x] Piston crafting

[x] Raw Ender: Issue: No texture in WAILA tip
[x] Faux Feather: Issue: Faux Feather not used in recipes
[x] Fossils: Double check mining drop
[x] Frozen Bubble
[x] Saltpeter: Double check mining drop
[x] Sulfur: Double check mining drop
[x] Vegetable Oil Ink
[x] Jute: Double check mining drop
[x] Kapok: Double check mining drop
[x] Blank Mob Head
[x] Plant Milk
[x] Proof of Suffering: Issue: Double check mining drop
[x] Resin: Issue: Double check mining drop
[x] Soap
[x] Straw Bed
[x] False Morel: Issues: Double check mining drop, items not used in recipes
[x] Vegetable Oil: Issue: Double check mining drop
[x] Seitan
[ ] JEI integration

[ ] Make fluid containers and fluids have the same recipes/usages

[ ] Bioplastic as a TiCo tool material
[ ] Frozen Bubbles in Fluid Transposer for Thermal Expansion
[ ] Potato Starch made in Pulverizer for Thermal Expansion
[ ] WAILA
[ ] BBQ Sauce and BBQ Tofu
[ ] Fluid container registry deprecated, need to convert to new system

Issues found

[x] MissingVariantException for washableWheat#inventory
[x] MissingVariantException for enderRift#inventory
[x] FileNotFoundException enderRift.json
[x] MissingVariantException for enderRift#normal

Untested:

Things that have "double check mining drop" as their note
All integration (including BBQ)

Thank you for making this list. 👍

@squeek502 For the JEI descriptions, should we migrate to the system already added by JEI, or should we reimplement it using the old code from the NEI TextHandler class? Using the JEI system would be significantly less code, and would have less room for error since so many people are already using it, but the text processing is significantly less powerful. Basically all formatting stuff would have to be rewritten. For example, from the lang file
example.string=Nested string example
item.VeganOption.exampleItem.name=Example
item.VeganOption.exampleItem.nei.usage=%1$s is nothing like [[minecraft:wheat]] or [[minecraft:golden_apple:1]], but it can do a {example.string}.\n\nLines broken.

would be
item.VeganOption.exampleItem.nei.usage=Example is nothing like Wheat or Golden Apple, but it can do a Nested string example.\n\nLines broken.

Either way, it the valid items would be determined by doing
		for (Item item : Item.REGISTRY)
		{
			if (item.getRegistryName().getResourceDomain().equals(ModInfo.MODID_LOWER))
			{
				if (item.getHasSubtypes())
				{
					List<ItemStack> subItems = new ArrayList<ItemStack>();
					item.getSubItems(item, null, subItems);
					for (ItemStack subItem : subItems)
					{
						String descKey = subItem.getUnlocalizedName() + ".nei.usage";
						if (LangHelper.existsRaw(descKey))
							// Register the itemstack somehow depending on the approach taken
					}
				}
				else
				{
					ItemStack toAdd = new ItemStack(item);
					String descKey = toAdd.getUnlocalizedName() + ".nei.usage";
					if (LangHelper.existsRaw(descKey))
						// Register the itemstack somehow depending on the approach taken
				}
			}
		}

I've done it both ways and that code works well.

@elifoster see https://github.com/squeek502/VeganOption/commit/160e12b9e51b3a469df7f872b547f08468a77c80

I provided a review. :+1:ing the stuff you agree with would be appreciative—I can work on the stuff I brought up no problem.
As for the fluid thing, I'm not sure I quite understand the problem. What is wrong with it being looked up using FluidStacks?

Replied to your comments, will commit something that addresses them in a bit.

As for the fluid thing, I'm not sure I quite understand the problem. What is wrong with it being looked up using FluidStacks?

The issue is currently that if you press R or U on any of the fluid blocks in JEI (Raw Ender, for example), nothing comes up. Raw Ender should have a Crafting description that shows up, saying that it's the byproduct of an Ender Rift (note: if you do usage of Ender Rift, Raw Ender shows up as a byproduct, so it's not a symmetrical bug).

Regarding "Make fluid containers and fluids have the same recipes/usages," it might be better to simply use the fluid containers for the recipes/usages, and then to blacklist the fluid blocks from showing up at all. What do you think?

it might be better to simply use the fluid containers for the recipes/usages, and then to blacklist the fluid blocks from showing up at all in JEI

This is complicated by the fact that certain things will output/use contained fluids, and certain things will output/use fluid blocks from the world. So, if everything shows up as a contained fluid, that divide will be confusing (for example, piston crafting showing input/output of bottled fluid, when that is not how it works).
It might be possible to get around this, though, by making what shows up in JEI independent of what gets set as the input/output in the Wrapper implementations (e.g. IRecipeWrapper.getIngredients sets the bottled fluid as the input, but the category implementation sets the slot to the fluid block--this is kind of what https://github.com/squeek502/VeganOption/commit/036f5ab1438a31b8ddf39be1d0f30e84063493f3 did). I'll look into this more and see what's possible.

Putting this here (copied from IRC) just so I don't forget about it:

squeek> @SatanicSanta, looks like there's still some stuff left that needs to be done with fluid containers
 VO's bottles/buckets dont interact with the basin anymore
 prob need to either update bottles to the capacity system or put backwards compatibility in the basin code
 or both

Also, with the description stuff working properly, the "Make fluid containers and fluids have the same recipes/usages" thing is not as important, since I believe all bottled fluids link to their fluid blocks with the byproducts/byproduct of sections. I think we should focus on the few remaining things and hold off on this bit (will make a separate issue for it and remove it from this checklist).

Maybe spoke too soon about closing this. Going to re-open and use it to track a few remaining things:

[ ] Frozen Bubbles don't get filled properly from Raw Ender in the world (a full bucket of Raw Ender only fills a bubble 1/8 of the way)
[ ] Stacks of empty containers don't work when right clicked on a basin they could be filled from, and multiple empty containers as an EntityItem don't get filled if they are inside a basin they could be filled from
[ ] Stacks of filled containers don't get consumed properly when used to fill something like a basin (the stack size doesn't change)


cherry-pick fix data correctness risk when load delete sign data into a table with sequence col (#32574)

Proposed changes
Issue Number: close #xxx
cherry-pick #32574

run buildall


Reserve WhoAmI for StepperDriver

All the devices in the harp-tech repo have the authorship and copyright as harp-tech. Shouldn't this one be the same? I'm ok either way, just to clarify for the future!

All the devices in the harp-tech repo have the authorship and copyright as harp-tech. Shouldn't this one be the same? I'm ok either way, just to clarify for the future!

I don't have a problem with either so long as the license is permissive.


[RS / SC / Semi-Remote] Senior iOS Developer

Empresa
DevGrid is a Full Stack software development company based in the United Kingdom with two delivery hubs in Brazil. We enable innovation and digital transformation by bringing together our strong company culture and our bright engineers with your innovative business vision.
Descrição da vaga

Design, develop and implement proof of concept and production-ready features.
Debugging and troubleshooting.
Design and implement user-facing features in mobile applications.
iOS development, focusing on UX design principles for mobile and tablet.
Analyse and optimise UX for resource efficiency and performance.
Work closely with our product and design teams.
Develop native Mobile applications in Objective-C and/or Swift
Optimise mobile applications on the iOS platform

Posição
Senior
Requisitos
Obrigatórios

Extensive knowledge of Xcode tool.
Solid understanding of the full mobile development life cycle.
Experience in Test Driven Development (TDD) and Agile methodologies.
Experience with multiple mobile platforms a plus (phone, tablet, watch).
Knowledge of the iOS SDK performance tools and optimisation techniques.
Experience with Apple Member Center and publishing apps on Apple Store.
Have at least 1 app published and approved in the Apple Store.
Integration with RESTful/JSON APIs and other back-end APIs.

Diferenciais

Degree in Computer Science, Information Systems, Software Engineering or related, or 7 years equivalent work experience.
Knowledge of OO concepts.
Continuous Integration/Continuous Deployment a plus.
Amazon AWS, Linux/Unix, a plus.
Have worked with international and remote teams.

Como aplicar
Please follow this link -> https://devgrid.recruiterbox.com/jobs/fk0mi58/
Contato
https://devgrid.co.uk/

O que significa semi-remote no caso? É necessário residir em RS ou SC?

O que significa semi-remote no caso? É necessário residir em RS ou SC?

Você pode inicialmente trabalhar remoto, o tempo de experiencia mas depois pode mudar para um dos polos de desenvolvimento que eh Floripa ou Caxias do Sul.
Depois voce pode trabalhar de casa, mas reunioes e decisoes tecnicas normalmente eh feita dentro do escritorio. :)
Aguardamos o teu apply!


%0A newline is breaking out

Hey guys, i hope the title is correct. =)
I want to edit a comment using steem_keychain.requestPost and patch it, newlines are displayed as %0A, the post gets updated but the response is not valid.
Reproduction:
https://pastebin.com/iBQ934RA

I can t reproduce it with the current version, could you confirm that this is working now ?


Issue with Item Sets (5 of 6 items displayed)

I have been having an issue with my in-game item sets.  Only 5 out of the 6 of the completed Items show up on my in-game shop for each of the item sets. For example, under the "highest win rate" item build path, only 5 of the 6 completed items are displayed.
I only use the champion.gg item sets.

Also confirmed that this issue exists for the "Most Frequent Used" Champion.gg set

Which ADC are you playing in the screenshot?

Xayah; but it’s an issue with all the champions I play.

Hmmm so I tested it with Yasuo and Vi to see if it was role specific. The issue exists with Yasuo, but VI seems to be ok? I mostly ADC/Mid.

Janna also has the issue

Also I just noticed sometimes a starter item (such as ancient coin) can be missing.

Ah. The issue is Ornn. The item missing for example Salvation which is built from Redemption only when Ornn is on your team. This is a dup of https://github.com/dustinblackman/Championify/issues/238, closing.


highlightjs dependency is missing from bower.json

highlightjs dependency is missing from bower.json
This results in 404 on the link:
.../lib/bower/highlightjs/highlight.pack.js
and buttons:

Show me the code
Show me the response
Show me the processed response

do not work.

install highlightjs using bower and it works fine

bower install highlightjs

Hi @sandipchitale,
thanks for pointing this out. I will try to fix it asap.
Thanks and regards,
Guy

Hi @sandipchitale,
now it should be fixed.
Thanks and regards,
Guy


Add/Delete group member not working on Windows Active Directory if groupname includes non ascii

If I start test ldap in docker as described in https://github.com/ldapjs/docker-test-openldap and create a group with non ascii characters cn=möte,ou=people,dc=planetexpress,dc=com, I can add and delete users with this code.
const mod: ldap.Attribute = { type: 'member', values: ['cn=Turanga Leela,ou=people,dc=planetexpress,dc=com'] }
const change = new ldap.Change({
  operation: 'add',
  modification: mod
});

// const mod: ldap.Attribute = { type: 'member', values: ['cn=Turanga Leela,ou=people,dc=planetexpress,dc=com'] }
// const change = new ldap.Change({
//   operation: 'delete',
//   modification: mod
// });

client.modify('cn=möte,ou=people,dc=planetexpress,dc=com', change, (err) => {
  console.log("Finnished", err)
});

If I try the same code against Microsoft Active Directory (Server 2019, objectversion 88) it works fine if the group name has only non ascii characters.
If the group name has ascii characters like "ö" in "möte" (cn=möte,ou=people,dc=planetexpress,dc=com) I get error message below:
[err: module.exports.<computed> {lde_message: 'No Such Object', lde_dn: 'OU=people,DC=planetexpress,DC=com', stack: 'NoSuchObjectError: No Such Object
    at mess…ad (node:internal/stream_base_commons:190:23)'}](url)

You stated in your report that the modify request works with a standard LDAP server implementation, but fails to work with a non-standard LDAP server implementation. Please consult with the maintainers of the non-standard LDAP server implementation about why their server does not follow the specifications.

But it is strange, if I use ldapjs 2.3.3 with this code it works great with Microsoft Active Directory (Server 2019 objectversion 88) to add and delete members even if the groupname includes non ascii characters.
const mod: ldap.Attribute = { 'member', ['cn=Turanga Leela,ou=people,dc=planetexpress,dc=com'] }
const change = new ldap.Change({
  operation: 'add',
  modification: mod
});

// const mod: ldap.Attribute = {'member', ['cn=Turanga Leela,ou=people,dc=planetexpress,dc=com'] }
// const change = new ldap.Change({
//   operation: 'delete',
//   modification: mod
// });

client.modify('cn=möte,ou=people,dc=planetexpress,dc=com', change, (err) => {
  console.log("Finnished", err)
});

If I try to run the same code with ldapjs 3.0.5 it fails with error "modification must be an Attribute".
If I change the code to
const mod: ldap.Attribute = { type : 'member', values : ['cn=Turanga Leela,ou=people,dc=planetexpress,dc=com'] }
const mod: ldap.Attribute = { type : 'member', values : ['cn=Turanga Leela,ou=people,dc=planetexpress,dc=com'] }
const change = new ldap.Change({
  operation: 'add',
  modification: mod
});

// const mod: ldap.Attribute = { type : 'member', values : ['cn=Turanga Leela,ou=people,dc=planetexpress,dc=com'] }
// const change = new ldap.Change({
//   operation: 'delete',
//   modification: mod
// });

client.modify('cn=möte,ou=people,dc=planetexpress,dc=com', change, (err) => {
  console.log("Finnished", err)
});

I am back to the error that the object is not found if the groupname includes an non ascii character.


All Table of Contents entries on pages where one or more headers are defined in a table cell link to null anchors

Description

All of the Table of Contents entries on my Must Read page link to null anchors (/#) (source).  I created a simple test page with a h2 header, and I was able to duplicate the issue.  My guess is that this is a kramdown issue, but I asked here and no response yet, so I wanted to check to see if maybe I was doing something wrong, or if this just isn't supported.
Environment

Minimal Mistakes version:
Ruby gem or remote theme version:
Jekyll version:

Get-Content -Path .\Gemfile.lock | Select-String -Pattern '^\s{4}(?:jekyll\s|minimal-mistakes-jekyll|jekyll-remote-theme)'

    jekyll (3.7.3)
    jekyll-remote-theme (0.2.3)
    minimal-mistakes-jekyll (4.11.2)

Git repository URL: https://github.com/troylindsay/troylindsay.github.io/blob/master/_pages/test.md
Operating system: Windows 10
GitHub Pages hosted (if yes provide URL to site): https://www.troylindsay.io

Expected behavior

The table of content links to page anchors should populate correctly with named anchors.
Steps to reproduce the behavior

Create test.html page with the following content:---
toc: true
---

        Test

Serve
The table of contents links will have null anchors

Couple of things I can clear up that might help.

toc: true method activates some Liquid logic using jekyll-toc that tries to parse out headings and build a TOC. It's by no means perfect and has a few gotchas. Looks like you're hitting one of them or maybe a bug.
Your test file above is a .html file... which Kramdown won't process. If you're looking for Kramdown to use its auto TOC feature it would need to be .md and the headings written in Markdown. You'd also need to manually include the Kramdown snippet to enable it... {% include toc %} in the body of your content.

I tried to look at your source but doesn't look like https://github.com/troylindsay/troylindsay.github.io/blob/master/_pages/test.md is valid anymore. If you can get a test repo up I can clone and test.

I would try a page with some headings and content, not in a table, and without  wrapped around them. jekyll-toc might be getting confused.

Did some testing of my own. Doesn't look like jekyll-toc doesn't work correctly HTML.

Test

Produces blank anchors with #. While:
## Test

Produces #test as expected. Using Kramdown with HTML doesn't seem to work either... heading is omitted from the TOC.
{% include toc %}

        Test

OK I know what's going on. Headings need an id attribute so they can be linked to. When you write headings with Markdown Kramdown automatically generates these... (## My Heading ~> id="my-heading").
Because you're writing HTML this doesn't happen. Therefore you need to manually add unique ID's to each of your headings. The following works for me:

        Test

Thanks much!


Could not find a version that satisfies the requirement decord==0.6.0

Hi,
While trying to install the requirements.txt I get:
ComfyUI-ToonCrafter % pip install -r requirements.txt
ERROR: Could not find a version that satisfies the requirement decord==0.6.0 (from versions: none)
ERROR: No matching distribution found for decord==0.6.0

Any available workaround?

Hi,
While trying to install the requirements.txt I get:
ComfyUI-ToonCrafter % pip install -r requirements.txt
ERROR: Could not find a version that satisfies the requirement decord==0.6.0 (from versions: none)
ERROR: No matching distribution found for decord==0.6.0

Any available workaround?

Are you use the mac os ?

Yes, macos Sonoma 14.4.

Yes, macos Sonoma 14.4.

Actually decord is not used by ComfyUI Node version. So we move decord to the end of requirements.txt.
But we have to say.. Mac OS Can't run this node...
I have a mbp with 64GB ram and M2Max cpu. Unfortunately, it can't work properly even fp16.

Well ... that is quite unfortunate .. what is standing in the way of it working on Mac OS ? I have a Mac Studio with 128 GB ram and it still doesn't work. Is this yet another instance of nVidia and CUDA ruining things for MacOS users? Or something else?

Well ... that is quite unfortunate .. what is standing in the way of it working on Mac OS ? I have a Mac Studio with 128 GB ram and it still doesn't work. Is this yet another instance of nVidia and CUDA ruining things for MacOS users? Or something else?

I've tested this node on my mbp with M2Max. And it works fine.

Well ... that is quite unfortunate .. what is standing in the way of it working on Mac OS ? I have a Mac Studio with 128 GB ram and it still doesn't work. Is this yet another instance of nVidia and CUDA ruining things for MacOS users? Or something else?

Have you tried the latest branch? We impl the low vram version within the latest branch.


Breaking the top block of doors prevents the item from dropping

ig I fixed this

ig I fixed this


Corrected array

Your PR does correct a problem, but you need to remember meaningful comments and signed commits, please!


fix(TDI-44856):fix tsalesforceoutput stringoutbounds issue.

What is the current behavior? (You can also link to an open issue here)
https://jira.talendforge.org/browse/TDI-44856
What is the new behavior?
Please check if the PR fulfills these requirements

[ ] The commit message follows Talend standard
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features) ?
[ ] The code coverage on new code >75%
[ ] The new code does not introduce new technical issues (sonar / eslint)

What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build / CI related changes
[ ] Other... Please describe:

Does this PR introduce a breaking change?

[ ] Yes
[ ] No

If this PR contains a breaking change, please describe the impact and migration path for existing applications: ...
Other information:

Pull request test result: success :star:


RSS Feed update

List items in reverse order of publication

Fix the publication date format to match what the RSS spec says.

Include  elements for each post, derived from any keywords specified in the post front-matter.

Fixes #1537

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: geeknoid
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [geeknoid]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Weather Cards

User - upon searching a city the user is presented with cards showing weather for the next 5-7 days
Acceptance - it is done when the user has a card that includes the date, temp, conditions

Acceptance - it is also done when the cards display a top middle and bottom of the outfit.


SCOPED SLOTS[✨]

Is your feature request related to a problem?
There is currently no way to pass down component state to it's  children through slots. For instance, If you have a data table and you want the ability to provide slots for the table's cell content, there is hardly a way to do so with the current API using Slot.
Describe the solution you'd like
Proposed api:
<Slot name={'cell'} props={cellData}/>

just like In vue:
<slot v-bind:cell="cellData"/>

And that would allow us to use slots like:

 // proposed api
{cellData.label}

Just as in vue:
<template #cell={props}>

{{props.cellData.label}}

<template>

Describe alternatives you've considered
Render Functions with Props:
import {$, Component, component$} from<EMAIL_ADDRESS>
const withCellSlot = (WrappedComponent: Component) => (props: any) => {

  const { cellData, ...otherProps } = props;

  const renderCell = $((cellProps: any) => {

    return  (

Cell Title:

{cellData?.label}

    )
  })

  return (
    <WrappedComponent
      {...otherProps}
      renderCell$={renderCell}
    />
  );
};

const MyCustomCellComponent = component$((props: any) => {

  return
{props.renderCell$(props.cellData)}
;
});

const TheirCustomCellComponent = component$((props: any) => {

  return

{props.renderCell$(props.cellData)}

;
});

export  const EnhancedMyComponent = withCellSlot(MyCustomCellComponent);

and then use i'd use it like:
export default component$(() => {
  const cellData = useSignal({label: 'Hey there'});

  return <EnhancedMyComponent cellData={cellData} />;
});

and they would use it like:
export default component$(() => {
  const cellData = useSignal({label: 'Hey there'});

  return <EnhancedMyComponent cellData={cellData} />;
});

Look at all this JS for a basic feature - also harder to read.
Additional context
No response

I'm sorry, I don't understand how the props would be used. A slot is a virtual placeholder, it gets replaced by the child for the slot.

@wmertens with the proposed API: We will be able to define CustomComponents like:
const CustomComponent = component$(() => {
const cellData = useSignal({label: 'Hi What!!!'})
 return (

Cell Title:

      <Slot name={'cell'} :cell-data={cellData.value}/>

  );
})

and use it like:
const MyCustomCellComponent = component$(() => {

  return <CustomComponent q:slot={'cell'} q:prop={cellData}>

{ cellData.label }

  </CustomComponent>
});

and
const TheirCustomCellComponent = component$(() => {

  return <CustomComponent q:slot={'cell'} q:prop={cellData}>
   <button>{ cellData.label }</button>
  </CustomComponent>
});

@wmertens . Here is where the idea is coming from.
https://vuejs.org/guide/components/slots.html#scoped-slots

Yeah I don't think this matches the Qwik architecture well. Is it so bad to use render functions instead, combined with context?
Remember that slots will use the context from the slot parent, which is deeper than the real parent.
So if you have a context cellDataCtx, you can read from it in the slotted cell component.

Thanks @jermsam and @wmertens for this interesting discussion.
The issue is stale and I think our Discord server is perfect for this kind of discussions. :+1:


a question

Where are the first eight data sets

Please follow README.md to obtain those eight data sets


Change temperature tolerance of all PhaseEquil constructor to 1e-4

Some small clean up, but hopefully it's an improvement.

Code changes look fine to me. Could we change the title to "Relax temperature tolerances"?
Also, maybe we can update the news? (we can add a precision changes expected note). This will ultimately result in fewer NL solver iterations at the expense of slightly less accurate solutions. That's what we're aiming to do, correct?

I changed the title and updated the news. I don't think we will see a difference in ClimaAtmos now as the constructor with a different tolerance is only used in some initial conditions. But we can expect changes in iterations if we further change the tolerance.


Hello world example does not compile

Bug Report

Version

Latests master on github
Platform
Linux dev-fedora 6.1.43-1.qubes.fc32.x86_64

Description

I did the following
git clone https://github.com/hyperium/tonic.git
cd tonic/examples
cargo build --bin helloworld-server
and then I get the following error:
   Compiling examples v0.1.0 (/home/user/dev/experiments/tonic/examples)
error: failed to run custom build command for `examples v0.1.0 (/home/user/dev/experiments/tonic/examples)`

Caused by:
  process didn't exit successfully: `/home/user/dev/experiments/tonic/target/debug/build/examples-e0eba3978652fe71/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-changed=proto/routeguide/route_guide.proto
  cargo:rerun-if-changed=proto
  cargo:rerun-if-changed=proto/helloworld/helloworld.proto
  cargo:rerun-if-changed=proto
  cargo:rerun-if-changed=proto/echo/echo.proto
  cargo:rerun-if-changed=proto/echo
  cargo:rerun-if-changed=proto/unaryecho/echo.proto
  cargo:rerun-if-changed=proto/unaryecho
  cargo:rerun-if-changed=proto/attrs/attrs.proto
  cargo:rerun-if-changed=proto
  cargo:rerun-if-changed=proto/googleapis/google/pubsub/v1/pubsub.proto
  cargo:rerun-if-changed=proto/googleapis

  --- stderr
  thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: "protoc failed: google/protobuf/descriptor.proto: File not found.\ngoogle/api/annotations.proto:20:1: Import \"google/protobuf/descriptor.proto\" was not found or had errors.\ngoogle/api/annotations.proto:28:8: \"google.protobuf.MethodOptions\" is not defined.\ngoogle/api/client.proto:19:1: Import \"google/protobuf/descriptor.proto\" was not found or had errors.\ngoogle/api/client.proto:27:8: \"google.protobuf.MethodOptions\" is not defined.\ngoogle/api/client.proto:66:8: \"google.protobuf.ServiceOptions\" is not defined.\ngoogle/api/client.proto: \"google.protobuf.ServiceOptions\" is not defined.\ngoogle/api/field_behavior.proto:19:1: Import \"google/protobuf/descriptor.proto\" was not found or had errors.\ngoogle/api/field_behavior.proto:27:8: \"google.protobuf.FieldOptions\" is not defined.\ngoogle/api/resource.proto:19:1: Import \"google/protobuf/descriptor.proto\" was not found or had errors.\ngoogle/api/resource.proto:28:8: \"google.protobuf.FieldOptions\" is not defined.\ngoogle/api/resource.proto:34:8: \"google.protobuf.FileOptions\" is not defined.\ngoogle/api/resource.proto:40:8: \"google.protobuf.MessageOptions\" is not defined.\ngoogle/protobuf/duration.proto: File not found.\ngoogle/protobuf/empty.proto: File not found.\ngoogle/protobuf/field_mask.proto: File not found.\ngoogle/protobuf/timestamp.proto: File not found.\ngoogle/pubsub/v1/schema.proto:19:1: Import \"google/api/annotations.proto\" was not found or had errors.\ngoogle/pubsub/v1/schema.proto:20:1: Import \"google/api/client.proto\" was not found or had errors.\ngoogle/pubsub/v1/schema.proto:21:1: Import \"google/api/field_behavior.proto\" was not found or had errors.\ngoogle/pubsub/v1/schema.proto:22:1: Import \"google/api/resource.proto\" was not found or had errors.\ngoogle/pubsub/v1/schema.proto:23:1: Import \"google/protobuf/empty.proto\" was not found or had errors.\ngoogle/pubsub/v1/schema.proto:70:50: \"google.protobuf.Empty\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:19:1: Import \"google/api/annotations.proto\" was not found or had errors.\ngoogle/pubsub/v1/pubsub.proto:20:1: Import \"google/api/client.proto\" was not found or had errors.\ngoogle/pubsub/v1/pubsub.proto:21:1: Import \"google/api/field_behavior.proto\" was not found or had errors.\ngoogle/pubsub/v1/pubsub.proto:22:1: Import \"google/api/resource.proto\" was not found or had errors.\ngoogle/pubsub/v1/pubsub.proto:23:1: Import \"google/protobuf/duration.proto\" was not found or had errors.\ngoogle/pubsub/v1/pubsub.proto:24:1: Import \"google/protobuf/empty.proto\" was not found or had errors.\ngoogle/pubsub/v1/pubsub.proto:25:1: Import \"google/protobuf/field_mask.proto\" was not found or had errors.\ngoogle/pubsub/v1/pubsub.proto:26:1: Import \"google/protobuf/timestamp.proto\" was not found or had errors.\ngoogle/pubsub/v1/pubsub.proto:27:1: Import \"google/pubsub/v1/schema.proto\" was not found or had errors.\ngoogle/pubsub/v1/pubsub.proto:159:3: \"Encoding\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:230:3: \"google.protobuf.Timestamp\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:261:3: \"google.protobuf.FieldMask\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:692:3: \"google.protobuf.Duration\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:760:3: \"google.protobuf.Duration\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:764:3: \"google.protobuf.Duration\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:809:3: \"google.protobuf.Duration\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:919:3: \"google.protobuf.FieldMask\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:1192:3: \"google.protobuf.FieldMask\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:1225:3: \"google.protobuf.Timestamp\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:1304:5: \"google.protobuf.Timestamp\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:118:48: \"google.protobuf.Empty\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:463:16: \"google.protobuf.Empty\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:476:16: \"google.protobuf.Empty\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:492:48: \"google.protobuf.Empty\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:529:16: \"google.protobuf.Empty\" is not defined.\ngoogle/pubsub/v1/pubsub.proto:607:54: \"google.protobuf.Empty\" is not defined.\n" }', examples/build.rs:33:10

If I remove the following lines from examples/build.rs it works again
    tonic_build::configure()
        .build_server(false)
        .compile(
            &["proto/googleapis/google/pubsub/v1/pubsub.proto"],
            &["proto/googleapis"],
        )
        .unwrap();

Hello, did you try re-installing the  protoc package ?
moreover can you specify the tonic commit you tried, telling "current master" is not enough.

I am having the same issue. I have a fresh install of protoc running on version v0.11.0 of tonic (https://github.com/hyperium/tonic/releases/tag/v0.11.0).

When in protoc 25.1 on arch , build ok. when protoc 3.14.0 on rockylinux, same issue.
Now just workaround as above remove line.

I am having this issue with protoc 3.19.6 and tonic 0.9 on Fedora 39.
Commands in the description give me the same error

Has someone solved this problem? I also have the issue on protoc 3.12.4.


COLR need to count for reachability (com.google.fonts/check/unreachable_glyphs)

Observed behaviour
Checks on Noto Color Emoji claim vast #s of unreachable glyphs. My guess is COLR v0 and/or v1 don't "count" when computing reachability.
WARN: Check font contains no unreachable glyphs ([com.google.fonts/check/unreachable_glyphs](https://font-bakery.readthedocs.io/en/stable/fontbakery/profiles/universal.html#com.google.fonts/check/unreachable_glyphs))
⚠ WARN The following glyphs could not be reached by codepoint or substitution rules:

glyph05311
glyph10700
glyph21429
glyph16679
glyph20585
glyph30485
glyph08398
glyph18035
glyph39093
glyph29709
And 35524 more.

Ref https://github.com/google/fonts/pull/4994#issuecomment-1197611451.
Expected behaviour
COLR should count as a way to reach glyphs.
Resources and exact process needed to replicate
https://github.com/googlefonts/color-fonts has a set of sample font sthat may be useful in testing.

Thanks! I'll try to come up with a fix for this using the sample files.

see my comment in https://github.com/googlefonts/fontbakery/pull/3838#pullrequestreview-1053613190

Thanks, @anthrotype. I've just read your comment. I will update the code to incorporate the fixes you suggested.

Like @khaledhosny suggested, I think the entier unreachable_glyphs check could be replaced by a simpler call to the fontTools subbsetter (passing --unicodes="" --layout-features="" to keep all the cmap-encoded glyphs as well as all the ones activated by GSUB features), comparing the set of original glyphs with the one after subsetting should give you the answer as to the unreachable glyphs

.. and by default the subsetter will also do the glyph closure over the COLR table, which is what you wanted to do in the first place

... and yes it also does the glyph closure for the glyf components. It does the right thing.

But maybe it could be better to not use the subsetter to avoid eventual bugs in it both affecting the font binary compilation and the font checking and, thus, not detecting a problem.
Having an alternative implementation on the check might help to catch such problems if they ever happen.

I will try to write the traversal code that you suggested. And it can also be an useful exercise for me to get more familiar with handling the data from the COLR table.

if you're going to use fontTools API anyway (which I think fontbakery does), you'll end up writing code that's very similar and you'll probably be tempted to peek at the subsetter code itself, so in the end you'll end up with two unnecessary copies. Anyway whatever you prefer.

yes, I had a similar thought a few minutes ago ;-)
I need some time to think about it.

All of the 52 font binaries available at https://github.com/googlefonts/color-fonts/tree/main/fonts pass the currently shallow check implementation.
Does anybody know where I can find font binaries with hierarchical graphics compositions for color glyphs (not referencing glyphs only at the root), so that I could use such samples to test a fully traversing implementation of this check?

A kind +1 to this. The https://github.com/justvanrossum/nabla/actions/runs/2845398858 reports more than 750 unreachable glyphs, even though they are all referenced by the COLR table.

A kind +1 to this. The current build of Nabla reports more than 750 unreachable glyphs, even though they are all referenced by the COLR table.

The Nabla.ttf saved as an asset of the GH Action you referenced here does not fail the check implementation in our current git main branch (future FB v0.8.10 release).

So, it means I still have not yet found samples of deeper graphical compositions in color font files.

Maybe @drott knows?
Perhaps this one: https://github.com/BlackFoundryCom/black-renderer/blob/main/Tests/data/more_samples-glyf_colr_1.ttf
(it is out of date, and I can't find the proper upstream for it off hand)

Btw. I bet @behdad's new visitor module will help you more easily traverse all PaintGlyph and PaintColrGlyph nodes: https://github.com/fonttools/fonttools/pull/2718/files

Perhaps this one: https://github.com/BlackFoundryCom/black-renderer/blob/main/Tests/data/more_samples-glyf_colr_1.ttf
(it is out of date, and I can't find the proper upstream for it off hand)

The canonical source is here: https://github.com/googlefonts/color-fonts#colrv1-test-font

Edit: it seems that font does contain deeper nested glyph references, but since all glyphs are encoded in the cmap, this won't help your test, either :( Maybe it's hackable, though.

I think it's possible to add a test where a COLRv1 referenced glyph is not added to the CMAP. See https://github.com/googlefonts/color-fonts/blob/main/config/test_glyphs-glyf_colr_1.py for the source.

new visitor module will help you more easily traverse all PaintGlyph and PaintColrGlyph nodes

COLR Paint objects already have a more specialized traverse method, which is also used by the fonttools subsetter, see:
https://github.com/fonttools/fonttools/blob/e494b118c4dd471fa268f817915249fd36858a10/Lib/fontTools/subset/__init__.py#L2112-L2123
https://github.com/fonttools/fonttools/blob/e494b118c4dd471fa268f817915249fd36858a10/Lib/fontTools/ttLib/tables/otTables.py#L1576-L1588

Please discuss further improvements still needed at issue #3881


Updated documentation to reflect recent changes made in installation …

…script

All is well.

All is well.


Fix typo in docs

Please ignore my pull request! User error on my part.

No worries.. I've seen a couple of cases where people have used the source repo rather than the plugin repo which is fine, but has annoying build time implications.
Perhaps in the future we can make it clear in the docs about the differences and why you might prefer to use the -plugin repo rather than having to compile the source each time.


feat: Added Localisation for English, Portuguese, Spanish, Mandarin and Cantonese

- What I did

Added Localisation for English, Portuguese, Spanish, Mandarin and Cantonese
The default language is based on the user device language
- How I did it

- How to verify it
- Description for the changelog

Will review the rest after lunch.

@CurtlyCritchlow can you open a ticket to have each of these translations verified with a native speaker.


Exchange ease feature

Pull Request for PyVerse 💡
Requesting to submit a pull request to the PyVerse repository.

Issue Title
Please enter the title of the issue related to your pull request.
[Code Addition Request]: Currency Converter

[x] I have provided the issue title.

Info about the Related Issue
What's the goal of the project?
The aim of the project is to provide a simple and reliable tool for users to convert currencies using real time exchange rates.

[x] I have described the aim of the project.

Name
Please mention your name.
Amrutha M

[x] I have provided my name.

GitHub ID
Please mention your GitHub ID.
amrutha-m206.

[x] I have provided my GitHub ID.

Email ID
Please mention your email ID for further communication.
<EMAIL_ADDRESS>
[x] I have provided my email ID.

Identify Yourself
Mention in which program you are contributing (e.g., WoB, GSSOC, SSOC, SWOC).
gssoc-ext, hacktoberfest, hacktoberfest-accepted

[x] I have mentioned my participant role.

Closes
Enter the issue number that will be closed through this PR.
Closes: #955

[x] I have provided the issue number.

Describe the Add-ons or Changes You've Made
Give a clear description of what you have added or modified.
A simple web interface which allows users to input amounts and select currencies for  conversion, enhancing usability. An external API was integrated to fetch real-time exchange rates  along with error handling for invalid inputs, ensuring accurate conversions

[x] I have described my changes.

Type of Change
Select the type of change:

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Code style update (formatting, local variables)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Describe how your changes have been tested.
Has been tested manually with inputs and results have been verified.

[x] I have described my testing process.

Checklist
Please confirm the following:

[x] My code follows the guidelines of this project.
[x] I have performed a self-review of my own code.
[ ] I have commented my code, particularly wherever it was hard to understand.
[ ] I have made corresponding changes to the documentation.
[ ] My changes generate no new warnings.
[x] I have added things that prove my fix is effective or that my feature works.
[ ] Any dependent changes have been merged and published in downstream modules.

@UTSAVS26 Can this be merged if the review is completed?

@UTSAVS26 Can this be merged if the review is completed?

Wait for sometime as mentors are reviewing too.


initial analysis

Would love to get some feedback on what changes in the tech note or the notebook would be useful.
I will then add the analysis for the de-acceleration after braking.

This looks good! Do you want to close this initial analysis until we get feedback? Then we can open a new ticket for updates. What do you think?


sysbuild: Add extensions for native_simulator based targets & reduce boilerplate in samples/tests sysbuild files

Add sysbuild cmake extensions/functions for native_simulator based targets
And use those in the current samples and tests to reduce boilerplate.

@tejlmand. Following what we discussed a few days ago. You are welcome to propose better naming :)


.tmp file created while formatting notebook

Type: Bug

With python installed
Open a jupyter notebook in a workspace with a python cell and try formatting it

bug
There's a flash of a file called notebook.ipynb.XXXXXX.tmp created next to the notebook. I also see running autopep8 in the status bar

Extension version: 2023.9.11311005
VS Code version: Code - Insiders 1.79.0-insider (Universal) (02298ef4180f2a5db39e0cb6288382860b96a596, 2023-05-11T05:26:00.234Z)
OS version: Darwin x64 22.4.0
Modes:
Sandboxed: Yes

This is expected with formatting support from the extension. Some formatters don't support formatting notebooks entirely, so each cell is copied to a temporary file and formatted and the content from the temp file is returned to the editor as edits. This approach will be removed soon in favor of doing this over LSP in the formatter extensions.
The recommendation here is to use the formatter extensions black or autopep8 which do this over LSP.

Does the temp file need be created in the workspace? I have format on save enabled so it's a pretty poor UX to see the temp file created every time I save a notebook

I have format on save enabled so it's a pretty poor UX

I agree that is why we are moving to better solution, of doing this over LSP. Unfortunately, formatters and import organizers in python in some cases have to do it in a file relative way. So for cases like from .something import Something, that is a file relative import. Creating a temp file in a /tmp folder will not work for the tool to determine where to put it in the order.


Server gets error json data when posting nested object

I want to post an article with several sections to the server. Here are my codes.
class Article: NSObject {
    var id = ""
    var desc = ""
    var title = ""
    var publisher_id = ""
    var updated_at: Int64 = 0
    var category_id = ""
    var sections: [Section] = []
    class func mappingForCreate() -> [String: AnyObject] {
        return [
            "desc":"description",
            "title":"title"
        ]
    }
    class func mapping() -> [String: AnyObject] {
        return [
            "id":"id",
            "description":"desc",
            "title":"title",
            "publisher_id":"publisher_id",
            "created_at":"created_at",
            "updated_at":"updated_at",
            "category_id":"category_id"
        ]
    }
}

class Section: NSObject {
    var article_id = ""
    var title = ""
    var content = ""
    var type = "text"
    var sub_type = ""
    var weight: Int = 0

    class func mapping() -> [String: AnyObject] {
        return [
            "id":"id",
            "article_id":"article_id",
            "title":"title",
            "content":"content",
            "weight":"weight",
            "type":"type",
            "sub_type":"sub_type"
        ]
    }

    class func mappingForCreate() -> [String: AnyObject] {
        return [
            "title":"title",
            "content":"content",
            "type":"type"
        ]
    }
}

let mapping = RKObjectMapping(forClass: Article.classForCoder())
mapping.addAttributeMappingsFromDictionary(Article.mapping())
let descriptor = RKResponseDescriptor(mapping: mapping, method: RKRequestMethod.Any, pathPattern: nil, keyPath: nil, statusCodes: nil)
var urlStr = "http://localhost:8080"

let requestMapping = RKObjectMapping(forClass: NSMutableDictionary.classForCoder())
requestMapping.addAttributeMappingsFromDictionary(Article.mappingForCreate())

let sectionMapping = RKObjectMapping(forClass: NSMutableDictionary.classForCoder())
sectionMapping.addAttributeMappingsFromDictionary(Section.mappingForCreate())

requestMapping.addRelationshipMappingWithSourceKeyPath("sections", mapping: sectionMapping)

let requestDescriptor = RKRequestDescriptor(mapping: requestMapping, objectClass: Article.classForCoder(), rootKeyPath: nil, method: RKRequestMethod.Any)

let manager = RKObjectManager(baseURL: NSURL(string: urlStr))
manager.addRequestDescriptor(requestDescriptor)
manager.addResponseDescriptor(descriptor)
manager.postObject(article!, path: "/articles", parameters: nil, success: { (op, result) -> Void in
    println(result)
}) { (op, error) -> Void in
    println(error)
}

I thought the server would get this:
{"description":"","sections":[{"content":"content1","title":"title1","type":"text"},{"content":"content2","title":"title2","type":"text"}],"title":"article title"}

but it got this:
{"description":"","sections":[{"content":["content1","content2"],"title":["title1","title2"],"type":["text","text"]}],"title":"article title"}

I googled this everywhere, but most of the results are talking about mapping for response, not for request.  Please help! Thanks!

GitHub Issues is for reporting bugs in RestKit and discussing changes to RestKit itself. Please check the documentation, wiki, and existing issues before opening a new issue.
Additionally, please do not post general usage questions to Issues, but instead take them to Stack Overflow.


Search editor crashes (OOM) when matching a very long line (e.g. minified JSON)

VSCode Version: 1.50.1
OS Version: macOS 10.14.6 (18G6032)

Steps to Reproduce:

Extract this file into a VS Code workspace workbench.web.api.js.map.zip
Open a search editor and search for "// TODO".
The VS Code window will crash prompting to reload.

 Yes

Not a new issue, but I'm going to close an old bug in favor of this because of your nice repro example.


Observers counterpart (retrievers?)

Right now there are observers that store info, config, metrics and artifacts to a database or the filesystem. But there is no way to retrieve and process the stored information via sacred, even though I guess everyone needs to retrieve and analyze their runs. I know about sacredboard, omniboard etc. but these are visualization tools and focus mainly on the MongoDB case. I think it would be useful to have a companion-class for every observer defined within sacred (a retriever?) that retrieves data, reconstructs a sacred.Run object and provides methods such as get_metrics, get_config, get_artifacts. This would abstract the way runs are retrieved and perhaps allow visualization tools to be observer-agnostic.
Edit 1: Would be happy to write/help writing a PR.
Edit 2: I see incense contains some of the logic described here but it also focuses on MongoDB and the visualization is specific to IPython.

I see incense contains some of the logic described here but it also focuses on MongoDB
I would be open to support other observer backends/formats in incense. However, as the observers do not use a common API one has to implement a new client for each observer. I would advocate to rather support a smaller set of observers, namely mongo and filestorage, but support those well.

visualization is specific to IPython
The IPython specific visualization only pertains artifacts. In non-interactive environments, I do not really see how you would use a custom for each artifact type visualization. Instead saving the artifacts back on disk should cover most non-interactive use case. I am open for suggestions though.

Hi @JarnoRFB, I agree, supporting less options well would be better - although I think with sqlalchemy one could write one sql-based observer to support them all. I also agree that one has to implement one client per observer. I just think these clients - the data-fetching part - should be part of sacred, as are the observers. The visualization part should be left to the users where they can use tools like incense or write custom solutions.

Hi @rueberger, are referring to the observers or sacred in general? I think, the observers are probably the core feature of sacred. Since some decisions have been made about the structure of a run_entry - which I think are perfectly logical - I'm just asking to expose this run_entry to be readable after the run.
I agree that writing retrievers on your own is not that hard, but isn't it better to write this once in sacred instead of forcing every user do this tedious work again and again? I think it would also be beneficial for sacred itself to enforce consistency (among observers) also for future versions. Even if there are fundamental changes coming to the structure of sacred, isn't the run_entry gonna stay pretty much the same? I find it hard to imagine how a retriever could somehow lock us.

I agree with @johny-c. I think it would be good for sacred to provide a common interface to access the data stored by the obervers. At least for the observers that actually store information, and not telegram, slack....

I've been writing my own "retrievers" for my projects thus far and imo it's really not that bad.

Everyone does this. And I think that sacred should provide a unified and easy way of retrieving data and make this process as easy as possible so that not everybody has to waste time on (re)implementing own versions of a "Retriever". It would also facilitate the process of loading data for external tools (like incense) independent of the actual obeserver used.
I think the observers should be in general replaceable, which means that the choice of the observer does not influence the rest of the experiment code. A unified "Receiver" concept would be a step into this direction.
Another point is that there is currently no easy way to actually reproduce an experiment (#212), and I think sacred should provide this functionality as stated in the README ("reproduce your results"). Retrievers could be a way to gather all data needed to reproduce an experiment independent of the used observer.

I just don't feel like the abstraction is mature enough to be adding new features that lock us in to the current abstraction.

Would "Retrievers" lock us? They are kind of an extension to the observers. Do you have an idea of a concept that is favorable over the observers? I think the abstraction of seeing the combination of an Observer and a Receiver as a database that stores data of a run and provides a simple interface to read the data is quite good. The actual implementation of the data storage does not matter. The only thing is that there are currently two types of Observers: 1) Observers that save data (FileStorage, Mongo...), and 2) Observers that inform the user (Telegram, Slack) and cannot be used to retrieve data later on. These should maybe be separated if Retrievers would be introduced, or type 1) is a special case of type 2).

Would "Retrievers" lock us?

No but it would create more friction. I'm thinking small refinements to the abstractions here, not rewrite it all from scratch. Such as support for parameters that change over time, which would be required to use population based training..

Retrievers have been on my personal wishlist for a long time. I just haven't gotten around to writing them.  And I agree that everyone writing their own version of these is a clear sign that this should be a core feature.
It is quite a project, because each observer requires their own retriever, and ideally we would have a common interface for all of them, such that you can use the retrieve interface independently of the backend. This means quite a bit of work and we need to think carefully about the interface such that:

experiments can be queried in a convenient way. This should be rather easy to implement for Mongo and SQL but quite a bit of work for FileStorage. So we need to think carefully about the feature-set.
A common format for the resulting entries, that makes it easy to inspect and reproduce.
A way to get summaries over many experiments similar to what incense does.

And while we are dreaming: I would love to have some common cleanup and database maintenance operations such as:

delete old experiments
batch modification of tags, comments and other meta-info
migrate/sync runs between backends
serialize/dump a subset of runs for archiving

The core observers clearly are the Mongo and FileStorage observers, so I think starting with support for just those two could be a good starting point. Unfortunately I currently do not have the time to tackle this, but would be happy to support any effort on this front.

Hi @Qwlouse, that's great to hear. I would be happy to start a PR on that.
I think, this common retriever interface should return something like a read-only CollectedRun instance that contains a subset of the Run attributes.
Perhaps, a good reference for how to handle interactions with a database in python is django ( django models )