Fixes #36: Unreliable id() and meaning of base().

This PR fixes issue #36: Unreliable id() and meaning of base().
Please review and merge.
In addition to what the commit message says, I have also updated the docstrings for those methods that had to do with the change.
Details from the commit message:

Added support for a reliable distro ID returned by id().
The normalization happens only for id(); the *_release_info() and
get_*_release_attr() methods do not provide a reliable ID; these
methods provide the values as obtained from the respective sources.
The set of distros for which the ID is reliable has been initially
taken from the _DIST_TO_BASE dict. A TODO has been added to make
sure we review that list.
These reliable distros are documented with their ID values in the
description of the id() method.
Removed the base() method and, because it was no longer used, its
_DIST_TO_BASE dict.
Adjusted testcases and standalone query script to the changed behavior
of the id() method.
Documented a TODO in the TestOverall test class for those distros that
have reliable IDs, but have no testcase representation so far.
Changed the mocking concept for the TestLSBRelease class to also use
the concept from DistroTestCase. This became necessary because the
previous mocking concept was making assumptions about the
implementation that were no longer met. As part of that, the LSB
output files in the "special" directory became simple lsb_release
scripts that print a hard coded set of lines, in the "testdistros"
directory.

Comments?


Node Grouping

Initial design note for grouping nodes in the editor.

I have updated this design to match what was implemented for 1.1.0.


feat(protocol): L2 1559 use gasUsed to adjust basefee

@Brechtpd  this PR enables L2 1559 gas fee to be calculated and verified when a block is to be verified. When a block is proven, gasfee and gasUsed must be provided by the prover, gasUsed will be part of the fork choice, and gasfee is used as a link to this fork choice.
Something to notice: now for a block, its basefee does not depends on its own actual gas used, instead, the basefee is the one updated after its parent block. In other words, the basefee is the spot price of the curve, not the average price of a gas sale. The major reason for such a change is to remove basefee calculate's self-dependency. If we keep using the average price of a gas sale, then we have to iterate through all fork choices to find the correct one, as basefee depends on gasUsed, while gasUsed maybe affected by the value of basefee. But I'm ok to implementation suggestions so we can still use the average price, not the spot price.
It seems Ethereum also uses the basefee calculated on the previous blocks, not the current block's gas used.
TODOs:

[ ] add more testing
[ ] Talk to client team so they know how to implement this and how to tell users the basefee value at any given moment.

Doing things in the anchor function keeps the fork choice logic simpler and moves everything to L2

I'll think about it and if it is possible, we should definitely move it inside L2 anchor.

Closed with a new PR: https://github.com/taikoxyz/taiko-mono/pull/13533


Hi! Thanks for the great work! I would like to ask if you can publish the code and dataset of the paper?

Hi! Thanks for the great work! I would like to ask if you can publish the code and dataset of the paper?

Dear @dante1024 ,
I feel sorry it is hard for me to provide because the code implemented in 3 years ago (I left school for nearly 2 years it has been published...). But it can quickly implemented by simply transfer some codes with ICON.
Please refer to https://github.com/mczhuge/CubeNet/issues/3.

Dear @dante1024 ,
I feel sorry it is hard for me to provide because the code implemented in 3 years ago (I left school for nearly 2 years it has been published...). But it can quickly implemented by simply transfer some codes with ICON.
Please refer to #2.

Dear @dante1024 ,
I feel sorry it is hard for me to provide because the code implemented in 3 years ago (I left school for nearly 2 years it has been published...). But it can quickly implemented by simply transfer some codes with ICON.
Please refer to #2.

Excuse me, I see that your paper mentioned the use of edge supervision. Where is the relevant edge label dataset?

@dante1024 just a structure loss, please see the details of https://github.com/mczhuge/CubeNet/issues/2.

AltLayer Airdrop Season One Announcement

Exciting news! AltLayer token (ALT) is set to launch on Binance on January 25, 2024. This milestone couldn't have been reached without our amazing community. Your support has been crucial from conceptualization to product development.
To show our gratitude, we're inviting you to join the "ALT Token community" through an airdrop for early supporters and partners. Below is a summary:

Total Allocated ALT Tokens: 300,000,000 (3% of total supply)
Distribution Categories:

NFT Holders
Altitude Campaign Participants
EigenLayer Restakers
EigenLayer Ecosystem Partners
Celestia Stakers and others

For detailed criteria and information, check the full announcement here.
Claim opens from January 25 to February 25, 2024. Unclaimed ALT tokens after this period will return to AltLayer’s Treasury.
Thank you for being part of AltLayer's success! More to come in Season One, stay tuned.
Note: Certain regions are restricted from the airdrop. Please review the disclaimer before participating.
Winners: @razdob15, @aurbina83, @oudommeas, @NashidAli007, @maruk0chan, @selcukvardar, @priom, @layupbolon, @hadiasghari, @bill816, @IrvanFza, @kaseaafnie2009, @Shikei001, @Rossco8, @FGU1, @pieroliviermarquis, @Kayne88, @nukezie, @gb4rr3t0, @ycaedes


Improve structure, formatting and wording

Method descriptions as headline to add structure
Small formatting changes for the ENUM values
Small wording changes to improve clarity
Link to cordova-config-utils

@piotrowski, can you update your PR, so that I can test and release 2.1.0

Hm? I think you want me to include the changes that happened on master... but to be honest I have no idea how to do that. Normally I just edit files in the online editor here.
Is there a way to "rebase" in the web UI of Github? Otherwise I will have to create a new PR and 'redo' my changes on the current version.

Found it: http://www.hpique.com/2013/09/updating-a-fork-directly-from-github/

Yay, I think I did it!  Luckily Sourcetree could help. More or less...) but I think I also fixed that one and didn't loose or break anything.
Right?

looks good thanks!


Capitalizing

Title: Capitalizing
About: The aim is to replace the first character of all the words with their upper case characters.
Name: Manasi Chhibber
Label: Feature Request
Assignee: 'Manasi2001'

Define You:

[x] DevIncept Participant
[x] Contributor

Is your feature request related to a problem? Please describe.
The purpose is to explore string functions like slicing.

Issue assigned. @Manasi2001


Set() should only be used in pre-operation

Under Best Practices, it says to used the Set() instead of Patch(), but Set() also need to be used in PreOpearation, as it will also cause infinite loop if used in PostOperation()

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 2e4d736d-b58b-6ad4-03ca-4b68be60f6db
Version Independent ID: 2e4d736d-b58b-6ad4-03ca-4b68be60f6db
Content: Example Dataverse low-code plug-ins - Power Apps
Content Source: powerapps-docs/maker/data-platform/lowcode-plug-ins-examples.md
Service: powerapps
Sub-service: dataverse-maker
GitHub Login: @Mattp123
Microsoft Alias: matp

Thanks for your comment, @xairrick. This issue has been assigned to the doc author for review.


Only coding 1st frame of my slide

I have 10-12 slides but it only codes my first frame

You have to split each page into separate files. At least that's what worked for me.

Checkout the latest version.
@dan-w1231 @piyushXC


UN-528: Alt text on highlights in the HighlightsGallery

Ticket URL: https://national-archives.atlassian.net/browse/UN-528
About these changes
Added alt_text field used on other pages on each Highlight item in the InlinePanel, and updated HTML to reflect this.
How to check these changes
Edit a HG page, see the alt_text field, enter some text, then view the page and ensure that the alt text is being applied on the images.
Before assigning to reviewer, please make sure you have

[x] Checked things thoroughly before handing over to reviewer.
[x] Checked PR title starts with ticket number as per project conventions to help us keep track of changes.
[x] Ensured that PR includes only commits relevant to the ticket.
[x] Waited for all CI jobs to pass before requesting a review.
[x] Added/updated tests and documentation where relevant.

Merging PR guidance
Follow docs\developer-guide\contributing.md
Deployment guidance
Follow docs\infra\environments.md

Hi @ababic, this should be good to go now! 😎


bring Hummingboard OS version up to current (presently at 1.6)

The latest Hummingboard OS is v1.6.0 and is missing, among other things, kernel module(s) for device mapper. Device mapper is required for crypto-loop operations.
Version 1.26.0-rc0 from staging (with live config.json values) leaves the device in a constant reboot cycle after the splash screen.
Please bring the OS to a working release.
Ref: https://forums.resin.io/t/hummingboard-os-update/763

@ab77 What base board and SOM are you personally interested in?

I am not physically in possession of the device, all I know that it is a CuBox-i

So you are interested in using the image on a cubox, right?

I have added support for this in our repositories.  Will send a pull request.

@agherzan Affirmatory, CuBox.

@ab77 if you don't mind testing I will send you links to my repositories after resin-fsl-arm is upgraded to the latest 2.0.0 version.

@linux4kix cheers, we'll get it tested once I have the v2 image.

@linux4kix any luck with the image update?

@ab77 I have the image update done and am now debugging various little issues.  I think I am down to the last one which is some missing symbols for the network stack.

Great, looking forward to testing it out.

Good day All,
Any luck with getting this working for the CuBox?


Did you already released #2081 ?

Did you already released #2081 ?

and when will be the next release?)

https://twitter.com/nuxt_js/status/935100400415502342
We now have an unstable release channel which a new version is published automatically with every successful commit. If you want to try the lastest version of Nuxt.js, you can use yarn add nuxt@next --exact or npm i --save-exact nuxt@next. Be aware that it's the unstable channel until 1.0 is hit. Also for a yarn lock file workaround make sure that version is stored in package.json without caret (^) or is set to next tag.
PS: Every single contribution is REALLY appreciated to reach 1.0.0 as fast as possible. (See roadmap)

Closing as a dup of #2142. Please track things from there :)

Closing as a dup of #2142. Please track things from there :)


With 170 days left until Python 2 EOL, it is time to upgrade default Python

A continuation of #1678 now that dist: xenial is the Travis CI default we can upgrade from Python 3.6 to 3.7 as the default Python on Travis CI Linux jobs.
Python 3.7 has a slightly stricter syntax than 3.6 because async and await are keywords in Python >= 3.7.  For projects that use Travis CI with the default Python, this pull request will allow them to find these issues before their users do.

@BanzaiMan @MariadeAnton Can anyone please help me to understand the failure message?
https://travis-ci.org/travis-ci/travis-build/jobs/558975609#L746

I believe docker is preinstalled, no need to install anything.
https://docs.travis-ci.com/user/docker/
https://travis-ci.org/native-api/test_travis/builds/565893910
And I really don't see why this PR attempts to add it.

https://docs.travis-ci.com/user/reference/xenial/#third-party-apt-repositories-removed

By "I don't see why", I meant because Docker is completely irrelevant to Python's default version thus to this PR's topic.

Our only failing test is Travis::Build::Script::Python behaves like a bash script full docker container execution runs and passes.  Do you believe this is not a useful test?

Docker tests are run in travisci-ci-garnet container which is Trusty-based. And 3.7 is not available for Trusty.

Docker tests are run in travisci/ci-garnet container which is Trusty-based. And 3.7 is not available for Trusty.

Thanks @native-api  Where would I fix that test given that Trusty is EOL?


Adding tooltips to graphviz representation

While working on https://github.com/dask/dask/issues/7919, I was looking for ways to represent the layer_type attribute in the graphviz representation. I stumbled upon the tooltip - Tooltip (mouse hover text) attached to the node, edge, cluster, or graph.
This opens up many doors to add more information :rocket:
Demo
(image shows the layer_type as the tooltip)

So, on hover, we can display any kind of information we want per Layer. We can use this discussion thread to discuss what information would benefit users when they see a Task Graph.
Idea
We can show the attributes stored in collecton_annotations in the tooltip.

This could be shape, dtype, chunksize, type, chunk_type (layer_type too) for Dask Arrays.
This could be npartitions, columns, type, dataframe_type, series_dtypes (layer_type too) for Dask DataFrames.
Anything else.

/cc @GenevieveBuckley @martindurant

I have opened a Draft PR where I will be working along with this discussion.
#7973

NOTE
This would only work for SVG outputs.
PNGs and PDFs are not supported.

So if we save out an .svg file, it retains the hover tooltip information?
If I remember correctly, the high level graph visualizations are saved by default as .pdf (and the low level graph visualizations are saved as .png, but we're not changing anything about the low level graphviz stuff). Perhaps we should change the high level graph default to .svg? It's worthwhile checking to see if most operating systems will be able to display svg files if double-clicked, without having to install other software.

So if we save out an .svg file, it retains the hover tooltip information?

Yes, true. The tooltips are stored in the .SVG image. If you open the SVG in a browser later, it would still possess the tooltips.

If you open the SVG in a browser later, it would still possess the tooltips.

I would not be certain of this across OSs

If you open the SVG in a browser later, it would still possess the tooltips.

I would not be certain of this across OSs

I have a linux machine and a mac we can use for testing if need be. I'm sure we can find someone with a Windows machine if need be, too.

My personal preference would be .svg if it's easy to open on all operating systems. If it isn't easy, then I'd prefer .png to match the default format used to save the low level graph visualizations. It's also worth double checking that default filenames used to save the high & low level graphs are not the same, and adequately describe which is which.


use host network

It would be really great if there would be options to connect to the host system. Like via NAT or briging.

Duplicate of #358


Replace standard with prettier

Coverage remained the same at 100.0% when pulling 6a125ca55c1887f14f3bb5c978f0c4d1ad211b3e on prettier into 6a49a23821e74bcbdcade6261897c77cad9b0d42 on master.


New utilities package: Set

This brings two reactive data structures: ReactiveSet and ReactiveWeakSet. Both are implemented as classes, but also have primitive function equivalents.
The way they work is very similar to object/array structures in solid-proxies, maybe with an exception of using a class instead of a proxy. But I needed this for different project, so I've build this instead of waiting for sets to be implemented there :P
An snippet of usage:
const set = createSet([1, 1, 2, 3]);

// listen for changes reactively
createEffect(() => {
  set(); // => [1,2,3] (reactive on any change)
  set.has(2); // => true (reactive on change to the result)
});

// modify like a normal Set
set.add(4);
set.delete(2);
set.clear();

The readme

Allllrrrrright, time to merge this thing :)

Nice!

I just noticed this implementation! Great job! Looks like we went with quite similar approach. Would you be up for merging it into 1 version (feature complete) so that I could remove them from solid-proxies? 😊

@Exelord Do you mean adding reactive Object and Array right? Can you make an issue for it so it's more clear?

That would be great as well! Actually I meant tweaking a bit your current implementation. I think I found a few issues that I caught in solid-proxies. Once they are equal, I would be happy to deprecate my proxies.
Also I have some thoughts on the TriggersCache. Its optimization for removing unused signals does help with memory consumption but it trades off operations performance. I was thinking about that, but then I realized it is more common re-access the property rather than access it once in a life time. In that case not cleaning up the signals would perform better inline with how the implementation would look like while using normal signals (without Map). Is that something you would reconsider?


[FLINK-6284] Correct sorting of completed checkpoints in ZooKeeperStateHandleStore

In order to store completed checkpoints in an increasing order in ZooKeeper,
the paths for the completed checkpoint is no generated by
String.format("/%019d", checkpointId) instead of String.format("/%s", checkpointId).
This makes sure that the converted long will always have the same length with
leading 0s.

LGTM +1

Thanks for the review @StefanRRichter. Merging this PR once Travis gives green light.


Fix assemNum parameter for uniform mesh reactor.

What is the change?
When creating a new uniform mesh assembly from a sourceAssem, copy the assemNum parameter over as the first operation.
Why is the change being made?
Previously, the assemNum was copied over much later in the process of creating a homogenized assembly, which resulted in the individual blocks within the homogenized assembly having a bad assemNum parameter and name.

Checklist

[x] This PR has only one purpose or idea.
[x] Tests have been added/updated to verify that the new/changed code works.

[x] The code style follows good practices.
[x] The commit message(s) follow good practices.

[x] The release notes (location doc/release/0.X.rst) are up-to-date with any important changes.
[x] The documentation is still up-to-date in the doc folder.
[x] The dependencies are still up-to-date in setup.py.

This fix works! 🥳
But we are doing some internal testing. I will merge when complete!


support for vIOMMU with mmap()

libvfio-user doesn't work with a vIOMMU that remaps GPAs to IOVAs and wants to access guest RAM via mmap() (it works fine if guest RAM is accessed via vfu_sgl_read and vfu_sgl_write).
From discussion with @jlevon and @mnissler-rivos on libvfio-user Slack (https://libvfio-user.slack.com/archives/C01AFGCSPTR/p1696411059172959):

IOMMU PT/identity DMA: works fine: incoming IOVAs are the same as GPAs
IOMMU remapping: doesn't work: IOVAs need translation somehow into the underlying GPA prior to the libvfio-user dma code
ATS on top: mostly about endpoint caching translations. would need some version of this between qemu/libvfio-user for any hope of performance
PRI on top: mostly about demand paging: same?
PASID on top: no idea!

I've spent some more time looking into IOMMU/ATS/PRI/PASID and wrote some exploratory code. With this background, I think I have a better grasp now on what would be needed to support these features:

Conceptually, the vfio-user server operates in the virtual I/O address space created by the IOMMU it is subordinate to. This means that all DMA operations are in terms of IOVA. This includes not only DMA read/write, but also MAP/UNMAP. As a result, when IOMMU mappings get created and torn down, the corresponding IOVA regions need to be propagated via VFIO_USER_DMA_{MAP,UNMAP} to the server. This is already implemented (but lacking PASID support, see below).
ATS primitives can be mapped to VFIO-user as follows:

A translation request can be satisfied simply by checking whether the server has a DMA mapping for the requested IOVA. At the hardware level, this returns the physical address if a mapping is present, which is mainly used to generate DMA requests with pre-translated addresses. Since the VFIO-user server operates in the IOVA address space, the physical address isn't really needed AFAICT. In case I am wrong and it is actually needed, it could be supplied by the client as an additional parameter when creating the DMA mapping.
Invalidations are represented by VFIO_USER_DMA_UNMAP. There is an interesting detail here where the invalidation completion is only OK to sent once the server has made sure to finish/abort any ongoing DMA accesses to the mapping. This is probably more relevant in hardware than it is for VFIO-user, but this may manifest in race conditions in the protocol that we need to think about.
PRI page requests are a concept that currently doesn't exist on the VFIO-user side. It will require a new server->client message. The expectation is that the client will set up DMA mappings for the requested address(es) if possible.
In hardware, page request group response messages (i.e. page request completions) are used to signal that a bunch of page requests have been processed. Since we have explicit VFIO_USER_DMA_MAP commands, these are probably sufficient to inform the server of the fate of a previous page request, at least for successful page requests. Something may have to be done about rejected page requests, perhaps by generating VFIO_USER_DMA_MAP commands with the protection flags zeroed out to indicate that the corresponding IOVA is in fact not accessible.

PASID support is on top of all the above (although on-demand paging with PRI is typically used with PASIDs). Each PASID basically refers to a separate IOVA address space. This means that all operations dealing with IOVAs need to support optional annotation with a PASID. This obviously affects VFIO_USER_DMA_{READ,WRITE}, but also VFIO_USER_DMA_{MAP,UNMAP}. This either means breaking changes to the existing protocol message format to add the PASID field, or introducing extended versions of these messages that include the PASID field. On the libvfio-user side, the DMA controller would be extended to manage mappings not only for a single I/O address space, but one independent address space per PASID.
Additional non-trivial work to integrate the above with qemu's vfio-user server and client will be necessary, including, but not limited to:

As far as I'm aware there is no support for page requests in qemu at the time of writing this. It would perhaps make most sense as an additional IOMMU operation, or perhaps a new function on AddressSpace for device models to call.
Depending on the previous design decision, the vfio-user server may have to provide an IOMMU implementation after all (it currently just adds/removes memory regions in response to DMA map/unmap commands).
Neither can I find PASID support in current qemu, although it seems a patch has been proposed years ago.
I expect additional work in qemu IOMMU implementations to wire up PASID / page request support.

So much for what I've been able to identify. Given the absence of device models that make use of ATS, I don't plan to implement the above in the predictable future, but wanted to dump it here in the hope that it'll be useful to whoever may pick this up in the future.

Thanks for the write up. Just a couple of comments:

There is an interesting detail here where the invalidation completion is only OK to sent once the server has made sure to finish/abort any ongoing DMA accesses to the mapping. This is probably more relevant in hardware than it is for VFIO-user, but this may manifest in race conditions in the protocol that we need to think about.

The server is required to handle this:
https://github.com/nutanix/libvfio-user/blob/master/docs/vfio-user.rst#id5

PRI page requests are a concept that currently doesn't exist on the VFIO-user side. It will require a new server->client message. The expectation is that the client will set up DMA mappings for the requested address(es) if possible.

Unless I'm missing something this looks quite straightforward.
Regarding the rest, I'll try to extend this once I've fully understood it ;).

FWIW, I have uploaded my proof-of-concept code at https://github.com/mnissler-rivos/libvfio-user/tree/ats
It successfully handles PASID-enabled DMA read/write against pages that get dynamically requested. Tested with a standalone implementation of qemu's edu device model (with some extensions to add PASID support) as the server and a heavily hacked qemu client based on oracle's qemu github repo.
Just sharing what I have in case it is useful - this code isn't suitable for merging, but intended to supplement my wall of text above.

Took me a while to read this. One question: I'm not quite clear why we would/should implement something like PRI
Since we (server) need to have mapped access to memory ahead of time anyway via DMA_MAP, why does it make sense for us to be demand-faulting in like this?

You will probably know/understand some/most of the below, but let me start at the beginning to present a coherent line of thought:
The premise of placing IOMMUs in the DMA data path is that devices actually don't have access to the entire system memory, but the OS can control what they have access to by programming the (v)IOMMU. Even then, the device/VFIO-user server can still pretend that it has full access to its I/O virtual address space: The host can perform DMA_MAP/DMA_UNMAP operations as the guest kernel manages the IOVA and reprograms the vIOMMU. As long as the device only hits accessible memory, everything will work fine (qemu actually supports this to some extent for vhost already IIRC).
The problematic point with the above approach is that we need to proactively set up IOMMU mappings for all memory that the device/VFIO-user server may potentially access. There are two issues with that:

Device-accessible memory is effectively pinned and cannot be paged out (by the guest kernel), potentially causing memory footprint issues
Shared Virtual Addressing (SVA), i.e. sharing an address space between the device and a user space process, is hard if not impractical to support since you'd have to proactively map most/all user space process memory.

Both cases are addressed by demand-paging as enabled by PRI.
So much for what problems PRI support can solve. Whether these problems are worth addressing in VFIO-user is a separate question though. I suspect that right now, setups that employ a vIOMMU are pretty rare, and VFIO-user servers that expose devices which actually would benefit from demand-paging / PRI are largely non-existent (as a side note, my own interest is motivated by the desire to model/mimic real hardware as closely as possible, but I appreciate that this isn't a primary goal for the VFIO-user protocol or libvfio-user project). The situation might change in the future though: If/when more hardware starts adopting ATS/PRI and SVA becomes more prevalent, sooner or later folks will want to use this within their VMs, and thus PRI/ATS will become relevant for VFIO-user.
Given the above, IMO it's perfectly fair to keep this open as a future enhancement, to be picked up if/when a more substantial use case appears.
That said, there is one angle that I think may warrant consideration now: PASID support in the protocol. DMA_MAP, DMA_UNMAP, DMA_READ, DMA_WRITE operations must convey the PASID they're targeting, which will necessitate additional protocol fields. I understand the protocol is still unstable, so we might want to add PASID fields now to avoid going through an awkward protocol upgrade at a later point. Then again, I don't see PASID support in VFIO at this point, so there's also an argument to wait for PASID support to appear there.

setups that employ a vIOMMU are pretty rare
desire to model/mimic real hardware as closely as possible
Specifically setups that do other than identity mappings. If we have identity mappings (and of course no PASID etc) everything already works.

desire to model/mimic real hardware as closely as possible

This is a worth goal - and potentially especially relevant in cases where the server side is acting as a proxy for some kind of real hardware underneath.
What's foxing me about PRI here is really about who is actually acting as the IOMMU, I think.
Presumably qemu+kvm itself still has all of guest memory mapped into its process space right? What I'm asking is whether we should do the same on the server side. That is, is the IOMMU effectively emulated in qemu or in libvfio-user - does that makes sense?
This isn't arguing for or against actually having ATS etc. support, just the nature of the implementation.

PASID support in the protocol

I would like to see that for sure.

What's foxing me about PRI here is really about who is actually acting as the IOMMU, I think.
Presumably qemu+kvm itself still has all of guest memory mapped into its process space right? What I'm asking is whether we should do the same on the server side. That is, is the IOMMU effectively emulated in qemu or in libvfio-user - does that makes sense?

Ah, thanks for drawing attention to this angle. I had actually thought about this and remote IOMMU approaches are definitely possible. I previously ended up concluding that the IOMMU probably makes more sense on the qemu side, but failed to mention any of this... Here are some thoughts:

I think it's safe to assume that the IOMMU would be on one side only - running two in series seems like a bad idea.
If the IOMMU is on the server side, the guest kernel must still be able to program it. So you end up running an IOMMU control interface over the VFIO-user socket?
If you run a full IOMMU on the server side, then it would have to be a separate IOMMU from the one that exists in qemu. Doable, but different from how typical hardware topologies look like. Or perhaps you could hook up notifiers for vIOMMU operations and then proxy them over to the remote IOMMU implementation. Doable, but probably not trivial given the need for I/O fences when invalidating for example.
One purpose of IOMMUs is enforcing security boundaries, i.e. making sure devices can't access any memory that they haven't been granted access to. This matches up nicely with the VFIO-user process boundary if the IOMMU is on the qemu side, but not when the server process can still access all memory and just performs local access checks. You'd still have to trust the server process to not be malicious. Whereas with the IOMMU on the qemu side, only allowing mmap()ed access selectively and/or using message-based DMA, you are in a much better spot. I expect some VFIO-user protocol hardening would still be needed for the security story to make sense, but it seems plausible.
The ATS design already includes the concept of an Address Translation Cache (ATC) on the device side to maintain a mapping of IOVA -> (G)PA to improve performance. For servers that want to access memory directly, it would probably be more natural to imitate that (DMA_MAP operations would have to indicate physical addresses in addition to IOVA), and then handle translation on the server side and access mmap-ed guest physical memory directly.

This is a worth goal - and potentially especially relevant in cases where the server side is acting as a proxy for some kind of real hardware underneath.

This might be feasible in the long run, but would certainly require more plumbing as the kernel would somehow have to communicate page requests from the device to the server (as opposed to just inspecting the address space and deciding that there's nothing there). Something like a page fault handler for hardware faults.


Expose undownload API

As discussed on fritter and probably elsewhere, it'd be nice if there was an easy way to "undownload" certain files from the local storage.
A great use case for this is removing old files from your cache that you're unlikely to use anymore.
I believe a useful API would look like hyperdrive.undownload(path, version?) where path is the path to the file or folder that should be removed, and version is optionally the point in history to remove the file from.
If version history is supplied, go through the dat history from that point and find the latest time that the file or folder was modified. Then use this.content.undownload to remove it (or the folder contents) from the content feed.
If version history isn't supplied, traverse the entire history and undownload every instance of the file.
This would be useful to have in the DatArchive API in Beaker in the future so that applications could provide users with control over which data should be stored locally.

Err, looking at the existing API it should probably be hyperdriv.checkout().undownload(path) instead. Where you undownload all the content up to the currently checked out version.

Sorry for the spam, but would a PR be welcome for this functionality?

Name
We should call it something else because we have hypercore.undownload() which simply means cancel the active download -- which is also any API we could use. Maybe .clearCache() ?
Perms / Risk
Do you think there are any security policy or management concerns to consider? I suppose the concern applies to download() too: are there times when any app having access to these APIs could just be an enormous headache, because of rampant download/upload calls?
Since we don't really have a good answer for download() then maybe we shouldn't let it hold up undownload() either for now.

hypercore.undownload() which simply means cancel the active download

Whoops, I totally misunderstood what that does. Shoulda looked at the hypercore docs instead of skimming the hyperdrive code. 😅
Regarding the name, I still like undownload because it's obvious that it's the opposite of hyperdrive.download(). Though I agree that clearCache or similar would be better in order to stay aligned with the hypercore naming conventions.
I agree we shouldn't need to be any special concerns for this for now since there aren't any for download(). I could see there being trouble if multiple applications are trying to do something fancy with the same archive, though.
As an aside, I think this will be useful to have for Beaker itself so that you could include a UI for managing cached content at a more fine-grained level.

Fair point about the name. I'll think about it. @mafintosh do you have any preference on this?
I agree that a Beaker UI for managing this would be good. Version & history management is really minimal right now, and a lot of that has to do with versions being really early. We know there's going to be more complexity once multiwriter lands, and we know we want tagged versions, so it feels premature to dive into that now.

In hypercore we use .clear(range) for clearing downloaded data. Using that name or clearCache sounds good to me if this actually clearing data

Kinda off topic, but are multiwriter dats still going to have a linear history from whatever the current checkout is?

@RangerMauve ish. Multiple linears that weave together. The version number becomes a version vector.

Multiple linears that weave together. The version number becomes a version vector.

I gotta say, that's cyberpunk as heck. 😂
So, would a PR for clearCache(path?) be welcome, or would y'all prefer to do it yourselves?

@RangerMauve 😎 aw yeah
Okay let me think. The question of how to clear historic versions (or not) seems important. I feel like it'd be useful to clear the cache of a file/folder/all for...

...all history and current
...all history but keep current
...a specific version

Using archive.checkout(v).clearCache() satisfies the last one but not the first two.
Perhaps we support the versioned cache-clear, PLUS we add two params for specifying whether the clear should delete the given version's cache and/or the previous versions:
archive.clearCache('/', {current: true, historic: true})

These would both default to true. If current == false then the given checkout is preserved. If historic == false then previous versions are preserved. Future versions are never deleted.
That seem decent?

I'm not a huge fan of having four different types of clearCache, but after thinking about it more, I think it'll be more annoying to not have these two flags. Mainly because AFAIK it's hard to get the current (or previous) version number for a given file without traversing history, so you won't be able to easily represent "delete everything from the previous version and before".

@RangerMauve Yeah that's exactly what went through my head as I wrote it. One alternative is to have it work like hypercore's clear() where you pass in revision numbers/ranges. I'm not sure how that would work with multiwriter though, so I'm inclined to just roll with this

@RangerMauve I'm 👍 to a PR for this, or I can put it on my backlog. The changes will need to be made in https://github.com/beakerbrowser/beaker-core

Cool, I was thinking it'd make sense to add it to hyperdrive first (since it'd be useful for non-beaker applications). I'm not sure when I'll be able to work on it, so let's race to see who has time first. :P

@RangerMauve oh that makes sense. In that case, I'd ask @mafintosh what function signature he wants for this, and then we can do our own thing with Beaker's web API.

@RangerMauve Dats that are being seeded have an "auto download latest" policy set. We can have the clear be a noop for latest but work on history

Ok, one more. If you created the dat, should we allow things with read access clear local storage? (since that could potentially delete data forever if you aren't using hashbase)

@RangerMauve probably not, for now.
The more I think about this, the more I feel local cache control as an API is wrong (including the existing download() API). The intention, I think, is to control

what's reliably available, and
what's fast to access.

The former intention needs something more sophisticated than local cache control, because (as you mention) over time we'll be integrating remote services which provide availability to data even the dat is locally-owned. We'll effectively begin doing what https://github.com/KonstantinSchubert/zero does. For the latter, I feel like we need more time to understand apps & dat before we can design good APIs to solve.

@pfrazee I agree with waiting to understand apps more. In the meantime it might be better for Beaker to provide more fine-grained control over what data is downloaded.
Would this functionality still be useful for hyperdrive? I think it would be useful for people attempting more advanced use-cases using the cli and also for making life easier for Beaker when the time comes.

@RangerMauve I guess the question I'd ask is, what are we trying to accomplish right now? If the answer is to reduce the cache usage, then maybe we ought to be talking about adding more sophistication to Beaker's internal cache management rather than pushing it into userland

I'm coming here from talking to people on IRC. I think the main concerns I saw were related to the CLI and not even beaker-specific. They wanted to be able to undownload parts of dats that the have saved locally which they no longer needed.
In the beaker space, I don't think I've seen as much buzz, but people wanted to clear data from their device that would be replicated elsewhere. I think that this would be better left to be figured out by applications that manage your files.

Hi! Any further thoughts about this? It seems like what would be useful is an equivalent of hypercore.clear() with a way of specifying "clear all versions of file at this path" or "clear everything but the latest version of the file at this path". This would be really useful for replicating hyperdrives with frequently updated files, so that peers do not need to keep a complete history of old versions of a file in their local cache.

I'm interested in this feature as well. Any updates on how to clear specific files or directories?

You can clear a file by getting it's stat object to get the start and end index for where it is in the content feed. From there you'll want to use some undocumented APIs to get the content feed (this can break any time) and use content.clear(start, end) to for that file.
Here's an example of me getting the stat for the file, getting it's content feed, and checking if a range has been downloaded: https://github.com/RangerMauve/hyperdrive-is-downloaded/blob/master/index.js#L4
This should be easy enough to adapt to invoke .clear instead of .has.

I don't have the time at the moment to push this through, but I think this is a super important feature to have and I think it's something that was present in the past.

I opened a PR to solve this issue


Add an error message if one tries to share a measurement file with a use who does not exist

Currently, the user does not get a notification when he tries to share a measurement file with another user and the other user does not exist. Currently, the app simply proceeds as if that user existed and was added. An error message should be shown if the user does not exist.

Done as part of the feature-branch dealing with issue #88:


chore(dev): remove obsolete image reference to superset-websocket + fix minor typo

chore(dev): remove obsolete image reference to superset-websocket + fix minor typo
SUMMARY

Remove obsolete image reference to superset-websocket in docker-compose.yml
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Before

After

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

Thanks for the PR!


Issue #33: Moved source scss, template includes (assets) and source vendor files (bootstrap scss) to src folder.. Updated paths (to remove template includes from static folder).

Rebased main branch again.


Add other requirement formats

This ticket should track the general progress for adding additional requirement formats.

[ ] poetry (project.toml)
[ ] poetry (poetry.lock)
[ ] PEP 518
[ ] Pipenv

... Well I'm watching this one for sure!

We might want to use poetry2nix to parse poetry requirements.
Of course this raises the question, if mach-nix is still needed at all since poertry2nix is able to complete the build on its own.
But:

poetry2nix probably doesn't handle non-python dependencies as well as mach-nix. (It doesn't take advantage of nixpkgs)
it isn't able to automatically mix in extra packages into the environment
It probably wouldn't integrate well with some other mach-nix features which I'm not aware of right now.

It would be super great to support PEP-517/PEP-518 the way poetry2nix does - so one can develop a python project the usual python way, but add a flake.nix into the directory and get toolchain-level reproducibility and optimal docker images essentially for free

An extractor for those could maybe look something (relatively straight forward) like the following:
{
  pyproject = builtins.fromTOML /path/to/pyproject.toml;
  pname = pyproject.tool.poetry.name;
  version = pyproject.tool.poetry.version;
  meta.description = pyproject.tool.poetry.description;
  meta.homepage = pyproject.tool.poetry.homepage;
  meta.license = pyproject.tool.poetry.license;
}


Vesting to move tokens to account derived from receiver and vesting to allow gov delegation

it is expected that team vested people are bening and should be able to use future tokens to vote.
so allow to setup proxy for vested account as desired by host runtime config to that account.
which will be usual voting

I do not understand the request. Vested token can be used for voting. There is no need to have proxy.

https://github.com/paritytech/substrate/blob/033d4e86cc7eff0066cd376b9375f815761d653c/frame/conviction-voting/src/lib.rs#L625


HTreeMap.replace destroys the map

I work on a project where we used mapdb 1.0.8 and I'm now trying to upgrade to version 3.0.4.
We use mapdb as a temporary data storage because we wanted a solution that purely runs on the JVM.
Temporary means: We load datasets from files and store them in the db. Then we request a backend for the items we just imported, and update those items from the backend with the data we just imported. After that, we need to replace the items in the mapdb with the merged items, before we upload the result back to the backend. After that, the db will be cleared&closed until we get the next file to import.
However, I'm creating a database and a hashmap:
DB db = DBMaker
                .tempFileDB()
                .transactionEnable()
                .closeOnJvmShutdown()
                .fileDeleteAfterClose()
                .make();

HTreeMap<String, Item> items = db
                .hashMap("items")
                .keySerializer(Serializer.STRING)
                .valueSerializer(ItemSerializer.getInstance())
                .createOrOpen();

Then I have a method that inserts/replaces values into the map:
public Item add(Item item) {
        final String key = item.getID();
        final Item previous = items.get(key);
        if (previous != null) {
            items.replace(key, previous, item);
        } else {
            items.put(key, item);
        }
        return previous;
    }

Replacement simply doesn't work any longer, with 1.0.8 it worked (code was a little different, I just used put(..) in both cases and it worked and just replaced the old value). Now if I step through it in the Debugger and have the items map "open", after the .replace(..) it says Unable to evaluate the expression Method threw 'java.lang.RuntimeException' exception..
After that I still can get the key-index from the map, but as soon as I try to iterate over the items or even get a specific one, the map just returns null.

Would you post more information? I need

stack trace when you try to reopen the map* ItemSerializer.equals
method content (to make sure that equals works always)* content of
Item, is it immutable?* what happens if you use on-heap db instead of
file?

{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c5
5493e4bb","name":"GitHub"},"entity":{"external_key":"github/jankotek/
mapdb","title":"jankotek/mapdb","subtitle":"GitHub
repository","main_image_url":"https://cloud.githubusercontent.com/ass
ets/143418/17495839/a5054eac-5d88-11e6-95fc-
7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent
.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-
b52498112777.png","action":{"name":"Open in
GitHub","url":"https://github.com/jankotek/mapdb"}},"updates":{"snipp
ets":[{"icon":"DESCRIPTION","message":"HTreeMap.replace destroys the
map (#825)"}],"action":{"name":"View
Issue","url":"https://github.com/jankotek/mapdb/issues/825"}}}

There's never an Exception being thrown that I could catch, so I don't have a stacktrace, I'm afraid. That's the only line I get from the debugger, because it is unable to analyse the object after the replace.
An Item is not immutable per-se, it's a rather complex thing containing multiple inner maps and flags. The Serializer serializes the object to a JSON-String. I didn't implement equals on the Serializer, is that required?
When I use heapdb, at first glance I'd say it is a bit strange: The item I get back as "previous" from the map is in fact the same instance as the one I'm trying to write into it. With the filedb it was 2 different instances, and that's what I'd expect since I thought they were serialized to json in between?

Alright, that was it, there was a case missing in the deserializer. All the unit tests work now again, therefore the upgrade was successful.
Thanks for your help.


Day view intervals start at wrong time

The day view intervals are starting at +/- GMT from UTC.

fixed in v1.0.0-alpha.12


Processing events is not allowed as a filter but is reported as a state

A resource can have processing_events as status. When this status is clicked the api will return a validation error.
{
    "message": "Invalid request: Filter validation failed while parsing status: 1 validation error for ContainsFilterResourceState\nfield -> 1\n  value is not a valid enumeration member; permitted: 'unavailable', 'skipped', 'dry', 'deployed', 'failed', 'deploying', 'available', 'cancelled', 'undefined', 'skipped_for_undefined', 'orphaned' (type=type_error.enum; enum_values=[<ReleasedResourceState.unavailable: 'unavailable'>, <ReleasedResourceState.skipped: 'skipped'>, <ReleasedResourceState.dry: 'dry'>, <ReleasedResourceState.deployed: 'deployed'>, <ReleasedResourceState.failed: 'failed'>, <ReleasedResourceState.deploying: 'deploying'>, <ReleasedResourceState.available: 'available'>, <ReleasedResourceState.cancelled: 'cancelled'>, <ReleasedResourceState.undefined: 'undefined'>, <ReleasedResourceState.skipped_for_undefined: 'skipped_for_undefined'>, <ReleasedResourceState.orphaned: 'orphaned'>]), values provided: {'status': ['available', 'processing_events']}"
}

The call:
http://<IP_ADDRESS>:8888/api/v2/resource?deploy_summary=True&limit=20&filter.status=available&filter.status=processing_events&sort=resource_type.asc

Isn't there an enum with all possible values?

processing_events does not exist in iso5 anymore. So:

Remove it from the query
Remote it from the tooltip
Code should still accept it

Hey team! Please add your planning poker estimate with ZenHub @andraskvr @Stijnkool @wouterdb

and remove it from the filter select box


Add  to sensitive tags in collapse_whitespace_filter.

Fixes #1772

Nice, thanks!

@oschaaf do you know why CI is failing?  I re-started it once, but both attempts ended in:
ld/apache/incubator-pagespeed-mod/third_party/giflib/src'...
No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received
The build has been terminated
I'll try a third time. If that fails we can I guess patch/build/test and then merge, but I wanted to see if you thought the CI was failing like this generally and if we should so something about that.

@jmarantz I noticed the same with another PR lately, but have not investigated yet. perhaps one of the dependencies broke, I’ll take a look later today

So the test failed on But do proxy content if the Content-Type header is present..
This was very reproducible because a required script that serves test responses was not running on modpagespeed.com. I recently rebuild the machine from scrap to simplify some things, and forgot all about that script. Restarted the CI job, expect this to be fixed.
(Though sometimes CI may still flake because of the tests requiring an amount of time that is pretty close to the timeout limit.)

@jmarantz +1 for merging
With regard to giflib, that started failing when sourceforge starting migrating
datacenters. I already attempted a fix [1] earlier because the previous location started
responding with 404 ..
But apparently the git repo we now point to times out every now and then, ugh.
[1] https://github.com/apache/incubator-pagespeed-mod/commit/8196fc74847e85085fa6f310d658842261acb329


Missing documentation on BotService connections

Bicep version
0.11.1
Describe the bug
Trying to add a Generic OAuth 2 connection via Bicep but there's nothing to show what the value should be for the serviceProviderId.
To Reproduce
There's no example of the properties for each connection similar to how there are examples for the Channels. https://learn.microsoft.com/en-us/azure/templates/microsoft.botservice/2022-09-15/botservices/connections?pivots=deployment-language-bicep#connectionsettingparameter

Can you try updating bicep? The latest version is 0.18.4 and there are two separate installations - one for VS code and then the bicep CLI is separate.

As @danscrima mentioned, you can open the dev tools (f12) and look at the network tab when you click 'Add OAuth Connection Strings' and it will show all of the providers and their Ids. Here is an example of the bicep template as well.
resource botServiceOauthConnection 'Microsoft.BotService/botServices/connections@2022-09-15' = {
  name: connectionName
  parent: botService
  location: 'global'
  properties: {
    serviceProviderDisplayName: 'Azure Active Directory v2'
    serviceProviderId: '30dd229c-58e3-4a48-bdfd-91ec48eb906c'
    clientId: aadAppClientId
    clientSecret: aadAppClientSecret
    scopes: 'User.Read'
    parameters: [
      {
        key: 'tenantId'
        value: aadAppTenantId
      }
      {
        key: 'tokenExchangeUrl'
        value: '<your token exchange URL>'
      }
    ]
  }
}


PR for issue #5169 "right-click and drag in plots broken"

only works if i press both buttons at the same time.


$$ Syntax not accepted

Versions

sequelize: 4.43.0
sequelize-typescript: 0.6.8
typescript: 2.9.2

I'm submitting a ...

[x] bug report
[ ] feature request
Actual behavior:
Whenever i am trying to use $$ syntax to add a where clause on joined table, it throws type error on the where clause.
Expected behavior:
It should not throw error and actually add a where condition on the joined tables.
Steps to reproduce:

Make a one-many Relationship.
include with required: false
add a where clause in main query using $$ syntax
type error

http://docs.sequelizejs.com/manual/models-usage.html#top-level-where-with-eagerly-loaded-models
Related code:
A.findAll({
    where: {
        $or: [
            {"$B.userId$" : 100},
            {"$C.userId$" : 100}
        ]
    },
    include: [{
        model: B,
        required: false

    }, {
        model: C,
        required: false
    }]
});

Hey @piyushkantm, thanks for bringing this up. The reason why it’s not accepted there is that the where options are restricted to keyof YourModel. Since the $A.column$ cannot be types and is more an exception for the rule, we decided to do it like this and offer a workaround instead:
{['$B.userId$' as any]: 100} should do the job. What do you think about it?
Hope this helps!

Hi @RobinBuschmann . Thanks for replying.
You're right. I was using any to avoid type errors... But don't have anything better in mind to overcome this use-case..
It'd be nice though to have a demo code for this on readme. It could help someone save some time :) Let me know if i can help in any way...

@piyushkantm Thanks for offering help: Yeah, any pull request adding this to the readme is appreciated :)

Yup, i would try to lend my hand over this weekend..

Closing this due to inactivity


Solarized Dark Theme makes certain command line arguments invisible in PowerShell

Does this issue occur when all extensions are disabled?:  There everything is fine. It's just the Powershell that has this bug. This bug appeared when I updated today VS Code to the latest version. In the morning when I worked with the older version everything was fine.

I'm using zsh on my pi4 with latest release and have the same issue with solarized dark.
Everything in solarized light works fine though.

I wonder if something changed with the Solarized Dark theme even in the editor itself. Pretty sure that my code now looks way more grey and less white than before:

Can repro, pwsh outputs these as "bright black" (terminal.ansiBrightBlack) which is #002b36, the same as editor.background. A similar thing happens in Windows Terminal:

I'm hesitant to change the colors here because solarized uses a fixed palette. Also this is where the recent changes came from https://github.com/microsoft/vscode/pull/142670
You can set "terminal.integrated.minimumContrastRatio": 4.5 to workaround the problem. @isidorn @daviddossett @meganrogge FYI I'm thinking of setting this to 4.5 by default this iteration, I was originally worried about performance overhead but I've been running it since it was introduced and never noticed any issues.

Marked as a feature request to remember to add to release notes

Changing the default to be have a good contrast makes good sense to me.
I just do not understand how this could affect performance :) I guess you scan colors in some way?

I didn't get that performance concern as well. And changing contrast works totally fine in my case, though I only changed it to 2 instead of 4.5. Powerline fonts and symbols will have some shade if the contrast ratio is too high.

Here's a screenshot with contrast ration = 2.

Just to confirm, the contrast ratio fix is only for terminal, it will not the look of the source code, right?
I understand that Solarized is defined with certain fixed colors, but my codebase now looks like it’s all comment and no code, due to being all gray. I suppose there is no other way to go back to the old theme than pull it from Git history and create a custom extension based on the old colors?

@Cellane yes that's right it this change will only affect the terminal.

Why the hell changing the default value for terminal.integrated.minimumContrastRatio?! If a terminal color scheme doesn't have enough contrast, it's the problem of this very color scheme. Please, stop messing our terminal colors.


Circom Adapter for Lambdaworks Groth16 Backend

Description
Converts Circom & SnarkJS generated *.r1cs.json and witness.json files into Lambdaworks-compatible QAP and witness instances.
Type of change

[X] New feature

Checklist

[X] Unit tests added
[X] This change requires new documentation.

[X] Documentation has been added/updated.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (ff2bdb7) 94.59% compared to head (784e78e) 94.61%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #781      +/-   ##
==========================================
+ Coverage   94.59%   94.61%   +0.01%
==========================================
  Files         144      146       +2
  Lines       33838    33976     +138
==========================================
+ Hits        32008    32145     +137
- Misses       1830     1831       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Amazing work


mixed-type arrays

I have an array of mixed types but only the LIFO type is used instead of object (I'm using the maven plugin). Pretty sure that the online version actually generated an Object but can't replicate it. How can this be done please? (If I need to write additional generators what is the best approach to handle mixed type arrays and also to generate List<List>?)
Custom code (what is the best approach with least code)?
Use a JSON schema (I'm currently generating from JSON)?

Hi
Quite honestly, I'm having trouble understanding what's the issue.
Perhaps it would be possible to provide a minimal working example that demonstrates/reproduces the issue (like for example in #1152) ?

No issue now. Using a JSON model I could only get a primitive type generated instead of java.lang.Object (for mixed arrays). I was able to achieve what I needed to achieve using a JSON schema with $ref as input and changing the maven plugin to jsonschema. Thank you for writing this plugin. It is a huge help.

From: unkish<EMAIL_ADDRESS>Sent: 14 November 2020 11:14
To: joelittlejohn/jsonschema2pojo<EMAIL_ADDRESS>Cc: neuralsea-lab<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [joelittlejohn/jsonschema2pojo] mixed-type arrays (#1174)
Hi
Quite honestly, I'm having trouble understanding what's the issue.
Perhaps it would be possible to provide a minimal working example that demonstrates/reproduces the issue (like for example in #1152https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjoelittlejohn%2Fjsonschema2pojo%2Fissues%2F1152&data=04|01||ffe41e8a826a4a83408408d8888e7d18|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637409492843992862|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=q9oAjtTCVWVquyROCvCxX%2F0gPV%2FolJYJOzErBQrk1NM%3D&reserved=0) ?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjoelittlejohn%2Fjsonschema2pojo%2Fissues%2F1174%23issuecomment-727190426&data=04|01||ffe41e8a826a4a83408408d8888e7d18|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637409492844007838|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=qyclrFSo%2BvCTdGoKJEH4Ku66BukgWKijk5Ne%2BKefr48%3D&reserved=0, or unsubscribehttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAPQ26VMRUFOWP3XQS7EEXYLSPZRCHANCNFSM4TTYIEXA&data=04|01||ffe41e8a826a4a83408408d8888e7d18|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637409492844027804|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=2yl9Shmdsu3OzkzIcSAJYmvTw%2FA7IbTJYQ5I%2Bj6gHI4%3D&reserved=0.

Glad to hear that You have managed to find solution :+1:
Regarding plugin: all credits go to @joelittlejohn and contributors.

Thanks for helping out around here @unkish!


Improve GraphViz output by using named events

Including GraphViz/dot support is a very nifty idea, thanks!
The only issue I see is the lack of named events. The example at https://github.com/LennartHennigs/SimpleFSM/blob/master/README.md#graphviz-generation would be a lot more legible IMO if instead of "(ID = 1)" it said "press button". I understand this won't be possible out-of-the-box as events are defined by number rather than by name (usually through an enum).
However, what if we could pass something like an "event name array" to the SimpleFSM constructor (or through a setter)?
To your question – as you'd only need the names in the output, I was thinking if it'd make sense to pass them as a parameter to the getDotDefinition().
What do you think?

Sure, why not. I'd be a little less invasive.

Ok, thx.
A while ago I also considered to name them in order to be able to do something like this:
Transition transitions[] = {
  Transition('from', 'to, light_switch_flipped),
  Transition('to', 'from' light_switch_flipped)
};

Which would make defining transitions more readable and easier to understand.
But that would require some larger changes to the code.
I'll think about it a little more.
And add it to my todo list.

pass them as a parameter to the getDotDefinition().

The obvious downside of this approach is that you would need to redo the whole dot_definition that you are currently aggregating behind the scenes on add(). If you allow the transition names to be passed (optionally) to the constructor you'd only need to change the _addDotTransition methods.

I also considered to name them in order to be able to do something like this:

That would also be an option. However, I am not so sure it would really make the code more legible. If I name my enum values (i.e. transition ids) expressively enough, I get a very well legible transition definition.

Sigh, I feel like such a fool 🙈 I only just realized that I can produce more legible diagrams using the existing code. I just have to name my transitions. Feel free to close this event if you don't plan to pursue the idea of named events.


Search field is missing in the vm listing pages

Search filed is missing in the vm list pages (Service==> workloads ==> All VMs & Instances, Infrastructure ==> Virtual Machines ==> All VM & Templates)

@rananda This works in upsteam/master already.

@rananda Please retest, I see this present as well.  Close if not still an issue.

@dclarizio Working fine in the latest build, hence closing this issue.


Enable automatically using import hook and env variable

Problem
Right now, users need to import lovely tensors and monkey patch:
import lovely_tensors
lovely_tensors.monkey_patch()

Doing this in every notebook, training script, evaluation script, python console, etc. can become repetitive.
Solution
This PR allows enabling lovely tensors automatically without any modification to the code. All is needed is that the pip package is installed and that a variable LOVELY_TENSORS=1 is set in the environment:
python -c 'import torch; print(torch.ones(3))'
# tensor([1., 1., 1.])

LOVELY_TENSORS=1 python -c 'import torch; print(torch.ones(3))'
# tensor[3] x∈[1.000, 1.000] μ=1.000 σ=0. [1.000, 1.000, 1.000]

Notes
Adding the variable to the environment can be done in multiple ways:

inline just before calling python, as shown above
for a single terminal session, eg export echo LOVELY_TENSORS=1
for all terminal sessions by modifying .bashrc, eg echo LOVELY_TENSORS=1 >> ~/.bashrc
for a specific conda environment eg conda env config vars set LOVELY_TENSORS=1

If the env variable is not set, or is set to 0 or an empty string, the import hook does nothing. In this case, the user can still follow the old workflow or importing and monkey patching, this PR does not modify that behavior.
References
This way of enabling a feature using only an env variable and without explicit imports is inspired by the awesome package better exceptions.

Summary by CodeRabbit

New Features

Introduced a new hook functionality for the lovely_tensors library, activated via an environment variable.
Added a new file, lovely_tensors_hook.pth, to the package distribution.

Enhancements

Improved setup process with new classes and functions to manage the hook file during installation and build processes.

Bug Fixes

Enhanced error handling in the hooking process for better user guidance.

Before merging we should also document the feature in the README and maybe bump the version.

Wow @coderabbitai got really annoying, I'll disable it for now.
@baldassarreFe Thank you, I like this idea. Let me test it a bit, and I'll merge it.

Wow https://github.com/coderabbitai got really annoying, I'll disable it for now.

Well, it's very verbose but it's not wrong, mostly :D

I'll skip the suggestion about logging because at that point logging is probably not init yet.
The suggestion about setuptools.command.build_py makes sense but I could not get it to work, I'll keep using distutils for now, hopefully someone more knowledgeable will pitch in
The suggestion about itertools and the list is good
The suggestion about removing the exec will not work because the hook file needs to be on a single line (not sure why but it doesn't run if it's multiple lines)

Thank you, I like this idea. Let me test it a bit, and I'll merge it.

Great! I'm glad you liked it

@baldassarreFe I gave it a go. 2 comments

pip install -e . fails on python=3.12.5. This is the normal way to do development with nbdev, as we need to keep the generated python files in sync with what other parts of the library import. I have not looked into it, but I think a bunch of stuff got deprecated lately.

How do you feel about moving/adding the check for LOVELY_TENSORS to init.py ? This way, I can set it in .env and import lovely_tensors, I will activate automatically.

The error:
$ pip install -e .
Obtaining file:///home/xl0/work/projects/lovely-tensors
  Preparing metadata (setup.py) ... done
Requirement already satisfied: torch in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from lovely-tensors==0.1.17) (2.4.0)
Requirement already satisfied: lovely-numpy>=0.2.13 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from lovely-tensors==0.1.17) (0.2.13)
Requirement already satisfied: numpy>=1.17 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (2.1.0)
Requirement already satisfied: fastcore in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (1.7.3)
Requirement already satisfied: ipython in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (8.27.0)
Requirement already satisfied: matplotlib in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (3.9.2)
Requirement already satisfied: filelock in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from torch->lovely-tensors==0.1.17) (3.15.4)
Requirement already satisfied: typing-extensions>=4.8.0 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from torch->lovely-tensors==0.1.17) (4.12.2)
Requirement already satisfied: sympy in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from torch->lovely-tensors==0.1.17) (1.13.2)
Requirement already satisfied: networkx in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from torch->lovely-tensors==0.1.17) (3.3)
Requirement already satisfied: jinja2 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from torch->lovely-tensors==0.1.17) (3.1.4)
Requirement already satisfied: fsspec in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from torch->lovely-tensors==0.1.17) (2024.6.1)
Requirement already satisfied: setuptools in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from torch->lovely-tensors==0.1.17) (72.2.0)
Requirement already satisfied: packaging in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from fastcore->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (24.1)
Requirement already satisfied: decorator in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from ipython->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (5.1.1)
Requirement already satisfied: jedi>=0.16 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from ipython->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (0.19.1)
Requirement already satisfied: matplotlib-inline in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from ipython->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (0.1.7)
Requirement already satisfied: prompt-toolkit<3.1.0,>=3.0.41 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from ipython->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (3.0.47)
Requirement already satisfied: pygments>=2.4.0 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from ipython->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (2.18.0)
Requirement already satisfied: stack-data in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from ipython->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (0.6.2)
Requirement already satisfied: traitlets>=5.13.0 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from ipython->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (5.14.3)
Requirement already satisfied: pexpect>4.3 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from ipython->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (4.9.0)
Requirement already satisfied: MarkupSafe>=2.0 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from jinja2->torch->lovely-tensors==0.1.17) (2.1.5)
Requirement already satisfied: contourpy>=1.0.1 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from matplotlib->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (1.3.0)
Requirement already satisfied: cycler>=0.10 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from matplotlib->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from matplotlib->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (4.53.1)
Requirement already satisfied: kiwisolver>=1.3.1 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from matplotlib->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (1.4.5)
Requirement already satisfied: pillow>=8 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from matplotlib->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (10.4.0)
Requirement already satisfied: pyparsing>=2.3.1 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from matplotlib->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (3.1.4)
Requirement already satisfied: python-dateutil>=2.7 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from matplotlib->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (2.9.0)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from sympy->torch->lovely-tensors==0.1.17) (1.3.0)
Requirement already satisfied: parso<0.9.0,>=0.8.3 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from jedi>=0.16->ipython->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (0.8.4)
Requirement already satisfied: ptyprocess>=0.5 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from pexpect>4.3->ipython->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (0.7.0)
Requirement already satisfied: wcwidth in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from prompt-toolkit<3.1.0,>=3.0.41->ipython->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (0.2.13)
Requirement already satisfied: six>=1.5 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from python-dateutil>=2.7->matplotlib->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (1.16.0)
Requirement already satisfied: executing>=1.2.0 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from stack-data->ipython->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (2.0.1)
Requirement already satisfied: asttokens>=2.1.0 in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from stack-data->ipython->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (2.4.1)
Requirement already satisfied: pure-eval in /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages (from stack-data->ipython->lovely-numpy>=0.2.13->lovely-tensors==0.1.17) (0.2.3)
Installing collected packages: lovely-tensors
  DEPRECATION: Legacy editable install of lovely-tensors==0.1.17 from file:///home/xl0/work/projects/lovely-tensors (setup.py develop) is deprecated. pip 25.0 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at https://github.com/pypa/pip/issues/11457
  Running setup.py develop for lovely-tensors
    error: subprocess-exited-with-error

    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> [71 lines of output]
        /home/xl0/work/projects/lovely-tensors/setup.py:1: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
          from pkg_resources import parse_version
        running develop
        /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/command/develop.py:42: EasyInstallDeprecationWarning: easy_install command is deprecated.
        !!

                ********************************************************************************
                Please avoid running ``setup.py`` and ``easy_install``.
                Instead, use pypa/build, pypa/installer or other
                standards-based tools.

                See https://github.com/pypa/setuptools/issues/917 for details.
                ********************************************************************************

        !!
          easy_install.initialize_options(self)
        /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
        !!

                ********************************************************************************
                Please avoid running ``setup.py`` directly.
                Instead, use pypa/build, pypa/installer or other
                standards-based tools.

                See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
                ********************************************************************************

        !!
          self.initialize_options()
        running egg_info
        writing lovely_tensors.egg-info/PKG-INFO
        writing dependency_links to lovely_tensors.egg-info/dependency_links.txt
        writing entry points to lovely_tensors.egg-info/entry_points.txt
        writing requirements to lovely_tensors.egg-info/requires.txt
        writing top-level names to lovely_tensors.egg-info/top_level.txt
        reading manifest file 'lovely_tensors.egg-info/SOURCES.txt'
        reading manifest template 'MANIFEST.in'
        warning: no files found matching 'CONTRIBUTING.md'
        warning: no files found matching 'lovely_tensors_hook.pth'
        warning: no previously-included files matching '__pycache__' found under directory '*'
        adding license file 'LICENSE'
        writing manifest file 'lovely_tensors.egg-info/SOURCES.txt'
        running build_ext
        Creating /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/lovely-tensors.egg-link (link to .)
        Adding lovely-tensors 0.1.17 to easy-install.pth file

        Installed /home/xl0/work/projects/lovely-tensors
        Traceback (most recent call last):
          File "<string>", line 2, in <module>
          File "<pip-setuptools-caller>", line 34, in <module>
          File "/home/xl0/work/projects/lovely-tensors/setup.py", line 85, in <module>
            setuptools.setup(
          File "/home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/__init__.py", line 108, in setup
            return distutils.core.setup(**attrs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "/home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 184, in setup
            return run_commands(dist)
                   ^^^^^^^^^^^^^^^^^^
          File "/home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
            dist.run_commands()
          File "/home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 964, in run_commands
            self.run_command(cmd)
          File "/home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/dist.py", line 945, in run_command
            super().run_command(command)
          File "/home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 983, in run_command
            cmd_obj.run()
          File "/home/xl0/work/projects/lovely-tensors/setup.py", line 53, in run
            path = os.path.join(self.build_directory, LOVELY_TENSORS_HOOK)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "<frozen posixpath>", line 76, in join
        TypeError: expected str, bytes or os.PathLike object, not NoneType
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [71 lines of output]
    /home/xl0/work/projects/lovely-tensors/setup.py:1: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
      from pkg_resources import parse_version
    running develop
    /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/command/develop.py:42: EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://github.com/pypa/setuptools/issues/917 for details.
            ********************************************************************************

    !!
      easy_install.initialize_options(self)
    /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
    running egg_info
    writing lovely_tensors.egg-info/PKG-INFO
    writing dependency_links to lovely_tensors.egg-info/dependency_links.txt
    writing entry points to lovely_tensors.egg-info/entry_points.txt
    writing requirements to lovely_tensors.egg-info/requires.txt
    writing top-level names to lovely_tensors.egg-info/top_level.txt
    reading manifest file 'lovely_tensors.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching 'CONTRIBUTING.md'
    warning: no files found matching 'lovely_tensors_hook.pth'
    warning: no previously-included files matching '__pycache__' found under directory '*'
    adding license file 'LICENSE'
    writing manifest file 'lovely_tensors.egg-info/SOURCES.txt'
    running build_ext
    Creating /home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/lovely-tensors.egg-link (link to .)
    Adding lovely-tensors 0.1.17 to easy-install.pth file

    Installed /home/xl0/work/projects/lovely-tensors
    Traceback (most recent call last):
      File "<string>", line 2, in <module>
      File "<pip-setuptools-caller>", line 34, in <module>
      File "/home/xl0/work/projects/lovely-tensors/setup.py", line 85, in <module>
        setuptools.setup(
      File "/home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/__init__.py", line 108, in setup
        return distutils.core.setup(**attrs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 184, in setup
        return run_commands(dist)
               ^^^^^^^^^^^^^^^^^^
      File "/home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
        dist.run_commands()
      File "/home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 964, in run_commands
        self.run_command(cmd)
      File "/home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/dist.py", line 945, in run_command
        super().run_command(command)
      File "/home/xl0/mambaforge/envs/torch/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 983, in run_command
        cmd_obj.run()
      File "/home/xl0/work/projects/lovely-tensors/setup.py", line 53, in run
        path = os.path.join(self.build_directory, LOVELY_TENSORS_HOOK)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "<frozen posixpath>", line 76, in join
    TypeError: expected str, bytes or os.PathLike object, not NoneType
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

Hey thanks for testing it!

Editable install with pip install -e . fails: yes, I tried on my side and fails too. I think it's a matter of getting the correct paths for creating the hook and "installing" it ie copying it to the right place. I'll look into this.

In my opinion, there are 3 levels of "magic" for enabling lovely tensors:

No magic, just import lovely_tensors; lovely_tensors.monkey_patch(), as it is now
Auto-patching upon import, ie it's enough to call import lovely_tensors and it will immediately call monkey_patch()
No import required (this PR), just install it and set an env variable, which is neat because it will work out-of-the-box for 3rd party scripts that the user doesn't want to edit

We can get all 3 behaviors but we'll need the import hook and a multi-choice env variable, eg LOVELY_TENSORS_ENABLED=no, LOVELY_TENSORS_ENABLED=upon_import, LOVELY_TENSORS_ENABLED=always.

Thank you, please let me know if you get stuck, I'll look into it.
I played with it some more, and the current implementation works as expected. I think I had some issues with environment variables between conda and Jupyter running in Cursor, which itself is an AppImage on Linux. I'm happy with the current implementation, no further changes required.

Updates:

I read up more on import hooks here: https://docs.python.org/3/library/site.html
Took keep things clean, I separated the logic:

_lovely_tensors_hook.pth is the import hook, its name must end in .pth and it can only contain simple one-line instructions
_lovely_tensors_hook.py contains the monkey patching logic: if LOVELY_TENSORS=always then import lovely tensors and monkey patch (delay the import after the check to avoid importing the whole pytorch unecessarily)

I also modified __init__.py to add the other option: if LOVELY_TENSORS=upon_import then call monkey_patch()

I would be great for you to try it on your machine, just to make sure that it works. The following commands assume that you have the repo at ~/lovely-tensors and use uv to set up a temporary environment.
Setup:
mkdir -p ~/tmp
TMP_DIR="$(mktemp -p $HOME/tmp/ -d)"
pushd "${TMP_DIR:?}"
uv venv --python 3.12
source .venv/bin/activate
uv pip install torch --index-url https://download.pytorch.org/whl/cpu
uv pip install lovely-numpy
uv pip install --no-cache-dir --verbose ~/lovely-tensors

Tests:
python -c 'import torch; print(torch.ones(3))'
# tensor([1., 1., 1.])

LOVELY_TENSORS= python -c 'import torch; print(torch.ones(3))'
# tensor([1., 1., 1.])

LOVELY_TENSORS=foo python -c 'import torch; print(torch.ones(3))'
# tensor([1., 1., 1.])

LOVELY_TENSORS=always python -c 'import torch; print(torch.ones(3))'
# tensor[3] x∈[1.000, 1.000] μ=1.000 σ=0. [1.000, 1.000, 1.000]

LOVELY_TENSORS=upon_import python -c 'import torch; import lovely_tensors; print(torch.ones(3))'
# tensor[3] x∈[1.000, 1.000] μ=1.000 σ=0. [1.000, 1.000, 1.000]

python -c 'import torch; import lovely_tensors; lovely_tensors.monkey_patch(); print(torch.ones(3))'
# tensor[3] x∈[1.000, 1.000] μ=1.000 σ=0. [1.000, 1.000, 1.000]

Clean up:
deactivate
popd
rm -rf "${TMP_DIR:?}"

rm -rf ~/lovely-tensors/build ~/lovely-tensors/lovely-tensors.egg-info

I found that cleaning up the build folders is necessary otherwise pip will skip the build and ignore changes across reinstalls.
Also, I fixed the editable builds so that uv pip install --no-cache-dir --verbose --editable ~/lovely-tensors works without errors. However, with an editable build, the .pth hook doesn't get called so LOVELY_TENSORS=upon_import works but LOVELY_TENSORS=always does not. I don't know if this is a limitation of editable builds or a problem with the installation commands. We can dig out more or simply mention this limitation in the docs. I don't think users will do --editable anyway.

To be honest, I feel the previous version was a better choice.
I see a clear case for having this feature - you have an existing script that prints out some tensors. You set LOVEY_TENSORS=1, the tensors now look nice. Same as with "always" now. Same for Notebooks - I was initially confused and thought the .pth was not handled properly, but that was my mistake, things just work.
But if you have a codebase that already imports lovely_tensors, I  think it's a bad idea to make the behavior depend on the variable. For simple print(t) it's ok, but monkey_patch() adds a dozed new methods to Tensor, and if you use any of them, your code just breaks without this environment variable, likely in the hands of a user that did not have the variable set, and they have no idea what's going on.
Can we just keep it as if os.environ.get("LOVELY_TENSORS") not in [None, "", "false", "0"]: import and patch?
Sorry, my earlier comment was not thought through.
2 other points, and sorry for making this review longer.

Is there a reason the files start with an underscore? I feel this makes it harder to find them, because they won't show near lovely_tensors in ls, and other .pth files in my system don't have underscores.

In setup.py we have

cmdclass = {
        'build': BuildWithHook,
        'easy_install': EasyInstallWithHook,
        'install_lib': InstallLibWithHook,
        'develop': DevelopWithHook,
    },

I don't think we need easry_install and develop - those seem to have been deprecated for a long time.
I don't think we need build either, simply copying the files in install_lib just works, right?

To be honest, I feel the previous version was a better choice.

I agree, just a binary variable that turns the automatic hook on or off is more clear, and more similar to better exceptions.
To summarize, we envision two use cases:

User is working in a notebook, imports lovely tensors and explicitly calls monkey patch to get all goodies like string, channels, plot etc.
User is debugging a 3rd party script that crashes mid-run e.g. due to a shape mismatch. However, the stack trace is messed up and not very informative. User installs better exceptions + lovely tensors and sets the corresponding env variables. Without any change to the 3rd party code, the next time the script crashes the stack trace will contain nicely formatted tensors.

I pushed a new commit for this change.

Is there a reason the files start with an underscore?

Yes, I just wanted to mark the distinction:

The files under lovely_tensors/ constitute the library, users are expected to import those files (or the module as a whole) and call methods from it
The hook files _lovely_tensors_hook.{py,pth} are "special" and are not meant to be imported directly by the user

I added the underscore after looking at more examples of import hooks, some of them use the underscore, some others don't:

RapidsAI: uses an underscore _rapids_dask_dependency.pth
Setuptools: no underscore for the hook distutils-precedence.pth, but uses an underscore for the module that gets imported in the hook _distutils_hack
Better exceptions: no underscore better_exceptions_hook.pth

I think the underscore is good to tell the user "don't import this file", but if you don't like it there is no requirement to have it.

I don't think we need easry_install and develop - those seem to have been deprecated for a long time.

Ok, removed the custom commands for easy_install and develop. The former is deprecated and the latter does not install the hook properly anyway.

@baldassarreFe Thank you! Merged.

@baldassarreFe Would you like to add it to the README?

Great! I'll leave the documentation to you if you don't mind. I'm not familiar with the nbdev workflow. Should we also bump the version?

Wow, looks really good with BE! I'll document it and bump the version.


TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/79217

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/94289

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/94441


fix: [Gemini] content filter errors handled properly

chunk.candidates may be None or empty, when the response is blocked.
Such a response will lead to out-of-bounds index runtime error when the adapter tries to access chunk.candidates[0].
So the fixes are the following:

check the existense of chunk.candidates[0] before using it,
check the blocking feedback chunk.prompt_feedback and translate it to content_filter finish reason.

=====
For a test request to reproduce the blocking response - ask Gemini a question about a medium-size wiki-page referencing in the middle of it a sensitive topic.

/deploy-review

/deploy-review

/deploy-review


add omitJvmStatic to Kotlin plugin

Description
Related #7631

Added an omitJvmStatic option to Kotlin plugin, so that its output can be used in Kotlin Multiplatform Project.
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

Screenshots/Sandbox (if appropriate/relevant):
Adding links to sandbox or providing screenshots can help us understand more about this PR and take action on it as appropriate
How Has This Been Tested?
Included a unit test with the PR.
Checklist:

[x] I have followed the CONTRIBUTING doc and the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules

Further comments
If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...

Thank you for your contribution!
It has been released in<EMAIL_ADDRESS>📦


Slow ranking speed

The speed for most of the queries are fine, but for queries that have lots of returned results such as "ocean temperature", the speed is very slow (>10s).
I am trying solve this problem through multi-threading, but there is an issue that I cannot fix. Maybe I can use some different eyes.
Detailed problem statement: normally, if a>b and b>c, a>c. But this may not be true in our case, because our ranking model is not overfitting, so what I did before is simple but costs too much time.
for(int j=0; j< resultList.size(); j++)
    {
      for(int k=0; k< resultList.size(); k++)
      {
        if(k!=j)
        {
          resultList.get(j).below += comp (resultList.get(j), resultList.get(k));
        }
      }
    }

What I am doing right now is to divide the resultList into multiple parts and run the same code simultaneously, which kind of improved the speed a bit, but not much.
Let me know if you have any ideas.

Problem solved by using Spark Mlib SVM to replace WEKA


style: added tokens for fluid spacing

Note to self; will have to add new format with support for outputReferences and typescale to get nested calc support (via css-variables)


AMC not detecting custom watch and output folder

I have a single folder called Data mapped to Data in the container

I changed the AMC output and input variables to

Tried couple of different variations but it seems AMC cant detect any folders that arent mounted to /watch and /output

You used wrong variables.  You should use AMC_INPUT_DIR and AMC_OUTPUT_DIR.

That did the trick, was using the default variables that came with the container.