Ability for Container to save state and rehydrate in absence of runtime-level ops

Work Item
Describe the outcome you expect

Add the ability for Container to snapshot its state and rehydrate when no runtime-level ops have been sequenced since its initial load. Future work will enable runtime to rehydrate when runtime-level ops have been sequenced since load.
Container should snapshot protocol state and anything else required for rehydration upon Container.closeAndGetPendingLocalState() being called.
Runtime should save its base snapshot and should fetch all snapshot blobs and return them upon ContainerRuntime.getPendingLocalState() being called.
The blob returned by closeAndGetPendingLocalState() can be passed to Loader.resolve(), which should result in container and runtime being rehydrated to the same state and able to continue operation as normal.
Additional context

#7594
#8247

implemented in #9557


Add icon-magnet.gif / add magnet links to the guide

I think a good addition would be to add magnet links in addition to just the torrent file.
It's not a requirement but sometimes magnet links can be more convenient since they open the torrent client directly instead of having to download the torrent file and then opening that.
You can add the magnet icon / link to the end of the filename like so:
[![Magnet Link](https://plailect.github.io/Guide/icon-magnet.gif)](magnet-url-here)

Here is a file with all the corresponding magnet links
magnet links.txt

Github flavored markdown blocks magnet URIs in both their markdown link and href forms, hence the use of .torrent files.


Interface support

Any plans to add Interface support soon? I wanted to use the package in a native app to query the github API, but it uses Interfaces heavily.

PR here #56

Released in version 0.2.8.


gherkin-perl: Add Perl infrastructure to the CI docker images

Summary
Add Perl infrastructure required to start testing Perl implementation.
Details
Before this change, there's insufficient Perl in the images
(none, actually) to run the Perl tests and thus to verify
validity of PR #694.
Incidentally add --notest flag to cpanm to stop it from
downloading half of CPAN in order to test dependencies it's
building.
How Has This Been Tested?
In my local Docker environment I verified that this change is enough to run Perl and install dependencies using the module installer cpanm on which the gherkin-perl Makefile depends.
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue).
[x] New feature (non-breaking change which adds functionality).
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected).

Checklist:

[ ] The change has been ported to Java.
[ ] The change has been ported to Ruby.
[ ] The change has been ported to JavaScript.
[ ] The change has been ported to Go.
[ ] The change has been ported to .NET.
[ ] I've added tests for my code.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

None of the above are applicable.

Great work @ehuelsmann! I'd like to improve the CI build so that new docker images are published automatically. Currently I have to push it manually, and that makes it tedious for me everyone to work with pull requests to the docker image.
I'll add this onto this branch and test it out. Then I'll merge this.
Aslak

Hi @ehuelsmann,
Thanks for your making your first contribution to Cucumber, and welcome to the Cucumber committers team! You can now push directly to this repo and all other repos under the cucumber organization! 🍾
In return for this generous offer we hope you will:

✅ Continue to use branches and pull requests. When someone on the core team approves a pull request (yours or someone else's), you're welcome to merge it yourself.
💚 Commit to setting a good example by following and upholding our code of conduct in your interactions with other collaborators and users.
💬 Join the community Slack channel to meet the rest of the team and make yourself at home.
ℹ Don't feel obliged to help, just do what you can if you have the time and the energy.
🙋 Ask if you need anything. We're looking for feedback about how to make the project more welcoming, so please tell us!

On behalf of the Cucumber core team,
Aslak Hellesøy
Creator of Cucumber


Swagger Documentation for LoansApiResource.java

@Sanyam96 Please Review it!!
I have not annotated the first and last endpoint as the first endpoint doesn't exist on the existing documentation and the last endpoint have many documentations according to the Path Param.
Note : - I have not tested the changes as first and last endpoints are remaining to be annotated.

@Sanyam96 Updated the PR


Update ConceptsOfModernProgramming.rst

sensible = vernünftig
sensitive = sensibel

@Knallcharge  you have to reopen this PR on neos/flow-development-collection because this repo is read only.

@danielkestler done


[] build failure

Host Environment

OS: [e.g. Windows10 rev1803 17134.1246]
Compiler: Visual Studio Community 2009

To Reproduce
Steps to reproduce the behavior:
./vcpkg install boost:x64-windows
Failure logs
Computing installation plan...
The following packages will be built and installed:
    boost[core]:x64-windows
  * boost-accumulators[core]:x64-windows
  * boost-algorithm[core]:x64-windows
  * boost-align[core]:x64-windows
  * boost-any[core]:x64-windows
  * boost-array[core]:x64-windows
  * boost-asio[core]:x64-windows
  * boost-assert[core]:x64-windows
  * boost-assign[core]:x64-windows
  * boost-atomic[core]:x64-windows
  * boost-beast[core]:x64-windows
  * boost-bimap[core]:x64-windows
  * boost-bind[core]:x64-windows
  * boost-build[core]:x64-windows
  * boost-callable-traits[core]:x64-windows
  * boost-chrono[core]:x64-windows
  * boost-circular-buffer[core]:x64-windows
  * boost-compatibility[core]:x64-windows
  * boost-compute[core]:x64-windows
  * boost-concept-check[core]:x64-windows
  * boost-config[core]:x64-windows
  * boost-container[core]:x64-windows
  * boost-container-hash[core]:x64-windows
  * boost-context[core]:x64-windows
  * boost-contract[core]:x64-windows
  * boost-conversion[core]:x64-windows
  * boost-convert[core]:x64-windows
  * boost-core[core]:x64-windows
  * boost-coroutine[core]:x64-windows
  * boost-coroutine2[core]:x64-windows
  * boost-crc[core]:x64-windows
  * boost-date-time[core]:x64-windows
  * boost-detail[core]:x64-windows
  * boost-dll[core]:x64-windows
  * boost-dynamic-bitset[core]:x64-windows
  * boost-endian[core]:x64-windows
  * boost-exception[core]:x64-windows
  * boost-fiber[core]:x64-windows
  * boost-filesystem[core]:x64-windows
  * boost-flyweight[core]:x64-windows
  * boost-foreach[core]:x64-windows
  * boost-format[core]:x64-windows
  * boost-function[core]:x64-windows
  * boost-function-types[core]:x64-windows
  * boost-functional[core]:x64-windows
  * boost-fusion[core]:x64-windows
  * boost-geometry[core]:x64-windows
  * boost-gil[core]:x64-windows
  * boost-graph[core]:x64-windows
  * boost-graph-parallel[core]:x64-windows
  * boost-hana[core]:x64-windows
  * boost-heap[core]:x64-windows
  * boost-histogram[core]:x64-windows
  * boost-hof[core]:x64-windows
  * boost-icl[core]:x64-windows
  * boost-integer[core]:x64-windows
  * boost-interprocess[core]:x64-windows
  * boost-interval[core]:x64-windows
  * boost-intrusive[core]:x64-windows
  * boost-io[core]:x64-windows
  * boost-iostreams[core]:x64-windows
  * boost-iterator[core]:x64-windows
  * boost-lambda[core]:x64-windows
  * boost-lexical-cast[core]:x64-windows
  * boost-local-function[core]:x64-windows
  * boost-locale[core]:x64-windows
  * boost-lockfree[core]:x64-windows
  * boost-log[core]:x64-windows
  * boost-logic[core]:x64-windows
  * boost-math[core]:x64-windows
  * boost-metaparse[core]:x64-windows
  * boost-modular-build-helper[core]:x64-windows
  * boost-move[core]:x64-windows
  * boost-mp11[core]:x64-windows
  * boost-mpl[core]:x64-windows
  * boost-msm[core]:x64-windows
  * boost-multi-array[core]:x64-windows
  * boost-multi-index[core]:x64-windows
  * boost-multiprecision[core]:x64-windows
  * boost-nowide[core]:x64-windows
  * boost-numeric-conversion[core]:x64-windows
  * boost-odeint[core]:x64-windows
  * boost-optional[core]:x64-windows
  * boost-outcome[core]:x64-windows
  * boost-parameter[core]:x64-windows
  * boost-parameter-python[core]:x64-windows
  * boost-phoenix[core]:x64-windows
  * boost-poly-collection[core]:x64-windows
  * boost-polygon[core]:x64-windows
  * boost-pool[core]:x64-windows
  * boost-predef[core]:x64-windows
  * boost-preprocessor[core]:x64-windows
  * boost-process[core]:x64-windows
  * boost-program-options[core]:x64-windows
  * boost-property-map[core]:x64-windows
  * boost-property-tree[core]:x64-windows
  * boost-proto[core]:x64-windows
  * boost-ptr-container[core]:x64-windows
  * boost-python[core]:x64-windows
  * boost-qvm[core]:x64-windows
  * boost-random[core]:x64-windows
  * boost-range[core]:x64-windows
  * boost-ratio[core]:x64-windows
  * boost-rational[core]:x64-windows
  * boost-regex[core]:x64-windows
  * boost-safe-numerics[core]:x64-windows
  * boost-scope-exit[core]:x64-windows
  * boost-serialization[core]:x64-windows
  * boost-signals2[core]:x64-windows
  * boost-smart-ptr[core]:x64-windows
  * boost-sort[core]:x64-windows
  * boost-spirit[core]:x64-windows
  * boost-stacktrace[core]:x64-windows
  * boost-statechart[core]:x64-windows
  * boost-static-assert[core]:x64-windows
  * boost-static-string[core]:x64-windows
  * boost-system[core]:x64-windows
  * boost-test[core]:x64-windows
  * boost-thread[core]:x64-windows
  * boost-throw-exception[core]:x64-windows
  * boost-timer[core]:x64-windows
  * boost-tokenizer[core]:x64-windows
  * boost-tti[core]:x64-windows
  * boost-tuple[core]:x64-windows
  * boost-type-erasure[core]:x64-windows
  * boost-type-index[core]:x64-windows
  * boost-type-traits[core]:x64-windows
  * boost-typeof[core]:x64-windows
  * boost-ublas[core]:x64-windows
  * boost-units[core]:x64-windows
  * boost-unordered[core]:x64-windows
  * boost-utility[core]:x64-windows
  * boost-uuid[core]:x64-windows
  * boost-variant[core]:x64-windows
  * boost-variant2[core]:x64-windows
  * boost-vmd[core]:x64-windows
  * boost-wave[core]:x64-windows
  * boost-winapi[core]:x64-windows
  * boost-xpressive[core]:x64-windows
  * boost-yap[core]:x64-windows
  * bzip2[core]:x64-windows
  * libffi[core]:x64-windows
  * liblzma[core]:x64-windows
  * openssl[core]:x64-windows
  * openssl-windows[core]:x64-windows
  * python3[core]:x64-windows
  * zlib[core]:x64-windows
  * zstd[core]:x64-windows
Additional packages (*) will be modified to complete this operation.
Detecting compiler hash for triplet x64-windows...
Starting package 1/148: boost-compatibility:x64-windows
Building package boost-compatibility[core]:x64-windows...
Could not locate cached archive: C:\Users\oijen\AppData\Local\vcpkg\archives\ca\ca8f5b16a87807b1f5815048d8f4a89d3a1c7030.zip
-- Using cached D:/oijen/_code/git/vcpkg/downloads/boostorg-compatibility-boost-1.73.0.tar.gz
-- Cleaning sources at D:/oijen/_code/git/vcpkg/buildtrees/boost-compatibility/src/ost-1.73.0-44642da97c.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source D:/oijen/_code/git/vcpkg/downloads/boostorg-compatibility-boost-1.73.0.tar.gz
-- Using source at D:/oijen/_code/git/vcpkg/buildtrees/boost-compatibility/src/ost-1.73.0-44642da97c.clean
-- Packaging headers
-- Packaging headers done
-- Downloading https://raw.githubusercontent.com/boostorg/boost/boost-1.73.0/LICENSE_1_0.txt...

-- Downloading https://raw.githubusercontent.com/boostorg/boost/boost-1.73.0/LICENSE_1_0.txt... Failed. Status: 6;"Couldn't resolve host name"
CMake Error at scripts/cmake/vcpkg_download_distfile.cmake:176 (message):

      Failed to download file.
      If you use a proxy, please set the HTTPS_PROXY and HTTP_PROXY environment
      variables to "https://user:password@your-proxy-ip-address:port/".
      Otherwise, please submit an issue at https://github.com/Microsoft/vcpkg/issues

Call Stack (most recent call first):
  installed/x64-windows/share/boost-vcpkg-helpers/boost-modular-headers.cmake:17 (vcpkg_download_distfile)
  ports/boost-compatibility/portfile.cmake:12 (boost_modular_headers)
  scripts/ports.cmake:79 (include)

Error: Building package boost-compatibility:x64-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `.\vcpkg update`, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
  Package: boost-compatibility:x64-windows
  Vcpkg version: 2020.06.15-nohash

Additionally, attach any relevant sections from the log files above.

Additional context
From where the failure occured the first time hightlighted above in italic, I have confirmed which that .txt file is able to fetch from brower whether I use proxy or not.

Duplicated to https://github.com/microsoft/vcpkg/issues/12383


533 reproduction note

https://github.com/uwlib-cams/MARC2RDA/blob/main/Working Documents/5XX.csv

Here's a link to the 2014 PCC policy statement, describing the practice of "cloning" a record for the original, with specific changes to fixed fields and other areas  but leaving most of the description for the original and using a 533 to describe the reproduction.  This started out being for microforms, but is being used for electronic reproductions as well.  It doesn't describe a "single record approach" for original and reproduction, but may be used with a single record approach for multiple electronic manifestations.    https://view.officeapps.live.com/op/view.aspx?src=https%3A%2F%2Fwww.loc.gov%2Faba%2Fpcc%2Fdocuments%2F1-11-LC-PCC-PS.docx&wdOrigin=BROWSELINK
Here's the MARC 533 field description:
Descriptive data for a reproduction of an original item when the main portion of the bibliographic record describes the original item and the data differ.
The original item is described in the main portion of the bibliographic record and data relevant to the reproduction are given as a note in field 533 when they differ from the information describing the original. It is used whenever an institution chooses to have the description reflect the original and the notes reflect information about the copy.
For mixed materials, this field contains information describing a copy of a record unit when the agency describing the materials possesses only a copy and, in accordance with conventions, the original is described in the main portion of the control record. This field is normally used in conjunction with field 535 (Location of Originals/Duplicates Note) which indicates the repository holding the original.

Challenge:  533 used for "single record" cataloging for photocopy / POD records following OCLC instruction would contain only 533 $a (type of reproduction).  If a record follows the guidelines, a marker for when 533 is used in this context is 040 $e containing "pd" . This subfield $a may be mapped to the manifestation being described (as a category of manifestation?), but,  should presence of such a field be used to modify the mapping for other fields in the record (to an IRI for the "original" manifestation) or should that be changed?.  OCLC 3.2 specifies fields to describe the original; the key differing data here would be the place, producer, and date of production of the manifestation being described (e.g. POD reproduction) which is never specified (unless there is a local practice).

Challenge:  As far as I can tell, tag 533 doesn't provide data to distinguish reproductions which are published from those that are unpublished.   RDA does not provide a "superelement" which could be used to map data when this is uncertain.  There is Manifestation Production Statement for unpublished material and Manifestation Publication Statement for published material. This question may have come up when mapping 260/264 as well - any advice @SitaKB ?
@CECSpecialistI

@CECSpecialistI   This is going to take more than a week to complete, but here's where I've gotten so far:
First pass / provisional mapping of 533 subfields EXCEPT for subfields $y (Data provenance) and $7 .

I don't remember if we discussed $y but it appears we haven't made decision about it so it needs its own discussion.  Appendix J:  Data Provenance
$7 needs more review/analysis relative to 008 mapping,  It provides 008 values for the reproduction.  I'll get to it before diving into the conditional mapping of other fields.

I have sketched out some of the fields that need conditional instructions for mapping, when a 533 is present:
008:
If 533 is present, mint a URI for an "original manifestation", manifesting the same expression of the same work as the reproduction represented by the MARC record.  Follow the mapping instructions for 008 except:
(detail of which bytes should be mapped in relation to the original manifestation URI
(detail of which bytes should be mapped in relation to the current (reproduction) manifestation URI)
(detail of which bytes should be mapped in relation to both reproduction and original manifestation, if any)
245  if 533 is present, in addition to mapping contents of 245 subfields to mainfestation elements related to item cataloged (the reproduction), map to relation to a manifestation for the original (minted in 008 mapping).
260 and/or 264:  if 533 is present, map manifestation properties in relation only to manifestation URI "for the original", as generated in the 008 mapping relation.
300:  if 533 is present, map manifestation properties in relation only to manifestation URI "for the original", as generated in the 008 mapping relation
800, 810, 811, 830    if 533 is present, map manifestation properties only to manifestation URI "for the original" as generated in the 008 mapping relation.

I also need to figure out how to determine "digital masters" records that follow the DLF practice (how to determine if a record is for a digital master, and how mappings are affected) to see whether this presents a variant to what I'm proposing, and how this related to the location / 535 mapping.

Challenge:  533 $m Dates and/or sequential designation of issues reproduced.
This implies relationship to an original that is an aggregating serial resource.  The contents may indicate that all, or only a portion of that resource is reproduced in this manifestation, but which cannot be determined from the data.
The best I could come up with was Note on manifestation.  If anyone involved with serials knows of any guidelines for describing reproductions that would give a more specific property/relation between the reproduction and the original, let me know!
Crystal, I don't know how to "@" everybody - can you see that these comments are shared to relevant people in the group?
@CECSpecialistI

Thank you @lake44me ! I added a review of these comments to the agenda for tomorrow.

From: Akerman, Laura<EMAIL_ADDRESS>Sent: Thursday, December 7, 2023 10:45 AM
To: Crystal Yragui<EMAIL_ADDRESS>Subject: 533 mapping
Hi Crystal,
For some weird reason I woke up at 2:30 this morning with energy and started working on the 533 mapping.  I think I’ve covered every subfield but $7 .  I’d appreciate it if you could review the fields I’ve mapped.
$7 is a selective “fixed field” corresponding to 008 general values for bytes 6-17 Date type, Date 1 and Date 2, and Country of publication, plus for continuing resources, byte 18 Frequency.   I thought I could just borrow the mappings from the 008 field and plug them in here.
But…  I need to be sure I understand the logic for at least the position 06 (Dtst) and 7-10 and 11-14 mappings, particularly for the notes that are getting generated and the transform notes.  Would you be able to explain this to me?  I think I wasn’t at those early 008 mapping parties…  If I could possibly get a little of your time (say ½ an hour) for a Zoom call on this, next week or week after, or even early next year, that’d be great.  Or suggest alternatives (should I talk to Sita?)
Later on, I would make a special version of 008 for records that contain a 533 tag, with instructions to mint an IRI for original manifestation and relate it to the expression and manifestation we are cataloging, and map designated position values to the description of that original manifestation based on the documentation we have about the previous PCC 533 practice.   I’d ask Theo if he’d prefer getting this version with just the positions that describe the original (and put a condition /= 533 or something on those positions in the full 008 mapping), or duplicate the whole thing…    Anyway, I need to understand the 008 mapping better than I thought I’d have to – or at least the dates part.
That may not be necessary for other tags – I could add conditional lines for what to do when 533 is present and make reference to the manifestation IRI for the original minted for 008 mapping.  That’s the plan, anyway.
Laura
P.S.  I see where we recorded a Decision in March to just map 533 as a Note on Manifestation, but from the discussion at the November 8 meeting, we are revisiting it, maybe we wait to change the Decision until after the mapping is done?

@CECSpecialistI  plans to work on this next week.

For $7, look for some 539's in OCLC/some examples in Alma

533 reproduction note examples
Use of $5
Millions like this in OCLC, and thousands in our Alma catalog (Hathi Trust):
oclc 	615373298
533	 	  	 	Electronic reproduction. ‡b [Place of publication not identified] : ‡c HathiTrust Digital Library, ‡d 2010. ‡5 MiAaHDL
Use of $7
Probably hundreds of thousands like this, Eighteenth Century Collections Online eletronic set, contains $7
Emory Alma MMSID<PHONE_NUMBER>80302486
533	__ |a Electronic reproduction. |b Farmington Hills, Mich. : |c Thomson Gale, |d 2003. |n Available via the World Wide Web. |n Access limited by licensing agreements |7 s2003 miunns
Also in our Alma:
oclc 52967803
533	__ |a Electronic reproduction. |b Atlanta, Georgia : |c Pitts Theology Library, Emory University, |d 2003. |f (Thanksgiving Day Sermons, ATLA Cooperative Digital Resources Initiative, CDRI). |n Joint CDRI project by: Andover-Harvard Library (Harvard Divinity School), Pitts Theology Library (Emory University), and Princeton Theological Seminary Libraries. |7 s2003 gaun s
An example (of perhaps many) with 2 533 tags - Hathi Trust and Harvard Library identified in $5's
OCLC 674405981
506	__ |3 Use copy |f Restrictions unspecified |2 star |5 MiAaHDL
533	__ |a Electronic reproduction. |b [Place of publication not identified] : |c HathiTrust Digital Library, |d 2010. |5 MiAaHDL
538	__ |a Master and use copy. Digital master created according to Benchmark for Faithful Digital Reproductions of Monographs and Serials, Version 1. Digital Library Federation, December 2002. |u http://purl.oclc.org/DLF/benchrepro0212 |5 MiAaHDL
583	1_ |a digitized |c 2010 |h HathiTrust Digital Library |l committed to preserve |2 pda |5 MiAaHDL
588	0_ |a Print version record.
506	__ |a No restrictions on access copy. |5 MH
533	__ |a Electronic reproduction. |b Cambridge, Mass. : |c Harvard College Library Digital Imaging Group, |d 2008. |f (Harvard College Library preservation digitization program). |5 MH
538	__ |a Digital master created according to Benchmark for Faithful Digital Reproductions of Monographs and Serials, Version 1. Digital Library Federation, December 2002. |u http://purl.oclc.org/DLF/benchrepro0212 |5 MH
583	1_ |j Harvard University Library |l committed to preserve |5 MH
I can look for more, but I think from what I've seen so far, the use of $5 indicates the institution that created the reproduction, not necessarily the one that owns a copy that was reproduced.  That's at least the case for the Hathi Trust 533's.
Do we need to see more $7's?  Or should we just attempt to map them based on the 008 bytes they represent?

for $5, would this do?
Creator corporate body of manifestation http://rdaregistry.info/Elements/m/P30421
either Identifier (the symbol),
or a nomen that has nomen string of the symbol along with  scheme of nomen https://www.loc.gov/marc/organizations/orgshome.html
or IRI (the id.loc.gov Cultural Heritage Institutions IRI corresponding to the symbol, if we take this to be a "real world object" IRI, but note some symbols aren't represented in this list, e.g. MiAaHDL for Hathi Trust Digital Library)

@cspayne @gerontakos @CECSpecialistI this needs your attention RE: how to process 008. You're getting an email from @lake44me

@cspayne @gerontakos @CECSpecialistI  Apologies for being slow to send that email - I realized there was a duck I didn't have in a row before I present choices to you:  Provider-neutral cataloging.  I'm examining OCLC's specifications here:
https://www.oclc.org/bibformats/en/specialcataloging.html#providerneutralcataloging which hopefully follow PCC specifications
and I think I'm finally getting some clarity:
PN cataloging could be for digitized reproductions of physical materials or for born-digital content.  PN records are expected to have an 040 $e with code "pn".
The PCC instructions say "All online resources are considered published. The provider-neutral model (in contrast to RDA) specifies that if the e-resource being cataloged is an online reproduction of a tangible M (Mandatory); A (Mandatory if applicable); O (Optional); X (not used); N/A (not applicable) RDA Provider-Neutral Guidelines for Serials, Integrating Resources, and Monographs resource, usually the Production, Publication, Distribution, Manufacture and Copyright notice information will come from the original tangible source record. "   So, elements like 008 Dates and 264 publication information describe an original.
But the kicker is this PCC instruction :  533:  Use only for records for DLF Registry of Digital Masters, HathiTrust Digital Library, and other digital preservation projects. Use with $5
This opens the possibility that there are PN records lacking a 533 which are still reproductions of an original and which mix descriptive elements of both in the record!
I don't know what to do about those at this point - that's going to be an area for future analysis, perhaps in Phase 2.  But it appears that at lest we can say that PN electronic manifestation records with a 533, like other records with a 533, are using the mixed original/ reproduction description approach, so I think we have one "sort" for processing them.
The question I'll be asking if we have a meeting, is, for ease of transformation, should we have separate mapping tables for records with 533 than from records without?  Rather than applying a 533 condition to every field/subfield/byte that needs one...
We could label the non-533 tables as "no 533" for tags that are affected.
Or, would you prefer getting the conditions for 533 in each line of the existing tables?
Or, some combination (it would be really rough to do this line by line for 008, but perhaps easier for 264 or other fields).
Tags potentially affected (I think, but will double-check):  008, 245, 260, 264, 300, 8xx.

@lake44me, would you be up for putting together a clear proposal on this for discussion during a meeting? I confess that I have not been keeping up with the complexities of reproductions and the 533, and have cataloged very few of them. I am not sure how feasible separate mapping tables will be for reproductions vs. originals but have an open mind, although it might need to wait for Phase II of the project (read: 2025 or beyond) if I'm being practical, unless I'm misunderstanding the complexity of what you're proposing.

@CECSpecialistI I was not up for doing it this week.
I think if I'm going to try to write up the complexity, I might as well take a crack at just adding the conditional lines to the 008 and other tags.  The 008 is the most complex.  Just to be safe, I will duplicate these mappings and if the conditions are approved, will just substitute them in - and I'll be as careful as I can be.   Will aim for end of month.

Discovered a new wrinkle for use of 533 tags.  Was looking at the OCLC page and it had an example in the definition for $a:
Record for the print version that notes the existence of a digital preservation version:
533	 	 	ǂ3 Volumes 78-81 (1996-1999): ǂa Also available as electronic reproduction. ǂb [Chicago] : ǂc University of Chicago Library, ǂd [2006] ǂ5 ICU
533	 	 	Also available as electronic reproduction. ǂb Washington, D.C. : ǂc Library of Congress, ǂd [2002-2003] ǂ5 DLC
I went looking in the Library of Congress catalog (advanced search on the phrase "also available as" in Notes).  Most of the ones I sampled toward the front used that phrase in a 530 Additional form available note, but starting at the end of the 5,000+ results, I found a record, LCCN<PHONE_NUMBER> https://lccn.loc.gov/2009374281/marcxml .  As best I can determine, the record describes a "tangible" resource, but with addition of these tags:

Also available as electronic reproduction.
Chicago :
Center for Research Libraries,
[2007]

Master and use digital copies. Technical details on the digital scanning are available at http://dds.crl.edu/technical_information.asp?tid=4464

Digitized
2007
Center for Research Libraries
pda
CRL.

I am thinking this is not impossible to deal with, but introduces a conditional wrinkle - if $a begins with "Also available as", treat the record as describing an original, and deal with the 533 and 538 as describing a different, reproduction manifestation.   Will catch up with details of that later.
As Charlie Brown would say, "Aaaaaaaugh!"

@CECSpecialistI @cspayne I haven't made it all the way through 008 yet (stopping after line 34) but I wanted you to get an idea of the pattern before I invest a lot of time in it.
https://docs.google.com/spreadsheets/d/14b6_gyuwoVf9_d-9j-yu85dEQwvzdblD87SLoibELXE/edit#gid=1517787708
Some questions:

Would it be better for the coding work to have a mapping document for "Reproduction Conditions 533 and other" and just throw all the conditional mappings with subject "Original Manifestation" IRI (or in some cases, both Original Manifestation and Reproduction Manifestation) for 008, 245, 260 etc. into it?  And then change the label for 008, 245, 260 etc. to say, 008 No Repro Cond., etc. ?

I'm thinking it would make it easier to "sort" the record for Reproduction Conditions at the outset and then apply the different transform templates, and anything not meeting those conditions would be handled as normal based on those spreadsheets
Or, should I continue inserting these conditional fields in each mapping table wherever they might apply?

As I examine the mappings of 008 general positions and specific format positions, I can see that some are likely to apply to both original and reproduction.  Certainly if these are expression properties (such as language 008 35-37) that will be automatic.
But if they are manifestation properties it would be a judgement call whether they should be the same between reproduction and original (I would think the 008 18-21 Illustrations for BOOKs format should be the same).
Mapping these to both original and reproduction would result in a fuller description of the original, but since this is not what we're primarily creating a record for, I'm thinking we could just apply them to the reproduction, unless we think there would be a value down the line in having a fuller set of data about the "original" for reconciliation with data created to describe the original somewhere (maybe in the same collection, maybe not).  Your thoughts?

I put a note in there about provider-neutral records (assuming we can identify them all with 040 $e pn).  The PCC document seems to encourage this approach for records for online resources in general.  There are a great number of PN records from digital preservation projects out there with 533's, but there are probably also quite a number that leave out the 533 as suggested in the PCC guidance.  The important part of those instructions is that for reproductions, continue to depart from RDA and still code the 008 dates, 264 etc. for the original rather than the repro.  Without a 533, I don't know if there is a clear indicator that something is an online reproduction rather than "born digital" material.  I need to study this more.
Cypress would you rather I wait until this aspect's conditions are provided (if we can get them)?  Or can I go ahead and give you the 533 conditions, and then update them with PN conditions, and maybe others, later?

@lake44me
I'm not completely caught up on the mapping for 008 or 533 yet, but I can answer the coding questions.
I think having a separate mapping document would work great, and it should be fine to go ahead with the 533 conditions and update them as needed. If updates are made after coding has started, we'll need a way to indicate what's changed or been added so I don't miss anything.

Thanks @cspayne!  That answers my big question (#1) and lets me know #3 (provider neutral) is not a holdup.

Hi @lake44me
I am not clear on the 533/008 issues either, and would appreciate an opportunity to be caught up. Do you think it would make sense for us to meet and talk about this before you go full steam ahead and invest more time in it, as you say? There are many comments spanning a good amount of time, and I'm not sure which ones still apply and which ones have been moved on from. Are you able to put together a succinct summary of what the issues are and what you're planning to do with the 008 and 533? Or, would you like me to try and put something on the calendar for you, me, and @cspayne in the next couple of weeks?

Proposal for mapping 533 $7 (fixed field information for the reproduction) - with wrinkles:
Tag 533 $7.  $7 is a "mini-fixed-field" for the reproduction manifestation being described by the MARC record.  My approach is to identify 008 mappings to apply to the positions in $7 and just say something like:
Mapping to Reproduction manifestation:
For byte 0, apply 008 mappings for byte 6 (Type of date/Publication status)
For bye 1-4, apply 008 mappings for bytes 7-10 (Date1)
For bytes 4-8, apply 008 mappings for butes 11-14 (Date2)"
For bytes 9-12, apply 008 mappings for bytes 15-17 (Place of Publication, Production or Execution)
For byte 13, for Continuing Resources format only*, apply 008 mapping for byte 18 (Frequency)
For byte 14, for Continuing Resources format only*, apply 008 mapping for byte 19 (Regularity)
For byte 15, if format is Maps or Visual Materials*, apply 008 mapping for byte 29 (Form of Item).  For all other formats*, apply 008 mapping for byte 23 (Form of item)
*conditions for determining format found elsewhere...?
Wrinkle:
Looking at the definitions in https://www.loc.gov/marc/bibliographic/bd533.html, we may need to break out the Date1 and Date2 mappings for Continuing Resources since the meaning is different.  For "non-serials" the meaning seems straightforward and applying the 008 mappings relatively uncomplicated - the dates are of publication, etc. of the reproduction.
For serially-issued items, 1-4 contains the original beginning date of publication of the issues that have been reproduced, as indicated in subfield $m of field 533.
For serially-issued items, 5-8 contains the original ending date of publication of the issues that have been reproduced, as indicated in subfield $m of field 533.
This reproduction-specific information probably will probably need to be a note field - or, if there are dates in subfield $m of the 533, just ignore these bytes.  The reproduction manifestation may or may not be a continuing resource... ?

Wrinkle #2 - Form of item (byte 15)
LC-PCC PS for 1.11 RDA (Original RDA toolkit)
Microform reproduction of print resources
	008/23: Record the value associated with the microform being cataloged (a, b, or c).
	This in contrast to other enumerated 008 fields which should be coded for the original (print resource)

Print on Demand (POD) Reproductions and Photocopies
	008/23 (Form of item) or 008/29 (Form of item): Record the value "r" indicating the form of item is a print reproduction.
	Again, the value in the 008 pertains to the reproduction, not the original

PCC Provider-Neutral E-resources
008 23 / 29 Form of item - says "use code 'o' ; all other bytes of the 008 should reflect the original manifestation.  (o = online)

If the form of item in the 008 pertains to the reproduction, how and why could the byte 15 value in 533 $7 be different?

@CECSpecialistI @AdamSchiff
Added this note to the mapping for 533:
Problem to be addressed when we tackle serial conversion (Phase 2)?
Definition for Date1 and Date2 in subfield 7 (fixed field information for the reproduction)  for continuing resources are defined as original beginning and ending dates of issue for the reproduced serial volume, not date(s) of issue for the reproduction.
I need to understand more about how this fits in with the cataloging of reproductions of serials/CRs.
Do the reproductions get the same record format (Continuing Resources) as the original?
Other than this question, I wrote out the $7 mapping and think I am done with the 533 mapping, but we can wait on approving it until the other fields mappings are done.

@CECSpecialistI @cspayne @dchen077  Crystal, can you review the 533 $7 mapping (the fixed field coding for the reproduction) - otherwise 533 is ready for code.  Does it need to wait until the 008 coding is stabilized?
All the logic in that field is in the transformation notes:
"/0 - Type of date/Publication status
Use 008 06 mappings
Note:  code r (original and reproduction dates) should not be used here!
/1-4 - Date 1
Use 008 07-10 mappings
/5-8 - Date 2
Use 008 11-14 mappings
/9-11 - Place of publication, production, or execution
Use 008 15-17 mappings
/12 - Frequency
Format must be CONTINUING RESOURCES.
Use 008 18 mappings
/13 - Regularity
Format must be CONTINUING RESOURCES
Use 008 19 mapping
/14 - Form of item
If Format is BOOKS, COMPUTER FILES, MUSIC, CONTINUING RESOURCES, or MIXED MATERIALS
Use 008 23 mapping for the respective format
If Format is MAPS or VISUAL MATERIALS
Use 008 29 mapping"

I don't have the capacity to be the reviewer on this in the next couple of weeks. If someone else can pick up the mapping review, I would be grateful. Since it relies on 008 values it might make sense to hold off on coding until the 008 mapping is finished?

@lake44me
I just marked it as reviewed, everything looks good to me. I added the link you provided me in a previous email to OCLC (https://www.oclc.org/bibformats/en/fixedfield/type.html) to help determine the format, which will be necessary for the transform.

Thanks Cypress!  I moved 533 to Ready for Transform

code on hold for $7/01-04 and $7/05-08 until #50 is updated

Hi @lake44me, I just updated 245 based on reproduction conditions and ran 5 records with 533 or 588 reproduction conditions through the transform to show what the output currently looks like. Not all fields that have reproduction conditions have been mapped at this point, but 006, 008, 533, and 264 should be set.
The 5 records are here, and output is in this folder if you would like to take a look!


instructions to use yaml_requests on Ubuntu (23.04)

Hello :)
Problem
I can't seem to get yaml_requests to work on Ubuntu (23.04).
Environment
]$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 23.04
Release:	23.04
Codename:	lunar

]$ python --version
Python 3.11.4

What I attempted

on the one hand, when I try to install using pip I get:]$ pip install yaml_requests
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

and on the other hand, when I use pipx I get:]$ pipx install yaml_requests
Fatal error from pip prevented installation. Full pip output in file:
    /home/me/.local/pipx/logs/cmd_2023-08-01_13.04.57_pip_errors.log

pip seemed to fail to build package:
    pyyaml~=5.0

Some possibly relevant errors from pip install:
    error: subprocess-exited-with-error
    AttributeError: cython_sources

Error installing yaml-requests.

Caveats
I don't typically work with python so bare with me if this is a truly newb issue.

I tried the developer approach:
]$ python3 setup.py build
]$ python3 setup.py install

and then I was able to do
]$ ./bin/yaml_requests tst/plans/minimal_plan.yml
Sending 1 requests:

✔ GET https://google.com
  HTTP 200 (408.060 ms)


All the nodes are shown as unreported

Hello,
I'm using puppetboard with Docker, I have downloaded the source from github and I build the image locally:
root@sofx-dckr024:~/puppetboard# cat Dockerfile
FROM python:3.8-alpine

ENV PUPPETBOARD_PORT 80
EXPOSE 80

ENV PUPPETBOARD_SETTINGS docker_settings.py
RUN mkdir -p /usr/src/app/
WORKDIR /usr/src/app/

# Workaround for https://github.com/benoitc/gunicorn/issues/2160
RUN apk --update --no-cache add libc-dev binutils

COPY requirements*.txt /usr/src/app/
RUN pip install --no-cache-dir -r requirements-docker.txt

COPY . /usr/src/app

CMD gunicorn -b <IP_ADDRESS>:${PUPPETBOARD_PORT} --workers="${PUPPETBOARD_WORKERS:-1}" -e SCRIPT_NAME="${PUPPETBOARD_URL_PREFIX:-}" --access-logfile=- puppetboard.app:app
root@sofx-dckr024:~/puppetboard#

And starting the container in that way:
root@sofx-dckr024:/docker/containers/sofk-puppetboard01# cat sofk-puppetboard01.sh
#!/bin/bash

docker run --detach \
  --hostname sofk-puppetboard01.home.lan \
  --name sofk-puppetboard01.home.lan \
  --add-host puppet.home.lan:<IP_ADDRESS> \
  --network vlan310 --ip <IP_ADDRESS> --dns <IP_ADDRESS> \
  -v /docker/containers/sofk-puppetboard01/ssl:/etc/puppetlabs/puppetdb/ssl \
  -e PUPPETDB_HOST=puppet.home.lan \
  -e PUPPETDB_PORT=8081 \
  -e PUPPETDB_SSL_VERIFY=/etc/puppetlabs/puppetdb/ssl/ca.pem \
  -e PUPPETDB_KEY=/etc/puppetlabs/puppetdb/ssl/private.pem \
  -e PUPPETDB_CERT=/etc/puppetlabs/puppetdb/ssl/public.pem \
  -e INVENTORY_FACTS='Hostname,fqdn, IP Address,ipaddress' \
  -e GRAPH_FACTS='architecture,puppetversion,osfamily' \
  -e ENABLE_CATALOG=True \
  puppetboard
root@sofx-dckr024:/docker/containers/sofk-puppetboard01#

As preparation of that I already collected puppet CA certificates, and mounted that directory in the container.
I also have generated certificate for that service, and puppet sees this certificates:
root@puppet.home.lan:~# puppetserver ca generate --certname sofk-puppetboard01.home.lan
Successfully saved private key for sofk-puppetboard01.home.lan to /etc/puppetlabs/puppet/ssl/private_keys/sofk-puppetboard01.home.lan.pem
Successfully saved public key for sofk-puppetboard01.home.lan to /etc/puppetlabs/puppet/ssl/public_keys/sofk-puppetboard01.home.lan.pem
Successfully submitted certificate request for sofk-puppetboard01.home.lan
Error:
    Signed certificate sofk-puppetboard01.home.lan could not be found on the CA
Successfully signed certificate request for sofk-puppetboard01.home.lan
Successfully saved certificate for sofk-puppetboard01.home.lan to /etc/puppetlabs/puppet/ssl/certs/sofk-puppetboard01.home.lan.pem
root@puppet.home.lan:~# puppetserver ca list --all
Signed Certificates:
    sofk-puppetboard01.home.lan       (SHA256)  B1:9E:FC:C6:8E:89:83:15:AD:B3:63:6E:66:39:F9:5C:67:75:11:C6:B2:6A:3A:38:CE:67:F7:4E:8C:C9:22:DE
    puppet.home.lan                   (SHA256)  5A:A4:36:64:20:6E:73:18:6E:DF:6A:C2:DE:16:7C:26:AE:DA:56:2E:25:65:1E:60:3A:30:53:E5:BE:91:38:9C alt names: ["DNS:puppet.home.lan", "DNS:puppet", "DNS:puppet.home.lan"]       authorization extensions: [pp_cli_auth: true]
root@puppet.home.lan:~#

So I expect everything to work, but I only see the puppet dashboard and all the nodes as unreported.
Although I can browse their facts. Can you help me with that.
Thanks.

My apologies, completely my fault.
For some reasons I've missed the "store facts" setting in puppet.conf, that's why all of the nodes was shown as unreported.
Will leave this one here, and hope it will help to someone.

We are that you have solved your issue! 😊


Cell crashes when the value is negative and threshold is enabled

Steps to reproduce:

Create a graph cell with the following query. I have a measurement 'cours' with the actual stock price of a share and another 'patrimoine' which has a cost price field. I join the two measurement to have the delta between the actual price and the price I got the share.

cours = from(bucket: "bourse")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "cours")
  |> filter(fn: (r) => r["_field"] == "price")
  |> filter(fn: (r) => r["nom"] == v.action)
  |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)

prix = from(bucket: "bourse")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "patrimoine")
  |> filter(fn: (r) => r["_field"] == "prix_revient")
  |> filter(fn: (r) => r["nom"] == v.action )
  |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)

join(tables: {cours:cours, prix:prix}, on: ["_time", "_stop","_start"])
|> map(fn: (r) => ({
    _time: r._time,
    _value: (r._value_cours - r._value_prix)
}))
|> yield(name: "mean")

Convert the cell into a Graph + Single Stat or into Single Stat
Add a threshold (base -> 'fire' / value >= 0 -> 'rainforest')

Actual behavior:
When the delta is negative, the cell crashes.
Video demonstration :
Expected behavior:
When the delta is negative, the cell doesn't crashes !
Environment info:

System info: Linux 5.4.0-65-generic x86_64
InfluxDB version: Influx CLI 2.0.3 (git: fe04d346df) build_date: 2020-12-15T01:00:16Z

Thanks for your help !
Paul Brissaud

@pbrissaud - would it be possible for you to include your data from this query into this ticket? I am looking for the CSV that is given by the query you posted in the description. Please attach it as a .csv file if you can. That would make it easier to reproduce. Thank you!

@TCL735 thanks for you help, here the csv_file.

@pbrissaud would it be possible to allow access to me as well? trying to use the csv but getting denied access.

@ShmuelLotman now it should be good :)


return proper error message if reading for configmap file fails

currently, we are returning Internal server error in case anything fails from https://github.com/ceph/ceph-csi/blob/74ba85f87b1526e615f1643d3475316137a18264/internal/rbd/rbd_util.go#L666-L681 we should return internal server error or invalid request based on the error messages from the caller.

Fixed in #1295


data_aug resulting index outside the box

data_aug in parse_annotation dataset.py resulting index value outside the array of label in preprocess_true_boxes (dataset.py). So, it is suffered with error (IndexError: index xx is out of bounds for axis 0 with size yy),(xx could be negative?) during training process. please advise how to handle the problem ?

I got the same problem? Have you solved it？

I have the same problem in (dataset.py)  and in (core\dataset.py") IndexError: list index out of range. Have you solved it?

make sure 0<box_height<image_height, 0<box_width<image_width

@fxcdl after the training process you need to export the weights again(generate .pb file)?
'$ python convert_weight.py'
'$ python freeze_graph.py'
Or once the training is done you just use: '$ python image_demo.py'?
Thank you!

I solved it, please be aware to your data. in my case, it is due to wrong bounding box and corrupt files.


Fix webpack bug where it copies extra root-level dependency

tl;dr The databases vsix is broken when I use the latest webpack stuff. This should fix it and I think it's low risk in the sense that we'd know pretty fast if it broke anything else.

I ran into this error for Databases, related to which "external" dependencies it copies over to the "dist" folder
Activating extension 'ms-azuretools.vscode-cosmosdb' failed: Cannot find module 'lru-cache'
Require stack:
- /Users/erijiz/.vscode-insiders/extensions/ms-azuretools.vscode-cosmosdb-0.16.0/dist/node_modules/semver/classes/range.js
...

It's a little complicated, but I'll try to explain. Say you have "pg" and "semver" as your root dependencies in package.json and they each have their own sub dependencies like this:

<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

There's two versions of semver involved, and only one depends on lru-cache. You're going to get a node_modules folder like this:

Here is what happens when you list "pg" as an "external".
Expected
It copies this over to dist

Actual
It copies this over to dist

The problem is it copied over the root<EMAIL_ADDRESS>even though we're already copying the<EMAIL_ADDRESS>under "pg". That alone might not be a problem (we just get some extra files), but it did not copy lru-cache, which is where I start to see errors.

Am I crazy, or are all three of those screenshots the same?

Yes the screenshot is the same, but the red box is different :P


Cannot run stable-diffusion-webui

When attempting to ./run.sh stable-diffusion-webui:r35.4.1, the code crashes with
+ docker run --runtime nvidia -it --rm --network host --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /home/rp/jetson-containers/data:/data --device /dev/snd --device /dev/bus/usb --name aa stable-diffusion-webui:r35.4.1 Python 3.8.10 (default, Nov 22 2023, 10:22:35) [GCC 9.4.0] Version: v1.7.0 Commit hash: cf2772fab0af5573da775e7437e6acdca424f26e Launching Web UI with arguments: --data=/data/models/stable-diffusion --enable-insecure-extension-access --xformers --listen --port=7860 Traceback (most recent call last):   File "launch.py", line 48, in <module>     main()   File "launch.py", line 44, in main     start()   File "/opt/stable-diffusion-webui/modules/launch_utils.py", line 460, in start     import webui   File "/opt/stable-diffusion-webui/webui.py", line 13, in <module>     initialize.imports()   File "/opt/stable-diffusion-webui/modules/initialize.py", line 34, in imports     shared_init.initialize()   File "/opt/stable-diffusion-webui/modules/shared_init.py", line 17, in initialize     from modules import options, shared_options   File "/opt/stable-diffusion-webui/modules/shared_options.py", line 3, in <module>     from modules import localization, ui_components, shared_items, shared, interrogate, shared_gradio_themes   File "/opt/stable-diffusion-webui/modules/localization.py", line 4, in <module>     from modules import errors, scripts   File "/opt/stable-diffusion-webui/modules/scripts.py", line 10, in <module>     from modules import shared, paths, script_callbacks, extensions, script_loading, scripts_postprocessing, errors, timer   File "/opt/stable-diffusion-webui/modules/script_callbacks.py", line 261, in <module>     def infotext_pasted_callback(infotext: str, params: dict[str, Any]): TypeError: 'type' object is not subscriptable
I investigated a bit and found out that recently the source code was changed from "params: Dict[str, Any]):" to "params: dict[str, Any]):" and Dict is no longer being imported. As "dict" was introduced in Python 3.9, the code from  "https://github.com/AUTOMATIC1111/stable-diffusion-webui)" is no longer compatible with Python 3.8, which we seem to be using in the Dockerfile.

so how to deal thsi bug, ths

@RPHllc @WalkerZJT this is a similar issue to https://github.com/dusty-nv/jetson-containers/issues/364 regarding Python 3.9/3.10 typing hints.   Typically I would just patch these in the dockerfile with sed commands.  Or on JetPack 5, we can lock the stable-diffusion-webui commit to before these changes were made (which given that JetPack 5 is only receiving minor updates/security fixes now that JetPack 6 is out, may be more appropriate)

😁ths
---Original---
From: "Dustin @.>
Date: Wed, Jan 31, 2024 03:50 AM
To: @.>;
Cc<EMAIL_ADDRESS>Subject: Re: [dusty-nv/jetson-containers] Cannot run stable-diffusion-webui (Issue #369)
@RPHllc @WalkerZJT this is a similar issue to #364 regarding Python 3.9/3.10 typing hints.   Typically I would just patch these in the dockerfile with sed commands.  Or on JetPack 5, we can lock the stable-diffusion-webui commit to before these changes were made (which given that JetPack 5 is only receiving minor updates/security fixes now that JetPack 6 is out, may be more appropriate)
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

I moved on to JetPack 6. There are other version compatibility issues with version 35.4.1, we are probably better off moving on. Thank you.

OK, so I tried patching it for Python 3.8, but there kept being more and more patches required, so I ended up just locking it to v1.6.0 on JetPack 5 in commit https://github.com/dusty-nv/jetson-containers/commit/d8f50633d19b8e460eade442fda19235104c4c7e .  A1111 v1.6.0 was the last version that would work with the minimal patch set.   On JetPack 6, it still uses A1111 master branch and Python 3.10.
I confirmed that SD-1.5 and SDXL are still working on JetPack 5 with A1111 v1.6.0, and the stable-diffusion-webui containers are all rebuilding now.


URL Build Updates

Update on how we build URL for API and Chat, taking things like the hublet and environment
https://github.com/user-attachments/assets/85a03afb-cb36-43d8-9ddc-5240620246a3

Thanks! @tuckertrainor
Unfortunately I do not have rights to merge this branch. Is this something that you could merge or you'd like me to request permission to somebody?


Project specific environment variables

Is the .zopen-config able to pick up project specific environment variables?
For example, vim has the VIM environment variable to specify it's runtime, which is currently set in the project's .env file.

.zopen-config should project specific env files in etc/profiled/, which should source the .env
Igor to test it out

As discussed, ports/packages that have a .env will have that file sourced during initialisation of the zopen system, sourced from within the project's install location so custom envvars should be configured as expected. Awaiting f/b...

I have tested vim , man-db and git. There were no issues. Hence closing it.


NoSuchMethodError exception when invoked from Java (Jetty)

Hello Jose,
We are trying to call shaclex from a java webapp project running inside jetty, but getting the following exception:
java.lang.NoSuchMethodError: es.weso.typing.Typing.getMap()Lscala/collection/immutable/Map;
	at es.weso.schema.ShaclexSchema.cnvShapeTyping(ShaclexSchema.scala:33)
	at es.weso.schema.ShaclexSchema$$anonfun$cnvResult$1.apply(ShaclexSchema.scala:28)
	at es.weso.schema.ShaclexSchema$$anonfun$cnvResult$1.apply(ShaclexSchema.scala:28)
	at scala.collection.immutable.List.map(List.scala:273)
	at es.weso.schema.ShaclexSchema.cnvResult(ShaclexSchema.scala:28)
	at es.weso.schema.ShaclexSchema.validateTargetDecls(ShaclexSchema.scala:21)
	at es.weso.schema.Schema.validateWithTrigger(Schema.scala:22)
	at es.weso.schema.Schema.validate(Schema.scala:39)
	at your.basepackage.name.SampleAdaptorManager.getServiceProviderInfos(SampleAdaptorManager.java:108)

POM file: pom.xml.zip
shaclex jars were installed locally this way: https://gitlab.com/assume/shaclex/tree/fixmvn#local-m2-installation (which was probably far from optimal)
cc @jadelkhoury

These are the jars that are packaged in the WAR file:
ᐅ l WEB-INF/lib | grep scala
json4s-scalap_2.11-3.3.0.jar
org.scala-lang-scala-library-2.11.8.jar
org.typelevel-scala-library-2.11.8.jar
scala-compiler-2.11.8.jar
scala-java8-compat_2.11-0.5.0.jar
scala-logging_2.11-3.5.0.jar
scala-parser-combinators_2.11-1.0.4.jar
scala-reflect-2.11.8.jar
scala-xml_2.11-1.0.4.jar
scalacheck_2.11-1.13.4.jar
scalactic_2.11-3.0.0.jar
scalaz-concurrent_2.11-7.2.5.jar
scalaz-core_2.11-7.2.6.jar
scalaz-effect_2.11-7.2.5.jar
scalaz-stream_2.11-0.8.4a.jar

It looks as a problem with the packaged Jars. I will see if I can reproduce it.
You may try to generate a fat jar from sbt using.
sbt assembly

Anyway, I will try to create a simpler interface to enable better interoperability from Java applications. See  issue #17

Thank you! I was thinking that I did something wrong with the SBT config.
This is the minimal webapp project that uses shaclex: https://gitlab.com/assume/adaptor-shacl
I will try sbt assembly to compare the contents of the fat jar from sbt and our war. I think publishing to bintray might solve these errors (#18).

The bug in my code seems to be hitting this: https://github.com/labra/shaclex/commit/afc1e5549b2dbaa2c9d85117d2cb29a1e83342cc
I built my local jars from master and I must have built not all the projects and the missing ones (from the commit above it seems to be shex) must have been loaded from bintray. Could you please tag the revision, which corresponds to 0.0.55 on bintray, please?
But I tried using 0.0.54 from bintray and got the following:
Caused by: java.lang.NoSuchMethodError: es.weso.typing.Typing$.empty(Lcats/Show;Lcats/Show;Lcats/Show;Lcats/Show;)Les/weso/typing/Typing;
	at es.weso.shacl.Validator.runCheck(ValidatorCats.scala:69)
	at es.weso.shacl.Validator.validateAll(ValidatorCats.scala:608)
	at es.weso.schema.ShaclexSchema.validate(ShaclexSchema.scala:20)
	at your.basepackage.name.SampleAdaptorManager.getServiceProviderInfos(SampleAdaptorManager.java:133)
	at your.basepackage.name.servlet.ServiceProviderCatalogSingleton.initServiceProviders(ServiceProviderCatalogSingleton.java:276)
	at your.basepackage.name.servlet.ServiceProviderCatalogSingleton.getServiceProviderCatalog(ServiceProviderCatalogSingleton.java:104)
	at your.basepackage.name.services.ServiceProviderCatalogService.getHtmlServiceProvider(ServiceProviderCatalogService.java:144)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)

I also copied your code from apiShaclex almost verbatim, so I start to suspect Jetty/Scala problems or I might have messed up with the exclusions.
apiShaclex is a very good starting point for using shaclex library in general (thank you!). I think both links to it and to https://bintray.com/labra/maven/shaclex# should be in the readme.

I have just published version 0.0.56 with the latest code in the repository. Could you try it?
My concern is that the jar published only contains the files of that module and omits the rest of the modules in the library which are published in different places, for example, shex code is published here and so on...in principle, Maven should take care of the dependencies, but I am not sure if there could be some hidden problem related with the use of the Typelevel compiler.
You may have seem that I created a new issue to update the code to use the latest Scala 2.12 compiler. I am planning to do that in the next days once I check that all the libraries have been published for 2.12 also.

Thank you for such a quick reply!
I tried it and it doesn't help. To move our debugging effort forward, here is a clean repo without jax-rs with just 3 unit tests and a reproducible build: https://gitlab.com/assume/shacl-try-standalone.
I looked at https://dl.bintray.com/labra/maven/es/weso/ (I added the https://dl.bintray.com/labra/maven/ repo to my pom file) and all the libraries seem to be there.
Can't really comment on Typelevel compiler and/or on its version as I have no idea what impact do they have.
cc @jadelkhoury

Thanks! Iwill try to see if I can spot the problem...although I suspect there may be some internal problem in the compiler and it may be better to just update the compiler to Scala 2.12...this action will take me some time but the effort may be worthwhile in the long term.

I have just updated to Scala 2.12 compiler and published a new version 0.0.57. Could you try it with the new version?

Thank you for the update! There seem to be 3 problems:

maven tries to resolve com.typesafe.play:twirl-api_2.12:jar:1.2.0 (which should be 1.3.0 - https://www.mvnrepository.com/artifact/com.typesafe.play/twirl-api_2.12/1.3.0).
maven tries to include some 2.10.x and 2.11.x dependencies still.
maven doesn't fail the build due to the first problem on my machine, so I might have some .m2 cleaning to do.

Full build log is available at https://gitlab.com/assume/shacl-try-standalone/builds/8340198

I have just published version 0.0.58 which fixes the problem with the twirl-api jar and now uses 1.30 (thanks for reporting it).
Anyway, the bad news is that the original problem remains.
I have updated this old project called shexjava where I just call the library from a simple command line java program and it also raises the exception:
Exception in thread "main" java.lang.NoSuchMethodError: es.weso.typing.Typing.getMap()Lscala/collection/immutable/Map;
        at es.weso.schema.ShExSchema.cnvShapeTyping(ShExSchema.scala:41)
        at es.weso.schema.ShExSchema.$anonfun$cnvResult$1(ShExSchema.scala:36)
        at scala.collection.immutable.List.map(List.scala:272)
        at es.weso.schema.ShExSchema.cnvResult(ShExSchema.scala:36)
        at es.weso.schema.ShExSchema.validateTargetDecls(ShExSchema.scala:27)
        at es.weso.schema.Schema.validateWithTrigger(Schema.scala:22)
        at es.weso.schema.Schema.validate(Schema.scala:39)
        at es.weso.shexjava.ShExValidator.validate(ShExValidator.java:27)
        at es.weso.shexjava.ShExValidator.validate(ShExValidator.java:48)
        at es.weso.shexjava.Main.main(Main.java:34)

So now, it is working with the Scala 2.12 compiler and we can discard a problem with a compiler plugin that I was suspecting.
At this moment I have no clue why it doesn't find that method 😕

I looked a lot through similar questions on SO and in most cases the solution was to change some library version. If you search for "2.10" or "2.11", you'll find a lot of results in your build log: https://travis-ci.org/labra/shaclex
I discovered https://github.com/jrudolph/sbt-dependency-graph and will try checking the dependencies thoroughly.

I think I have found (and solved) the problem. It was a duplicated set of files which where both in the rbe... and typing... jars and made their corresponding classes to overlap.
I have published version 0.0.60 which I think works.
Could you try it?

Great! 0.0.60 works like a charm! https://gitlab.com/assume/shacl-try-standalone build passes now. Not sure if it could become a good sample Java app for the library in the future, we plan to do some simple tests with Jena model.
CC @jadelkhoury


Add logic to parse starter team URLs

Description
Figma has different pricing / plans - see link. For starter teams, we do not support webhooks. In the figma-for-jira configuration page, currently, if you put a starter team URL, then you will see an error like You do not have admin permissions for this team.
This PR updates the flow to show a more descriptive error message.
Fixes: https://figlassian.atlassian.net/jira/software/projects/FABB/boards/3?assignee=64139c639d2bc6c90a8a20fd&selectedIssue=FABB-2
Testing
I wrote a unit test that passes before and after.

I tested the changes in the UI.

Ha, as it turns out - webhooks aren't enabled for starter accounts

hence why the request is failing.
Unfortunately this isn't super well documented but @akostevich-atlassian do you have thoughts here RE: how best to parse error responses from Figma's REST API?
This isn't well documented but from testing, I found that this call will return:
{
  error: true,
  status: 400,
  message: 'Access Denied',
  i18n: null,
  reason: 'Upgrade to professional team to enable webhooks'
}

in error.response?.data.
Updating HttpClientError to something like:
export class HttpClientError extends CauseAwareError {
  constructor(message: string, cause: AxiosError) {
    super(message, cause);
  }
}

could work since we'd be able to access error.cause.response?.data but this didn't feel compatible w the other errors that extend CauseAwareError eg. MissingOrInvalidCredentialsFigmaAuthServiceError. WDYT?

Good question, @Roystbeef !
I believe there 2 possible solutions here.
1. Use a team URL as an indicator of an account type
Our frontend could use a given URL to determine a Figma Starter account and show an error even before hitting API.
This solution can be the easiest to implement but we need to consider whether a Figma URL structure is reliable indicator of a Figma account type (e.g., are there any other URL formats, is a URL subject to change?).
2. Use API response to show an error
Our frontend could show a more specific error using a response from the POST admin/teams/:teamId/connect endpoint.
I suggest to show something like "Cannot connect the team. Please, verify that you have a Figma Professional account and are the admin of the team." when the endpoint returns HTTP 403 Forbidden.
Currently, POST admin/teams/:teamId/connect, likely, returns HTTP 500 Internal Server Error for this use case, so we need to extend error handling for it to consider the new error.
Before jumping in into how to do this, please, let me provide the overview of the error handling design in the app. Overall, the app has several layers of errors:

Base Errors. They represent errors, which extend a built-in Error with additional details and can be used as a base class for any other errors in the app. Currently, there is only one base error, common/errors/CauseAwareError, which include information on the original cause of an error (also known as an inner error).
External Client Errors. They represent app-agnostic errors returned from external system (e.g., HttpClientError). These errors are thrown by corresponding clients. For example, FigmaClient throws HttpClientError.
Service Errors. They represent domain-specifc errors, which we are interested in our business logic (e.g., MissingOrInvalidCredentialsFigmaAuthServiceError, UnauthorizedFigmaServiceError). There errors are thrown by services. Usually, our services translate relevant External Client Errors to Service Errors (see an example here) or throw the latter directly.
Use Case Errors. They represent app-specific user-facing errors of a business use case (e.g.,  ForbiddenByFigmaUseCaseResultError). Our use cases translate relevant Service Errors to Use Case Error . There errors are mapped to corresponding HTTP codes and returned to a user.

For our problem, I recommend the following:

Add BadRequestHttpClientError to /infrastructure/http-client-errors.ts, which corresponds to HTTP 400 Bad Request (we haven't added it before since we haven't been interested in these errors).
Extend HttpClientError to include response data. A response will be needed to determine whether an error on webhook creation is a Starter account-related error

export class HttpClientError extends CauseAwareError {
  constructor(message: string, readonly response?: unknown, cause?: unknown) {
  	super(message, cause);
  }
}

Update translateAxiosError to translate HTTP 400 Bad Request to BadRequestHttpClientError. Include response error.response.data

const translateAxiosError = (error: unknown): unknown => {
  if (!isAxiosError(error)) return error;

  switch (error.response?.status) {
  	case HttpStatusCode.BadRequest:
  		return new BadRequestHttpClientError(error.message, error.response.data, error);
  	case HttpStatusCode.Unauthorized:
  		return new UnauthorizedHttpClientError(error.message, error.response.data, error);
  	// ...
  }
};

Update FigmaService.createFileUpdateWebhook to translate BadRequestHttpClientError (thrown from figmaClient.createFileUpdateWebhook) with error.response.message === 'Access Denied' to UnauthorizedFigmaServiceError and throw the latter further. You might need to check a type of error.response using validateSchema.

Considering that connectFigmaTeamUseCase.execute already handles UnauthorizedFigmaServiceError(🙌 ), the result of these steps should be POST admin/teams/:teamId/connect returning HTTP 403 Forbidden. Done!
Please, let me know, @Roystbeef , if you would like to discuss this in more detail, need help or have any questions.

@akostevich-atlassian this makes sense to me! Thanks for laying this out!

@akostevich-atlassian Thank you for the detailed guidance - it was very helpful. I updated the PR to implement the suggestions. I'm OOO for the next week. I'll be back Tue 11/28. @Roystbeef will take point on wrapping this up.

@elainefigma , I created a separate PR to address the comments here:
https://github.com/atlassian-labs/figma-for-jira/pull/171
Thank you very much for your great work on this!


Scroll down to flag section automatically in chrome 57

In old version, works well and then when scrolling, automatically go to flag section.
In ie11 and edge works now.

fixed


Can't read body on manifest requests on errors (HEAD request)

Our video backend returns a detailed error message in the manifest request body on errors like 403 Forbidden. This information is used to tell the user what went wrong and also for internal logging purposes. However, we can't seem to figure out how to get this information using shaka2.0.
The reason to our problem is that shaka does a HEAD request to probe the backend for content type to figure out what parser to use. If the HEAD request gives 403 Forbidden shaka won't even do the GET request and we will miss out on the detailed message.
We have identified a few different approaches to this problem.

Do we really need to make both HEAD and GET requests? What if shaka could do the GET request immediately and re-use the response for parsing later (after figuring out parser type).

A way to force a certain parser type immediately. In our case we know that the content will always be dash so there is no point in doing the HEAD request at all.

By exporting the dash parser we could pass this in during load and avoid probing for manifest type.

Do you see more approaches or perhaps even solutions to our problem without modifying the source code? We realize that we can possibly do this with a request interceptor and fake a 200 response code to force the client to do the GET but it feels extremely hacky.

Hi Patrik,
We only do a HEAD request to probe the manifest type if we are unable to deduce the type from the manifest URI's file extension.  If your URI ends with .mpd (not counting any GET parameters), then we will not have to probe with a HEAD request.
We do provide the HTTP response body when there's an HTTP error, but this only works for GET and POST requests.  A HEAD request does not result in a response body, so we can't provide the text.  The best way to get the error text for a manifest request is to avoid the HEAD request.
As you've noticed, Player.load()  has an optional parameter opt_manifestParserFactory that allows you to specify the manifest parser and avoid the registry completely.  We intended this for custom manifest parsers rather than for overriding detection.  This is why we don't export DashParser already.
We'd be happy to export DashParser going forward because that could be useful for other reasons, but we still feel that the best way to solve this problem is to have a recognizable file extension in the URI.
Does this help?

Joey,
Thanks for the response. I believe it might get difficult for us to change our backend at this moment unfortunately. I will discuss this with the team but having DashParser exported would suit us much better.

I filed a new issue to export all plugins in general.

Closing this issue now.  To track progress on exporting plugins, follow #551.  Thanks!


Solution

Ok, I have done another PR, from branch main with reference to a workflow with Docker CI job.


multi node instance use broadcastStateless crashed

Description
multi node instance use broadcastStateless crashed, and I submitted a pr #556
Steps to reproduce the bug

There are 2 hocuspocus instance managed by pm2, which connect to the same redis with extension-redis, and the 2 hocuspocus instance  listen port 3001 and 3002.
Client1 connect 3001  and Client2 connect 3002,
If client1 call provider.sendStateless('hello') and hocuspocus-instance1 will broadcast msg to all connctions which will  publish message to Redis channel through  beforeBroadcastStateless

    const message = new OutgoingMessage()
      .writeBroadcastStateless(data.payload)

    return this.pub.publishBuffer(
      this.pubKey(data.documentName),
      Buffer.from(message.toUint8Array()),
    )
  }

Client2 recevied message through

   this.sub.on('pmessageBuffer', this.handleIncomingMessage)

and handle it in
  private handleIncomingMessage = async (channel: Buffer, pattern: Buffer, data: Buffer) => {
  const channelName = pattern.toString()
  const [_, documentName, identifier] = channelName.split(':')
  const document = this.documents.get(documentName)

  if (identifier === this.configuration.identifier) {
    return
  }

  if (!document) {
    return
  }

  new MessageReceiver(
    new IncomingMessage(data),
    this.logger,
  ).apply(document, undefined, reply => {
    return this.pub.publishBuffer(
      this.pubKey(document.name),
      Buffer.from(reply),
    )
  })
}

 //and in apply function will call
  ...
  case MessageType.BroadcastStateless:
      document.getConnections().forEach(connection => {
        connection.sendStateless(message.readVarString())
      })

the message.readVarString() will cause crash because message can only read once, when read Again it will throw Integer out of range if we try and catch apply function.
and the the readVarString in lib0/decoding.js as follow, message is like an stream, offset pos will change after readVarString or readVarUint
export const readVarString = decoder => {
  let remainingLen = readVarUint(decoder)
  if (remainingLen === 0) {
    return ''
  } else {
    let encodedString = String.fromCodePoint(readUint8(decoder)) // remember to decrease remainingLen
    if (--remainingLen < 100) { // do not create a Uint8Array for small strings
      while (remainingLen--) {
        encodedString += String.fromCodePoint(readUint8(decoder))
      }
    } else {
      while (remainingLen > 0) {
        const nextLen = remainingLen < 10000 ? remainingLen : 10000
        // this is dangerous, we create a fresh array view from the existing buffer
        const bytes = decoder.arr.subarray(decoder.pos, decoder.pos + nextLen)
        decoder.pos += nextLen
        // Starting with ES5.1 we can supply a generic array-like object as arguments
        encodedString += String.fromCodePoint.apply(null, /** @type {any} */ (bytes))
        remainingLen -= nextLen
      }
    }
    return decodeURIComponent(escape(encodedString))
  }
}

and now the error happen:
You have triggered an unhandledRejection, you may have forgotten to catch a Promise rejection:
Error: Integer out of range!
    at readVarUint (file:///export/backend/node_modules/node_modules/lib0/decoding.js:247:13)
    at readVarString (file:///export/backend/node_modules/node_modules/lib0/decoding.js:328:22)
    at IncomingMessage.readVarString (file:///export/backend/node_modules/@hocuspocus/server/src/IncomingMessage.ts:46:12)
    at file:///export/backend/node_modules/@hocuspocus/server/src/MessageReceiver.ts:82:44
    at Array.forEach (<anonymous>)
    at MessageReceiver.apply (file:///export/backend/node_modules/@hocuspocus/server/src/MessageReceiver.ts:81:35)
    at Redis.handleIncomingMessage (file:///export/backend/node_modules/@hocuspocus/extension-redis/src/Redis.ts:282:7)
    at Redis.emit (events.js:400:28)
    at DataHandler.handleSubscriberReply (/export/backend/node_modules/ioredis/built/DataHandler.js:89:28)
    at DataHandler.returnReply (/export/backend/node_modules/ioredis/built/DataHandler.js:47:18)
    at JavascriptRedisParser.returnReply (/export/backend/node_modules/ioredis/built/DataHandler.js:21:22)
    at JavascriptRedisParser.execute (/export/backend/node_modules/redis-parser/lib/parser.js:544:14)
    at Socket.<anonymous> (/export/backend/node_modules/ioredis/built/DataHandler.js:25:20)
    at Socket.emit (events.js:400:28)
    at addChunk (internal/streams/readable.js:293:12)
    at readableAddChunk (internal/streams/readable.js:267:9)
You have triggered an unhandledRejection, you may have forgotten to catch a Promise rejection:

hocuspocus configure like this.
  const server = Server.configure({
  name: 'hocuspocus-instance',
  port: config.server.port, // managed by pm2 range[3001, 3002]
  address: config.server.host,
  extensions: [
    new Redis({
      host: config.redis.host,
      port: config.redis.port,
      options:{
        password: config.redis.password,
      },
      prefix:config.redis.keyPrefix,
    }),
  ],
  async connected() {
    console.log('connections:', server.getConnectionsCount())
  },
  async onStateless(data:onStatelessPayload) {
    const { connection, documentName, document, payload} = data
    const info = JSON.stringify({"event":"this is a broadcast msg"})
    document.broadcastStateless(info)
  },
})

server.listen()

Expected behavior
Client A and Client B can receive broadcast messages through the redis channel
Screenshot, video, or GIF
If applicable, add screenshots to help explain your problem.
Environment?

operating system: CentOS Linux release 7.9.2009 (Core)
browser:  chrome 111.0.5563.111
mobile/desktop:
Hocuspocus version: 1.1.1

Additional context
Add any other context about the problem here.

Thanks! I've just released v2.0.1 including your fix :)


Added Portuguese Language

Hi Edlyn!
I've just added Portuguese (pt) to the list of supported languages.
Thanks for your work on this calendar, I have used it in a few projects already!

I've been wondering why there's no approve options in the review changes... :sweat_smile:


make maxConcurrentReconciles configurable

Please provide a description of this PR:

@zirain: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
b5d82f5e8fbab502daa04ff1acace9d187106277
link
true
/test release-notes

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Partially fixes broken py26 support #141

This PR removes the dictionary comprehensions. shutil.make_archive is used as well (added in py27) which will need to be changed. There may be other changes required as well.

Thanks for the PR @Vye, can you create another on against the develop branch?

Closed in favor of #143


MODE-2573 - Using OFFSET on queries which use indexes loops over the same first 100 nodes

Test Case demonstrating issue.

superseded by https://github.com/ModeShape/modeshape/pull/1533


Problem with gdbinit

Hey, me again. gdb 7.4.1-debian is giving me this error:
Exception <type 'exceptions.ImportError'> raised while loading Voltron: No module named blessed
/home/admin/.gdbinit:2: Error in sourced command file:
Undefined command: "voltron".  Try "help".

I'm assuming this is from the 'import blessed' line in dbgentry.py. Any ideas?
Sorry I'm treating this as a forum since there's no official forum for voltron. Cheers

@defau1t Although past one more years, I meet the same problem ''No module named blessed" and found that's python version problem.
1). The error cause by the python version:

Show Ubuntu 14.04 publish the gdb with python3-interface

$ gdb
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.

warning: ~/.gdbinit.local: No such file or directory
Exception <class 'ImportError'> raised while loading Voltron: No module named 'blessed'
/home/wilson/.gdbinit:3834: Error in sourced command file:
Undefined command: "voltron".  Try "help".
(gdb) python-interactive
>>> import sys
>>> print sys.version
  File "<stdin>", line 1
    print sys.version
            ^
SyntaxError: Missing parentheses in call to 'print'
>>> import blessed
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'blessed'
>>> print(sys.version)
3.4.3 (default, Oct 14 2015, 20:31:36)
[GCC 4.8.4]
>>>
(gdb) quit

2). So we should change the setup steps a little:

For gdb use python3, so we should install module by pip3, not pip

cd <your-work-dir>
git clone https://github.com/snare/voltron.git
cd voltron
sudo pip3 install flask          <<< used by next voltron setup
sudo python3 setup.py install
sudo pip3 install blessed        <<< used by voltron/dbgentry.py

If voltron view backtrace have connect error, please refer the comment #97.

This renewed my interest in this project. It loads properly now! Just have that connect issue but will comment in that thread. Thanks @huawenyu !

https://github.com/snare/voltron/wiki/Voltron-on-Ubuntu-14.04-with-GDB https://github.com/snare/voltron/wiki/Voltron-on-Ubuntu-14.04-with-GDB

On 14 Jan 2016, at 5:22 PM, Huawen Yu<EMAIL_ADDRESS>wrote:
@defau1t https://github.com/defau1t Although past one more years, I meet the same problem ''No module named blessed" and found that's python version problem.
1). The error cause by the python version:
Show Ubuntu 14.04 publish the gdb with python3-interface
$ gdb
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
warning: ~/.gdbinit.local: No such file or directory
Exception <class 'ImportError'> raised while loading Voltron: No module named 'blessed'
/home/wilson/.gdbinit:3834: Error in sourced command file:
Undefined command: "voltron".  Try "help".
(gdb) python-interactive

import sys
print sys.version
File "", line 1
print sys.version
^
SyntaxError: Missing parentheses in call to 'print'
import blessed
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named 'blessed'
print(sys.version)
3.4.3 (default, Oct 14 2015, 20:31:36)
[GCC 4.8.4]

(gdb) quit
2). So we should change the setup steps a little:
For gdb use python3, so we should install module by pip3, not pip
cd
git clone https://github.com/snare/voltron.git
cd voltron
sudo pip3 install flask          <<< used by next voltron setup
sudo python3 setup.py install
sudo pip3 install blessed        <<< used by voltron/dbgentry.py
3) If voltron view backtrace have connect error, please refer the comment #97 https://github.com/snare/voltron/issues/97.
—
Reply to this email directly or view it on GitHub https://github.com/snare/voltron/issues/64#issuecomment-171546219.

I'm facing the problem on OSX
I have Python 2.7.11 installed via brew but Apple's Python is 2.7.10.
I installed gdb 7.11 also via brew.
When I launch /usr/local/bin/gdb and I print sys.version, I get back 2.7.10.
Now I don't really know how to make this gdb use brew's python despite usr/local/sbin coming first in path:
(gdb) path
Executable and object file path: /usr/local/bin:/usr/local/sbin:/usr/local/opt/ruby/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/sbin:/usr/local/opt/ruby/bin

Can you open a new issue please? Since GDB is linked against the system Python you're probably best just to install Voltron into the system Python.

Well I wasn't sure about opening an issue. The fact that you mentioned again GDB is linked against the system Python made me realize reinstalling it with brew install gdb --with-python rebuilds it and links it against brew's Python.
So it's all good now, and people may find that comment in the current issue.


fix: CSR dev server evaluates user modules

Taken from https://github.com/solidjs/solid-start/pull/595
but this should fix the failing tests i believe?

it only solved like 1-2 issues
Will close this for now and let Oren handle this


automation of multi architecture build of secondary for arm7

What are our intended devices for this, and who do we think will be using them?

PI 2/3 users and IoT devices using arm7 hardware

Pi 3 is Arm64, so we really need to decide if the Pi 2 is worth the bother. Looking at my own collection I'd say no. Lots of 1s and 0s, I think one 2 and one or two 3s, then a couple of 4s.
Also the 2 was only out for a year before the 3 was launched, and later 2s had an A53 (Arm64) SOC.

Pi3 defaults to 32bit mode I think.. Need to try Ubuntu 64 on it, unless you have in the past?

Seems Ubuntu support it so.. My guess is we leave it as and autobuild but if someone wants it we can show them how..
I certainly can do a build once in a while if there is demand.

Raspbian is 32bit, and so Docker on Raspbian uses Armv7 images (even on the later SOCs).
Perhaps the compromise here is that we do manual (but heavily scripted) builds for Armv7 each time we cut a release. That's what I've been doing for the buildimage (which is a dependency).

Here are the scripts I use for the buildimage and dartshowplatform:
build_push.sh - runs on a Raspbian Pi to generate the Armv7 specific images
#!/bin/bash
if [[ -z "${DART_VERSION}" ]]; then
  echo 'Set DART_VERSION'
  exit 1
fi
ARCH="arm"

sudo docker build -t atsigncompany/buildimage:"$DART_VERSION"-"$ARCH" \
--build-arg DART_VERSION="$DART_VERSION" -f at-buildimage/Dockerfile .

sudo docker push atsigncompany/buildimage:"$DART_VERSION"-"$ARCH"

sudo docker build -t atsigncompany/dartshowplatform:"$DART_VERSION"-"$ARCH" \
--build-arg IMAGE_TAG="$DART_VERSION" -f dartshowplatform/Dockerfile .

sudo docker push atsigncompany/dartshowplatform:"$DART_VERSION"-"$ARCH"

dart_pull_push.sh - runs on the Armv8 and X64 machines to generate platform specific tags for those
#!/bin/bash
if [[ -z "${DART_VERSION}" ]]; then
  echo 'Set DART_VERSION'
  exit 1
fi
ARCH="arm64"
sudo docker pull atsigncompany/buildimage:"$DART_VERSION"
sudo docker pull atsigncompany/dartshowplatform:"$DART_VERSION"
sudo docker tag atsigncompany/buildimage:"$DART_VERSION" atsigncompany/buildimage:"$DART_VERSION"-"$ARCH"
sudo docker tag atsigncompany/dartshowplatform:"$DART_VERSION" atsigncompany/dartshowplatform:"$DART_VERSION"-"$ARCH"
sudo docker push atsigncompany/buildimage:"$DART_VERSION"-"$ARCH"
sudo docker push atsigncompany/dartshowplatform:"$DART_VERSION"-"$ARCH"

dart_manifests.sh - pulls together the platform specific tags into :latest and a version specific tag.
#!/bin/bash
if [[ -z "${DART_VERSION}" ]]; then
  echo 'Set DART_VERSION'
  exit 1
fi

sudo docker manifest rm atsigncompany/buildimage:"$DART_VERSION"
sudo docker manifest create atsigncompany/buildimage:"$DART_VERSION" \
  --amend atsigncompany/buildimage:"$DART_VERSION"-arm \
  --amend atsigncompany/buildimage:"$DART_VERSION"-arm64 \
  --amend atsigncompany/buildimage:"$DART_VERSION"-x64

sudo docker manifest push atsigncompany/buildimage:"$DART_VERSION"

sudo docker manifest rm atsigncompany/buildimage:latest
sudo docker manifest create atsigncompany/buildimage:latest \
  --amend atsigncompany/buildimage:"$DART_VERSION"-arm \
  --amend atsigncompany/buildimage:"$DART_VERSION"-arm64 \
  --amend atsigncompany/buildimage:"$DART_VERSION"-x64

sudo docker manifest push atsigncompany/buildimage:latest

sudo docker manifest rm atsigncompany/dartshowplatform:"$DART_VERSION"
sudo docker manifest create atsigncompany/dartshowplatform:"$DART_VERSION" \
  --amend atsigncompany/dartshowplatform:"$DART_VERSION"-arm \
  --amend atsigncompany/dartshowplatform:"$DART_VERSION"-arm64 \
  --amend atsigncompany/dartshowplatform:"$DART_VERSION"-x64

sudo docker manifest push atsigncompany/dartshowplatform:"$DART_VERSION"

sudo docker manifest rm atsigncompany/dartshowplatform:latest
sudo docker manifest create atsigncompany/dartshowplatform:latest \
  --amend atsigncompany/dartshowplatform:"$DART_VERSION"-arm \
  --amend atsigncompany/dartshowplatform:"$DART_VERSION"-arm64 \
  --amend atsigncompany/dartshowplatform:"$DART_VERSION"-x64

sudo docker manifest push atsigncompany/dartshowplatform:latest

NB DART_VERSION is essentially the release, so we should substitute that with the release tag for at_server


Post creation takes too much time

Yesterday when I tried to create a post for the CBI version, the app got stuck in "creating":

It may have failed for some reason. Do you have the post now, or did you have it after a reload?

this issue is purely subjective and without any lead of a bug, its simply a statement and therefore we can close this issue


Added Minimum Length feature

Hi Adam,
Ajax-Bootstrap-Select is excellent - thanks for putting this together.  I've just used it in a project but found there was one extra feature I wanted: to prevent the AJAX call being made until the user had entered a minimum of three characters (as it's pulling from a very large list).  I added this feature in.  I've never used GitHub for this purpose before, only having consumed others work rather than contributed, but thought I'd have a go at making a pull request in case you wanted to include this function.
Essential only a few changes have been made: I added a minLength option for when initialising the select (default to 0 so unless changed it works as normal).  I then added a test of the length of the query against the minLength in the event that's fired.  I added a minLength message to the locale files and if the query length is less than the min length, the function displays the message and returns.
Despite that, a lot of files are showing changes but this seems to mostly be updated build dates.  Here is the summary I wrote when committing the change to the branch:
"Added ability to specify a minimum length for the search string before
the AJAX request will be made.  The minimum length is specified using
the minLength option when initialising the select.  The message
displayed is set in the local files."
Anyway, love to hear any feedback,
Simon

Please include this!

Please only include changes to the source code in the PR. Including the compiled output makes reviewing difficult and can lead to stale/unmergeable PRs (as in this case).

Closing in favor of #97


Filed to run chat_sample on NPU

Hi, we have successfully ran your chat_sample using our 1.8b llm on NPU based on version nightly-20240726, but it doesn't work since we updated openvino and openvino.genai to version nightly-20240812. Is there any change? The CPU of our device is Intel Core Ultra 5.
Can you give us any suggestions?
Thanks a lot!

@liuxt670 thanks for report, please let us know what is the OS / driver version you use.

@dmatveev Thanks for the reply, our OS is Windows 11 and NPU driver version is <IP_ADDRESS>0

@dmatveev Thanks for the reply, our OS is Windows 11 and NPU driver version is <IP_ADDRESS>0

@liuxt670 Could you also elaborate on nightly-20240726 version, where did you take it from?

@TolyaTalamanov
Openvino: https://storage.openvinotoolkit.org/repositories/openvino/packages/nightly/2024.4.0-16130-a3ed68a590b/
Openvino.genai: https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2<IP_ADDRESS>.dev20240726/

@TolyaTalamanov Openvino: https://storage.openvinotoolkit.org/repositories/openvino/packages/nightly/2024.4.0-16130-a3ed68a590b/ Openvino.genai: https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2<IP_ADDRESS>.dev20240726/

Hi wanna ask, how do you resolve the dependencies using these version?
pip install --pre "openvino<=2<IP_ADDRESS>.dev20240726" "openvino-genai<=2<IP_ADDRESS>.dev20240726" "openvino-tokenizers<=2<IP_ADDRESS>.dev20240726" --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly

After I install and view from pip list
the version that I got is
openvino                  2024.3.0
openvino-genai            2<IP_ADDRESS>.dev20240726
openvino-telemetry        2024.1.0
openvino-tokenizers       2<IP_ADDRESS>.dev20240711

@TolyaTalamanov Openvino: https://storage.openvinotoolkit.org/repositories/openvino/packages/nightly/2024.4.0-16130-a3ed68a590b/ Openvino.genai: https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2<IP_ADDRESS>.dev20240726/

Hi wanna ask, how do you resolve the dependencies using these version?
pip install --pre "openvino<=2<IP_ADDRESS>.dev20240726" "openvino-genai<=2<IP_ADDRESS>.dev20240726" "openvino-tokenizers<=2<IP_ADDRESS>.dev20240726" --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly

After I install and view from pip list, it seems like openvino-tokenizers and openvino version are dependant of each other. so I wanna ask what is your version of openvino-tokenizers and openvino? the version that I got is
openvino                  2024.3.0
openvino-genai            2<IP_ADDRESS>.dev20240726
openvino-telemetry        2024.1.0
openvino-tokenizers       2<IP_ADDRESS>.dev20240711

@szeyu Could you check https://github.com/openvinotoolkit/openvino.genai/issues/834 and https://docs.openvino.ai/2024/learn-openvino/llm_inference_guide/genai-guide-npu.html, please?
Perhaps you need to start with clear python environment (python -m venv ov-env)

@liuxt670 Can you double check with the latest available versions, please?
python -m venv ov-env
ov-env\Scripts\activate
pip install --pre openvino openvino-tokenizers openvino-genai --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly

pip list:
openvino 2024.5.0.dev20240909
openvino-genai 2<IP_ADDRESS>.dev20240909
openvino-telemetry 2024.1.0
openvino-tokenizers 2<IP_ADDRESS>.dev20240909

The following works fine with Tiny-Llama converted like this: https://docs.openvino.ai/2024/learn-openvino/llm_inference_guide/genai-guide-npu.html
cd openvino.genai\samples\python\chat_sample
python chat_sample.py <path-to-model>

@liuxt670 Can you double check with the latest available versions, please?
python -m venv ov-env
ov-env\Scripts\activate
pip install --pre openvino openvino-tokenizers openvino-genai --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly

pip list:
openvino 2024.5.0.dev20240909
openvino-genai 2<IP_ADDRESS>.dev20240909
openvino-telemetry 2024.1.0
openvino-tokenizers 2<IP_ADDRESS>.dev20240909

The following works fine with Tiny-Llama converted like this: https://docs.openvino.ai/2024/learn-openvino/llm_inference_guide/genai-guide-npu.html
cd openvino.genai\samples\python\chat_sample
python chat_sample.py <path-to-model>

Hi I have followed the exact step that you have provided until before the chat_sample.py. I think that yours is working maybe you forgot to change the
device="NPU" in the chat_sample.py?
since this thread is about NPU. Of course I have succeeded in running in CPU as well but failed on NPU.
This is the error output that I have gotten when running in NPU
Traceback (most recent call last):
  File "C:\Users\hpintel\Documents\openvino.genai\samples\python\chat_sample\chat_sample.py", line 39, in <module>
    main()
  File "C:\Users\hpintel\Documents\openvino.genai\samples\python\chat_sample\chat_sample.py", line 33, in main
    pipe.generate(prompt, config, streamer)
RuntimeError: Exception from src\inference\src\cpp\infer_request.cpp:223:
Check 'byte_size == 0 || data != nullptr' failed at src\inference\src\dev\make_tensor.cpp:208:
Failed to allocate memory

My NPU driver version is <IP_ADDRESS>88
My pip list:

openvino                  2024.5.0.dev20240910
openvino-genai            2<IP_ADDRESS>.dev20240910
openvino-telemetry        2024.1.0
openvino-tokenizers       2<IP_ADDRESS>.dev20240910

Also from what I have found from the NPU_Win_Release_Notes_v2688.pdf, it is stated that the supported version of OpenVINO framework is 2024.3

I think this might be related to this issue as well

Alright! Now then it works. Seems like the documentation of Intel driver need some changes, else it will be confusing. The documentation need to state the driver version that work for Openvino running on NPU.
Anyways thanks for your help. It works on my side with

NPU Driver : Intel® NPU Driver - Windows <IP_ADDRESS>14*

Pip lists :
openvino                  2024.5.0.dev20240911
openvino-genai            2<IP_ADDRESS>.dev20240911
openvino-telemetry        2024.1.0
openvino-tokenizers       2<IP_ADDRESS>.dev20240911

LLM Model :
optimum-cli export openvino -m microsoft/Phi-3-mini-4k-instruct --weight-format int4 --sym --group-size 128 --ratio 1.0 Phi-3-mini-4k-instruct-sym-int4-ov

Python code :

import openvino_genai as ov_genai
model_path = "Phi-3-mini-4k-instruct-sym-int4-ov"
pipe = ov_genai.LLMPipeline(model_path, "NPU")
print("testing")
print(pipe.generate("The Sun is yellow because", max_new_tokens=100))

I think there is no issue for now. @TolyaTalamanov you can close the issue for now. Thanks for you help.