Support MPO-(MPS with dangling site indices) contraction

I made a fresh start after #44 was merged into the main branch.
For QTT, we may want to contract an MPO (with two site indices at each vertex) and an MPS with dangling site indices. I found several issues, but let me sout them out one by one. The first issue is the following.
A minimum example for naive algoritm works:
nbit = 3
sites = siteinds("Qubit", nbit)

H = randomMPO(sites) + randomMPO(sites)
H = replaceprime(H, 1=>2, 0=>1)

psi = randomMPO(sites) + randomMPO(sites)

# MPO-MPO
Hpsi_ref = contract(H, psi; alg="naive")

# MPO-MPS
Hpsi_ref2 = contract(H, MPS([A for A in psi]); alg="naive")

@show H
@show psi
@show Hpsi_ref
@show Hpsi_ref2
@assert MPS([A for A in Hpsi_ref]) ≈ Hpsi_ref2

H = MPO
[1] ((dim=2|id=58|"Qubit,Site,n=1")'', (dim=2|id=58|"Qubit,Site,n=1")', (dim=2|id=68|"Link,l=1")')
[2] ((dim=2|id=656|"Qubit,Site,n=2")'', (dim=2|id=656|"Qubit,Site,n=2")', (dim=2|id=121|"Link,l=2")', (dim=2|id=68|"Link,l=1")')
[3] ((dim=2|id=270|"Qubit,Site,n=3")'', (dim=2|id=270|"Qubit,Site,n=3")', (dim=2|id=121|"Link,l=2")')

psi = MPO
[1] ((dim=2|id=58|"Qubit,Site,n=1")', (dim=2|id=58|"Qubit,Site,n=1"), (dim=2|id=787|"Link,l=1"))
[2] ((dim=2|id=656|"Qubit,Site,n=2")', (dim=2|id=656|"Qubit,Site,n=2"), (dim=2|id=46|"Link,l=2"), (dim=2|id=787|"Link,l=1"))
[3] ((dim=2|id=270|"Qubit,Site,n=3")', (dim=2|id=270|"Qubit,Site,n=3"), (dim=2|id=46|"Link,l=2"))

Hpsi_ref = MPO
[1] ((dim=2|id=58|"Qubit,Site,n=1")'', (dim=2|id=58|"Qubit,Site,n=1"), (dim=4|id=716|"CMB,Link"))
[2] ((dim=2|id=656|"Qubit,Site,n=2")'', (dim=2|id=656|"Qubit,Site,n=2"), (dim=4|id=206|"CMB,Link"), (dim=4|id=716|"CMB,Link"))
[3] ((dim=2|id=270|"Qubit,Site,n=3")'', (dim=2|id=270|"Qubit,Site,n=3"), (dim=4|id=206|"CMB,Link"))

Hpsi_ref2 = MPS
[1] ((dim=2|id=58|"Qubit,Site,n=1")'', (dim=2|id=58|"Qubit,Site,n=1"), (dim=4|id=596|"CMB,Link"))
[2] ((dim=2|id=656|"Qubit,Site,n=2")'', (dim=2|id=656|"Qubit,Site,n=2"), (dim=4|id=816|"CMB,Link"), (dim=4|id=596|"CMB,Link"))
[3] ((dim=2|id=270|"Qubit,Site,n=3")'', (dim=2|id=270|"Qubit,Site,n=3"), (dim=4|id=816|"CMB,Link"))

fitting algorithm does not work:
# MPO-MPS
contract(H, MPS([A for A in psi]); alg="fit")

BoundsError: attempt to access 1-element Vector{Index{Int64}} at index [2]

Stacktrace:
  [1] getindex
    @ ./array.jl:924 [inlined]
  [2] replaceinds(is::Tuple{Index{Int64}, Index{Int64}, Index{Int64}}, inds1::Vector{Index{Int64}}, inds2::Vector{Index{Int64}})
    @ ITensors /sharehome/shinaoka/.julia/packages/ITensors/fpBnt/src/indexset.jl:570
  [3] replaceinds(::NDTensors.DenseTensor{Float64, 3, Tuple{Index{Int64}, Index{Int64}, Index{Int64}}, NDTensors.Dense{Float64, Vector{Float64}}}, ::Vector{Index{Int64}}, ::Vararg{Vector{Index{Int64}}}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ ITensors /sharehome/shinaoka/.julia/packages/ITensors/fpBnt/src/itensor.jl:1480
  [4] replaceinds(::NDTensors.DenseTensor{Float64, 3, Tuple{Index{Int64}, Index{Int64}, Index{Int64}}, NDTensors.Dense{Float64, Vector{Float64}}}, ::Vector{Index{Int64}}, ::Vector{Index{Int64}})
    @ ITensors /sharehome/shinaoka/.julia/packages/ITensors/fpBnt/src/itensor.jl:1479
  [5] replaceinds!(::ITensor, ::Vector{Index{Int64}}, ::Vararg{Vector{Index{Int64}}}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ ITensors /sharehome/shinaoka/.julia/packages/ITensors/fpBnt/src/itensor.jl:1484
  [6] replaceinds!(::ITensor, ::Vector{Index{Int64}}, ::Vector{Index{Int64}})
    @ ITensors /sharehome/shinaoka/.julia/packages/ITensors/fpBnt/src/itensor.jl:1483
  [7] contract(::ITensors.Algorithm{:fit}, A::MPO, psi0::MPS; init_state::MPS, nsweeps::Int64, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ ITensorNetworks /sharehome/shinaoka/git/ITensorNetworks.jl/src/treetensornetworks/solvers/contract_operator_state.jl:39
  [8] contract
    @ /sharehome/shinaoka/git/ITensorNetworks.jl/src/treetensornetworks/solvers/contract_operator_state.jl:13 [inlined]
  [9] contract(A::MPO, ψ::MPS; alg::String, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ ITensors /sharehome/shinaoka/.julia/packages/ITensors/fpBnt/src/mps/mpo.jl:567
 [10] top-level scope
    @ In[14]:2

The replacement of site indices does not work with an MPS with dangling site indices. Shall we just remove this part?
https://github.com/mtfishman/ITensorNetworks.jl/blob/main/src/treetensornetworks/solvers/contract_operator_state.jl#L38-L42
If we move those lines, it will be user's responsibility to provide an appropriate init_state with the same site indices as H * psi0. The following default value assumes that init_state lives in the same space as psi0, which is the case if H is a time-evoluation operator. For a more general MPO, this default value is not appropriate. What do you think about removing these lines for the replacement and adding a check on the site indces on init_state?
https://github.com/mtfishman/ITensorNetworks.jl/blob/main/src/treetensornetworks/solvers/contract_operator_state.jl#L17

Yes, I think we should remove any automated index replacement and leave it up to the user to provide a proper init_state.
For example, it doesn't make sense to try to replace the indices if the matrix A in Ax = b is rectangular, in which case x and b fundamentally have different spaces. Additionally, if the MPO and MPS have multiple sites per index, that index replacement isn't even well defined.

Finally, in the signature of the function I don't think we should specialize A::IsTreeOperator and psi0::IsTreeState.
Instead I think it should more generally be defined as:
function ITensors.contract(
  ::ITensors.Algorithm"fit",
  A::IsTree,
  b,
  x0;
  kwargs...,
)
  [...]
end

where:
const IsTree = Union{MPS,MPO,TTNS,TTNO}

to make it as general as possible. Going forward, we might make this into a trait, and also just get ride of TTNS and TTNO in favor of a single type TTN, since I think there is generally not much reason to have different types, at least based on how they are designed right now.

Agreed with your comments & suggestions. Should we set a default value to init_guess by automatically analyzing the site indices (and creating a random TNN object with some bond dimensions)? This may be possible but sounds a bit complicated.
Candidate 1 is backward compatible. Candidate 2 is simple and general.
Candidate 1:
function ITensors.contract(
  ::ITensors.Algorithm"fit",
  A::IsTree,
  B::IsTree;
  init_guess=B, # Error is thrown if B does not have the same idices as `A * B`.
  kwargs...,
)
  [...]
end

Candidate 2:
function ITensors.contract(
  ::ITensors.Algorithm"fit",
  A::IsTree,
  B::IsTree;
  init_guess, # The user must specify explicitly
  kwargs...,
)
  [...]
end

@shinaoka I don't think using init_guess=B makes sense since it won't have the same external indices as A * B.
I don't think there is any particular challenge to analyzing A and B and determining the proper external indices for the resulting network (related to #40). However, it is tricky creating random initial guesses, particularly for determining the QNs of the link indices if there are symmetric tensors. For DMRG, we use randomMPS for that, which internally uses a random circuit to create a random state of some finite bond dimension.
I think like for DMRG we should have the users specify an initial guess unless we come up with something good that generally works. I know this algorithm has a tendency to get stuck in local minima so the initial guess is particular important. Something to consider would be to use a cheaper algorithm for contracting the TTNs approximately, such as the zip-up algorithm. But we would have to test that out to decide if it is a good default in general.

Also, we've decided that for now we won't explicitly support accepting MPS/MPO types in the functions in ITensorNetworks.jl (I'll change this throughout the package), so the signature should be:
function ITensors.contract(
  ::ITensors.Algorithm"fit",
  A::AbstractTTN,
  B::AbstractTTN;
  init_guess,
  kwargs...,
)
  # [...]
end

Users can convert their MPS/MPO to TTN if they want to use the new functions available in this package.

Thank you for letting me know about the decision! I will test the code once you made changes throughout the package.

@shinaoka in #50 I changed this contract function to have the signature:
function contract(
  ::Algorithm"fit",
  tn1::AbstractTTN,
  tn2::AbstractTTN;
  init,
  kwargs...,
)
  # [...]
end

which helps us move towards not having a distinction between states and operators.
Unfortunately that function now seems to be broken, which I think happened when we originally generalized the solver functions from MPO/MPS to TTNs in #44.
If you have time, it would be very helpful if you could investigate why it isn't working!

Thank you!
It seems to be working on my side. Do you have any example code?
nbit = 3
sites = siteinds("Qubit", nbit)

M1 = replaceprime(randomMPO(sites) + randomMPO(sites), 1=>2, 0=>1)
M2 = randomMPO(sites) + randomMPO(sites)
M12_ref = contract(M1, M2; alg="naive")
t12_ref = TreeTensorNetwork([M12_ref[v] for v in eachindex(M12_ref)])

t1 = TreeTensorNetwork([M1[v] for v in eachindex(M1)])
t2 = TreeTensorNetwork([M2[v] for v in eachindex(M2)])
t12 = contract(t1, t2; alg="fit", init=t12_ref)
#t12 = contract(t1, t2; alg="fit")

function toarr(t)
    arr = reduce(*, [t[v] for v in vertices(t)])
    return Array(arr, vcat(sites, sites''))
end

@show t12 ≈ t12_ref
@show toarr(t12) ≈ toarr(t12_ref)

Turns out there were some issues in the tests due to a bug in inner and changes in the default initial state for contract (#52), but the function itself should be fine I think.

I see. So, shall we add the above code to the test?


Bug: Unable to connect to the database

Describe the bug
Please need help. I can't connect to the database not from model or manually connect. The database is exist. Try to change .env or config still doesn't work.
error:
SYSTEMPATH\Database\BaseConnection.php at line 425 (something about connID)
SYSTEMPATH\Database\BaseConnection.php : 646   —  CodeIgniter\Database\BaseConnection->initialize ()
.env:
database.default.hostname = localhost
database.default.database = baru
database.default.username = root
database.default.password =
database.default.DBDriver = MySQLi
CodeIgniter 4 version
4.0.4, Mysqli driver
i have already follow the documentation and change hostname to <IP_ADDRESS> but still can't connect to he database.

PHP version 7.4.8

Most likely not a bug. Try changing localhost to <IP_ADDRESS>.
In the future please ask for help on the forums, we use GitHub for tracking issues.

change localhost to <IP_ADDRESS> doesn't help

You have not provided a password. Or you have not registered yet the root user to MySQL server.

when i run in xampp project/public/ it can connect
and somehow when use spark serve i can't connect to the database

Try setting your logs threshold to 4 so that errors will be logged too. Then run the code again. Then tell us the ERROR line you get in your logs.


global: make use of FunctionalInterface

Now that we're using Java 8, it seems reasonable to annotate.

@grimreaper Would it make sense in general to also annotate interfaces that just duplicate the ones in java.util.function with @Deprecated?

IMHO yes, though that should be done separately.

Coverage remained the same at 87.824% when pulling 598a42b5904f9519bf313a72f0b5d80adc48ec52 on grimreaper:eax/finter/now into ca6851e3ecff84f5a5148e26caed6e113f444198 on apache:master.


Macro Deck glow icon pack - (Animated) glow icon pack (WIP)

** Name of the icon pack**
** Version **
The latest version of the icon pack
** Author of the icon pack **
The author name that will appear in the package manager
** Description of the icon pack **
What does the icon pack contain? A short description 20-30 words
** GitHub repository ** (Optional)
A link to the GitHub repository of the icon pack

@OvaKrdozo Hi, I really like your icon pack but since the new Extension Store it's not available anymore because many things changed. Are you interested in publish it again? If yes, please contact me on Discord. I'll guide you trough the steps. SuchByte#5461


DB.NoSync, other performance considerations

We're using bolt on some spinning disks and we really need some substantial performance boosts compared to what we are currently getting. Some things are write intensive, and others just require us to be doing a bunch of serial transactions per second. = atomic?
What are the risks of using DB.NoSync? Obviously you aren't guaranteed that the data has hit the platter, but as long as we're taking that into consideration are there other footguns? Can we assume that DB.NoSync will still provide fully atomic transactions?
For the write-heavy loads, will it be faster to sort our entries before we insert them into the database? Is that something that bolt would consider (or already does) implementing on the backend?
Will it help to move some of the bulkier objects (>64kb) into their own files? At that point we have to add a lot of overhead, because we need to keep things atomic & consistent.
Are there other things you can think of that will help us squeeze more performance out of the database? We're at the point where for some of the code we've begun writing custom databases to handle things.

What are the risks of using DB.NoSync? Obviously you aren't guaranteed that the data has hit the platter, but as long as we're taking that into consideration are there other footguns? Can we assume that DB.NoSync will still provide fully atomic transactions?

NoSync is pretty dangerous. The order of writes can be rearranged by the OS/disk to optimize the write throughput. However, that means that the meta page could be written before the data pages and in the event of the computer crashing you would corrupt your database. It's really just meant for bulk loading where you can load all your data, ensure your computer hasn't crashed, and then re-enable sync.

For the write-heavy loads, will it be faster to sort our entries before we insert them into the database? Is that something that bolt would consider (or already does) implementing on the backend?

Sorting can help in some instances but usually not unless you're doing appends to your bucket.

Will it help to move some of the bulkier objects (>64kb) into their own files? At that point we have to add a lot of overhead, because we need to keep things atomic & consistent.

If you can shard your data into multiple bolt instances then I would expect that to improve your write performance. The sync() blocks an individual Commit() but you would effectively batch those if you were syncing across multiple databases. That's not always an option though.

We're using bolt on some spinning disks and we really need some substantial performance boosts compared to what we are currently getting.

Honestly, spinning disks aren't going to have very good performance with Bolt. Bolt uses a lot of random writes which works well on SSDs but is pretty rough for spinning disks. If write performance is a big issue then you may need to look at a write-optimized data store like RocksDB.

Are there other things you can think of that will help us squeeze more performance out of the database?

If you can batch your transactions together then I would expect that to improve performance on a spinning disk.


Use strict: not found [v2.0.0]

Hello,
since the 2.0.0 update we are facing an issue in our CI

/.../node_modules/.bin/sarif-codeclimate: 1: use strict: not found
/.../node_modules/.bin/sarif-codeclimate: 2: Syntax error: word unexpected (expecting ")")

or localy

//.../node_modules/.bin/sarif-codeclimate: line 1: use strict: command not found
//.../node_modules/.bin/sarif-codeclimate: line 2: syntax error near unexpected token 'exports,'
//.../node_modules/.bin/sarif-codeclimate: line 2: Object.defineProperty(exports, "__esModule", { value: true });'

The command that we run:
sarif-codeclimate --input relativePathLocation.json --output gl-code-quality-report.json
Is there anything we need to do in our package.json file?

the destination of the files have been changed as the package was re-written in typescript.
You could check the Readme for usage or set the version to '1.0.2' in your package.json to keep the same behavior
Hope it will help !

Fixed in version 2.0.1 @simon-nowak !

Thank you !!
I confirm the last version is now working. I will do some test and update the other open issue accordngly.


Implement functionality that allows users to create new variant and evidence pages

We've added an Add Evidence page that adds evidence as well as new gene and variant records.


refs #32255 changed has_perm signature to forward **kwargs

It refers to: https://code.djangoproject.com/ticket/32255
I changed the has_perm and has_perms signature. All tests runs fine.
It is somehow a breaking change in the meaning that all auth backends should change and also accept a **kwargs from now on.
I'm open to suggestions to avoid this.

I'm open to suggestions to avoid this.

You'd need to add a backwards compatibility shim that calls the auth backends without kwargs till the drepecation is over.

Something like this right?  https://github.com/django/django/pull/13785/commits/eb7a6b55a921dd24ca82bd55c3496bb70e1b392e

Jupp, though using TypeError is kinda dangerous because the TypeError could be raised from inside the function for various reasons; maybe it might be worth to inspect the function signature.
That said I am not convinced that adding this "feature" is a good idea at all; I'll comment on the ticket to that extend.

Closing due to lack of consensus, see ticket-32255.


Explain how to show a plugin's description

Adds an example of how to show the description of a plugin

related #352.
@jbucaran should we go for working examples (e.g. fisher ls-remote fzf) or man page-like (e.g. fisher ls-remote [package].
Having copy-paste ready has some drawbacks, but non-working are annoying. What do you think?

Thanks @asymmetric!
@edouard-lopez Working examples are nicer, but no hard rules. I am unsure about documenting this bit, after all @asymmetric's other PR already illustrates this, kinda.

@jbucaran yep add the same hesitation. Maybe listing packages on a one-per-line basis with their description will be better. Many packages manager do that (apt, npm, etc.) so you got enough information. Cause right now some package have very low information smell. For instance:
omf/gi                          omf/theme-bira                  omf/theme-taktoa

That's what ls-remote --format does. So, you are suggesting to present more information by default. That would make sense. I actually prefer just using fish pager, that fisherman already fills in with the relevant info, e.g


Ingest pdf document from blob storage, and change default values of "Retrieve this many search results"

Please provide us with the following information:

This issue is for a: (mark with an x)
- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

process and upload pdf documents from blob storage, 2. change the default value of "Retrieve this many search results" from 3 to 5

OS and Version?

Windows 10

azd version?

run azd version and copy paste here.

Versions

1.9.3

Mention any other details that might be useful
Hi Azure team,
Is there a native way to ingest the pdf documents from blob storage? Currently we can only copy those docs to local data folder, then run scripts/prepdoc. Also, can we also change the default values too (i.e. "Retrieve this many search results")? Thank you!

Thanks! We'll be in touch soon.

For changing default values, see this section in the customization guide:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/customization.md#making-settings-overrides-permanent
For ingesting PDF documents, you can use integrated vectorization:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/data_ingestion.md#overview-of-integrated-vectorization
That feature does not yet track page numbers of PDFs, however, so you'd need to write a custom skill for that if that is needed.

Hi @pamelafox
Has any skills been added that are able to track page numbers on PDFs (in integrated vectorisation), or are they in the pipeline :)
Thanks in advance for your work.


Add support for logging PROTT #58

I added supprot for PROTT as requested in #58
https://github.com/ABAP-Logger/ABAP-Logger/issues/58

Hi, thanks for the PR.
Could you add unit test as well? (see can_log_bapiret1 and the like for an example).

Just added the unit test to the same change.


Customizeable columns of participants list PDF

Select desired columns of participants overview and create participants list as PDF with selected colums only.

This issue seems to be stale.
Please reopen if it is still relevant.


Putting the Token into the console

The console asking for your token instead of you having to edit config.json

Maybe you could check if the config has the token inside, if not then ask for it

Yeah I was thinking something along those lines but I am not familiar with promises and such in nodejs.

I'd think something like this:
var conf = JSON.parse(require('fs').readFileSync('config.json', 'utf8'))

if (conf.token=="")
    // ask for token
else
    // run bot from config


bad code practice

Preparing review...

Preparing review...

Preparing review...

PR Analysis

🎯 Main theme: Bad code practice
📝 PR summary: The PR introduces bad code practices, such as unused functions and variables, and lack of documentation.
📌 Type of PR: Bug fix
🧪 Relevant tests added: No
⏱ Estimated effort to review [1-5]: 3, because the PR is small and the code is simple.
🔒 Security concerns: No security concerns found

PR Feedback

General PR suggestions: The PR should include more documentation, such as docstrings and comments, to explain the purpose and usage of the code.
The PR should also include more tests to ensure that the code is working correctly and to prevent future regressions.


Use just string and symbol keys in objects

Currently the objects can have any keys (objects, arrays, null, undefined, etc.). However, this could lead to multiple issues when executing ES5 code, for example:
// Example 1:
var obj = { true: 10 };
console.log(obj[true]); // undefined in this engine

// Example 2:
var key = { toString: function () { console.log('stringified'); return 'test'; } };
var obj = { test: 10 };
console.log(obj[key]); // key.toString won't get called, because no string conversion is done

From my experience with working with this engine, this isn't a problem 99% of the times, but doesn't provide too much of an advantage either. Ponder whether or not its worth it to sacrifice ES5 compliance for such little tradeoff

The issue is fixed in the ES6 support branch


[Core] Restructure repo to follow industry golang standards

Description
This refactors the codebase to follow the standards outlined in https://github.com/golang-standards/project-layout.
Additionally some changes on the Makefile were applied to make it mode readable and maintainable.
Issue
Fixes #379
Type of change
Please mark the relevant option(s):

[ ] New feature, functionality or library
[ ] Bug fix
[x] Code health or cleanup
[x] Major breaking change
[ ] Documentation
[ ] Other

List of changes

Refactor codebase to follow golang project layout standards
QOL changes to make the Makefile more readable and maintainable by splitting it into multiple domain oriented files

The new sub makefiles are locates at ./scripts

Updated documentation pointing to internal packages
Updated golang codebase to take in account the new package location
Updated the protobuf files and its corresponding scripts to build correctly with the new package paths

Testing

[x] make develop_test
[x] LocalNet w/ all of the steps outlined in the README

Required Checklist

[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have tested my changes using the available tooling
[x] I have updated the corresponding CHANGELOG

If Applicable Checklist

[x] I have updated the corresponding README(s); local and/or global
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added, or updated, mermaid.js diagrams in the corresponding README(s)
[x] I have added, or updated, documentation and mermaid.js diagrams in shared/docs/* if I updated shared/*README(s)

Summarised some comments here for whoever picks this up in the future.


Fix transcoding crunch to DXT in IE11

TypedArray.prototype.slice is not supported in IE11: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray/slice
For #2860.

@lilleyse Updated.

Test passes now - https://github.com/AnalyticalGraphicsInc/cesium/issues/2860


voucher-client: send User-Agent

Modify the voucher client to send a customizable User-Agent header. This is easily spoofed, but helps a voucher-server to understand the distribution of its clients.
When voucher client is included as a library, the string voucher-client/2 is used. There's a method for clients to customize (e.g. my-awesome-builder/1.0).
When voucher_client is used as a binary, the tagged version (e.g. voucher-client/2.6.2) is used.
Related

Ref https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent

Is this PR already merged in https://github.com/grafeas/voucher/pull/54?

Is this PR already merged in #54?

Yep, #54 targets this as a base branch so we'd have to merge this before it.
That's because #54 is a refactor to client construction, so we wanted to include all the "current" usage.
I think we can consider this feature independently from #54 , as all it does is refactor the method of customization.


Added maybeDeferred and test

Thanks for working on this!
The test is half-way there, but should probably go the whole way to really make sure we're not losing updates to other connections. So do that and resubmit.

Doesn't look like forced a build: http://build.clusterhq.com/boxes-flocker?branch=catch-all-exception-FLOC-3221

Address the above.
Force a build.
Merge when all builders in Buildbot are green. (it will add info here in the PR).


Add streaming example for PS5000A

Adaption of the code example from the ps2000a

This seems really useful. When will this be pulled into the master?

We don't accept pull requests onto our repositories I'm afraid but we have added streaming examples and functions for this device series to the repository.


Adding django core to requirements.txt

Adding to django core to requirements.txt to fix error message "ImportError: No module named 'django.core.context_processors"  when running python manage.py runserver

this looks relevant: https://stackoverflow.com/questions/32828536/django-no-module-named-context-processors-error-after-reboot

We need to fix the requirements.txt to force install django 1.8 instead of django 1.11


Previewify needs prefix to work correctly

I wasn't aware that previewify attributes always need a prefix. So I had to change the implementation to comply with that requirement.

Are title and content the only two used values? If so, perhaps we could check for the presence of the prefix, and add it if it's not there? In the previewify() method. That way the previous, simple syntax would be supported.

Previewify has completely custom data attributes, I just made the implementation work like the flippa one. We could also automatically prefix every value with previewify:. Whats your opinion @Larsklopstra?

I'd check each key if it str_starts_with() previewify:, and if not, automatically prefix it.

Sounds like better approach. Should I work on it or would you like to do it.

If you want you can send a PR simply prefixing the keys in that method 👍🏻


Add support for getting a single personal access token

This API route is missing from the package: https://docs.gitlab.com/ee/api/personal_access_tokens.html#get-single-personal-access-token
Especially interested in the /personal_access_tokens/self route: https://docs.gitlab.com/ee/api/personal_access_tokens.html#using-a-request-header

PR's are always welcome so if you want to take a stab at it, go right ahead! Please do have a look at the rest of the code in the package and make sure you follow the existing style as much as possible...

Fixed by #1686


Fine tuning?

Great paper and work! Thank you for publishing this.
In your github repo I don't see training or fine tuning code or details. Are those available?
Thanks!

Yes we didn't include training/finetuning codes in this repo.
However, as the RemoteCLIP checkpoint is provided in OpenCLIP format, it's possible to finetune RemoteCLIP with the OpenCLIP codebase.
Please see here for more details.
Thank you : )

Thank you for your excellent work, I would like to ask a follow up question, are you using tif format files or jpg format files for training? (Since this may affect fine tuning).
As a lot of remote sensing image data are tif files, if I want to fine-tune based on tif, is it unable to do so with openclip's format?
Hoping you can help clear up my confusion, thanks for your time!


[Upstream] NonMaxSuppression

This upstreams PRs #832 and #833.

Can one of the admins verify this patch?


Need manual merge please. Add CoinGecko Id

I'm submitting a ...
[ ] bug report
[ x] feature request
[ ] question about the decisions made in the repository
[ ] question about how to use this project

Hi,
I need manuel merge to add CoinGecko Id for the token BlockFileSystem (BFS)
Original PR : https://github.com/solana-labs/token-list/pull/20690
Current PR : https://github.com/solana-labs/token-list/pull/27271
Thanks for your help

Hello @keone
Can you please review and merge PR https://github.com/solana-labs/token-list/pull/27271 ?
Thx a lot.


Debug builds should work on wasm32-unknown-unknown now

https://github.com/rust-lang-nursery/rust-wasm/issues/1
Hopefully this means cargo-web doesn't need to force release mode anymore.

@Kwarrtz It's in-progress. I need to switch the js! macro to use custom sections first.

I haven't looked too closely yet, but it seems like procedural macros were fully stabilized in Rust 1.30 so I might be able to fix this soon.

don't really know that much about custom sections in wasm, can I still somehow help with the js! macro?

@ForsakenHarmony I stared to work on this already, however it's currently on the backburner. I haven't yet started on converting the js! macro itself though. (This needs three pieces - support in cargo-web, which I've already done, support in __js_raw_asm! which I've partially done, and support in js! which I haven't started yet.)
That is, we need a procedural macro which will take this:
js!(
    console.log( 123, @{variable} );
);

and emit something like this (not exactly, as the js! macro does more stuff, but this is the hard part so it will be easy to convert to what we need in the end):
__js_raw_asm!( r#"console.log( 123, $0 );"#, variable );

Basically, the procedural macro has to take its input (the JS code), stringify it replacing every @{...} with placeholders ($0, $1, $2...), and extract the interpolated expressions itself (in this case the sole variable, but this can be any Rust expression). The hardest part of this is probably parsing arbitrary Rust expressions inside of @{...}. (syn should be able to do it, but I'm not sure if it can easily parse an expression explicitly delimited with a } at the end)
If you have some free time and feel like helping out then you could try your hand at writing such a basic procedural macro (it doesn't have to be actually functional; I can take it from there).

(syn should be able to do it, but I'm not sure if it can easily parse an expression explicitly delimited with a } at the end)

Rather than chopping off the { and then trying to parse up to the }, why not instead just parse the entire {...} expression with syn? Since {} is already an expression (specifically a block expression), it should work.
And then you could do some validation on the AST to ensure that the block expression only contains a single expression inside of it (and no statements).

@Pauan Yep! Good point. As long as it stringifies the code just as the current js! macro it should be fine.

ok I'll try to work on this


Fix resetting to initial state

Fix #65.
Bug can be reproduced with description given in #65, using the included sample project.

Thank you! :)


Add 'Getting Started' section to Readme

It uses the <details> tag for collapsible regions. You can take a peek at the branch.

Thanks for this!


Is the plug-in not available now?

I installed the plug-in in Eclipse, but there was an error. Isn't it currently supported?

This project seems to be dead, but there is another plugin which works just fine:

Unofficial GitHub Copilot integration with Eclipse


[BUG] Flye does not allow more than 128 threads.

Describe the bug
Flye does not allow more than 128 threads.
To Reproduce
run hybracter with -t 190 on 192 core machine
Expected behavior
scale down Flye rule to use 128 cores max only
Additional context
Add any other context about the problem here.
See https://github.com/mikolmogorov/Flye/pull/730 for a pull request to allow Flye to use more cores.
Other software in the pipeline does not seem to have a problem with >128 cores.

@vdejager
Thanks for alerting me to this - I simply wouldn't recommend anyone to use more than 128 cores (I will just wait for the next Flye release if they fix this and leave Hybracter as is).
I can't imagine that it is optimal to run Flye with 128 cores in any case if you have multiple samples anyway, I'd assume running e.g. 8 samples with 16 threads will be more resource efficient than 1x128 threads sequentially.
George


[C++][Parquet] Parquet Fuzzing Enhancement

Describe the enhancement requested
Currently, Parquet uses LLVM libFuzzer and corpus to run fuzzing. Corpus will generate a multi-column file, and copy all valid test-data to the fuzz directory.
However, libFuzzer mutate value by byte, though libFuzz uses coverage-based fuzzing, it's hard to generate valid input, like different read and write options, different kinds of encoding. Instead, it's likely to generate some bad-data (lol...)
Besides, there is no Oracle function to verify whether the read-result is ok. Currently, FuzzReader only checks whether the read would throw exception or return an bad status. If read-data is not equal to write-data, it should report to user.
I intent to use config parsing like https://source.wiredtiger.com/develop/tool-libfuzzer.html , and cover more encoding and test cases.
Component(s)
C++, Parquet

You may also want to look at the integration tests we have for the Arrow IPC format for inspiration.  This is in dev/archery/archery/integration.  There are tools there to generate random valid IPC files which we then verify the various implementations can all read.

Great! datagen.py and cross generation helps a lot! Previously I found that Rust's arrow has a cross-verification test, which write from pyarrow and read from it self's implemention. Let me do it step by step: https://github.com/jorgecarleitao/parquet2/pull/104/files#diff-b1ec89bd2022a3ed591256cf38963132122a430d36841741185a65ac4177b249


Kubectl Shell loading failed

when clicked Kubectl Shell , the kubectl shell popup window show message : LOADING TERMINAL... , but disappeared after a while, open chrome develop console to check network request:

@wowtrxqn
I was not able to reproduce the issue.  Please see attachment.  Note:  My K8s environment is using NodePort 30778 and not Load Balancing 9001.
Can you expand the two "WebSocket connection to..." errors at the end, for review?

Thanks!

@wowtrxqn
I was not able to reproduce the issue. Please see attachment. Note: My K8s environment is using NodePort 30778 and not Load Balancing 9001.
Can you expand the two "WebSocket connection to..." errors at the end, for review?

Thanks!

i solved , thanks

Only because I had this happen and sorted it too:

I used Helm to launch Portainer and my setup involves an NGinx Ingress controller.
NGinx won't handle websockets (wss://) so when i clicked f12 in my browser, I saw all the vomits "webSocket connection to 'wss://portainer.freshbrewed.science/api/websocket/kubernetes-shell?token=eyJasdfasdfasdfasdfasdfasdfasdfasdfasdasdfD00A&endpointId=1' failed: "

There is a way though, if the kubectl shell is a break-glass need.   I exposed the NodePort from the K8s service externally.  This did mean accepting two things: self signed certs and OAuth would not work (because of my OAuth redirect) - but i was then able to login as a break-glass admin user and fire a shell.
If your k8s can expose a service directly (eg you use a cloud provider), you might not have this issue (kubectl expose...).

I have the same situation in TrueNAS scale, with Portainer running as an app inside k3s... not sure also how to fix this, the shell popup shows and closes immediately with the same error.
I actually have a valid SSH cert through cert-manager, just on port 9443, but still get this error.
I suspect traefik for this... any idea?


Multi camera navigation training

Hello,
Are you aware of any project using Carla to train navigation with two or more cameras?
I'm new to autonomous driving so would love to see some examples.

This question seems better suited for our chat https://discord.gg/8kqACuC
Let's keep GitHub for technical issues and questions specific about Carla.


Issue 181

Add Drugs@FDA source
Breaking changes to regulatory approval API response objects and values

I didn't approve / request changes because I didn't know your thoughts on my PR Review comments. If you don't think my suggestions should be made, I think everything else looks good and can be merged.


Single VCF output

Add steps to generate single filtered VCF files - probably using bcftools.

Verified as working in v0.19.


badge component implementation

Added badge component
Refererneced from Button component
Opacity and onclick are not customizable
only border, color and bg are customizable
styles referenced from here

Please fix the failing test

Test case fixed locally. Any other review comment in the implementation?
Any other prop to update(add/remove) for Badge?

If above PR is acceptable, can you please add the label "hacktoberfest-accepted" to the PR?

Please push the fix. We can then have a review again to see if it everything looks alright.

Looks alright, thanks!


FirebaseFirestoreSwift creates invalid ios 14 bundle from version 8.11.0-beta and later

[REQUIRED] Step 1: Describe your environment

Xcode version: 13.3
Firebase SDK version: 8.11.0
Installation method: CocoaPods
Firebase Component: Firestore (Swift extension)
Target platform(s): iOS

[REQUIRED] Step 2: Describe the problem
Steps to reproduce:
After upgrading FirebaseFirestoreSwift from 8.10.0-beta to 8.11.0-beta (or later) IPA's build for ios 14 is no longer accepted by Appstore connect. After uploading the follow mail is sent by apple:

ITMS-90562: Invalid Bundle - One or more dynamic libraries that are referenced by your app are not present in the dylib search path.

Unfortunately, the error doesn't manifest until the build is uploaded, so it's very slow to debug.
When setting the deployment target to ios 15.0 the build is correctly approved by apple, but on ios 14.0 the above mail is sent, and the build marked as "invalid binary"
Relevant Code:
From Podfile
target "AppExp" do
    platform :ios, '14.0'. # Note: this only change the version for the pods, the version set in xcode for the primary target is the important one.
    use_frameworks!

    pod 'FirebaseFirestoreSwift' , '~> 8.11.0-beta'

end

This looks like an Apple issue with Xcode 13.3. See https://developer.apple.com/forums/thread/701949
Does using Xcode 13.2.1 resolve it?

Great catch, thank you!
I'll downgrade to xcode 13.2.1 and post an update

I can confirm that the older version of XCode solves this issue

I installed via SPM. Same issue, downloading another version of Xcode is quite heavy :(


${} is not useful

I use ${},but it is not useful

Can you provide more information? Closing this until there are reproducable steps


Revocation changes

This pull request modifies the behaviour of managing revocation registries so that controllers need to track and manage less.
When a credential definition is created, if revocation is enabled, the agent will create and publish revocation registry entry, definition, and tails file. It also publishes an additional set of registry and tails file to be swapped out when the current registry is full.
When issuing credentials, the correct revocation information is used automatically and rotates registries when needed creating a new ready-to-swap one in the background.
Demo has been updated.
With a tails server running, run the demo like so:
# Linux
./run_demo performance --revocation --tails-server-base-url http://<IP_ADDRESS>:6543
# Mac/Windows
./run_demo performance --revocation --tails-server-base-url http://host.docker.internal:6543

I'm closing this PR and re-opening in #619 due to DCO signoff issues.


Document TilerFactories attributes

https://github.com/stac-utils/titiler-pgstac/blob/ed26e44c951e97d6478609ea72b0de4f94c400d2/titiler/pgstac/factory.py#L115-L140
in https://stac-utils.github.io/titiler-pgstac/tiler_factories/

done in https://github.com/stac-utils/titiler-pgstac/commit/14e7c22c8ff9e40e4155b09edfbbb3e2e120d5f6


Make it possible to boot NixOS from a SCSI Disk on KVM

Currently NixOS can't boot from a SCSI disk as a KVM Guest.
I found this out while installing it on the new Linode KVM
platform.

That's something for nixos/modules/profiles/qemu-guest.nix I think, not something to be enabled for everyone.

Thanks @lethalman! Let me correct...

:+1:

I have corrected. Can I merge this then?

@rushmorem think so


Make sure path is not incorrectly modified via ref

Sometimes, the path name would be incorrectly listed as the last path processed. This makes sure the path is not updated via the path pointer.

Coverage increased (+0.07%) to 92.793% when pulling 1afa7d488ff3253c7ac400ce0ced8cedcd211773 on kibra:fix_path_name_munging into 7b706999d935b04cf2dbc71a5a5afcbd288aeb48 on d4l3k:master.

Coverage increased (+0.2%) to 91.556% when pulling b6527c038900e2d9cf86ea52d342a3306bf4f5d4 on kibra:fix_path_name_munging into e5fe1781f088e0805b61697842e909ccd9ea477a on d4l3k:master.

Tests updated and feedback addressed :)

Thanks for fixing that!


Log base 10 in powertoys run calculator module

Description of the new feature / enhancement
Currently typing log(x) for the calculator returns the natural log of x, with no way of using other bases. My suggestion would be to either make log(x) return the log base 10 of x and ln(x) the natural log of x, or to keep log(x) as it is and allow log10(x) to be the log base 10 of x.
Scenario when this would be used?
Base 10 logarithm is used pretty often.
Supporting information
No response

This should be easy, https://github.com/FlorianRappl/Mages/blob/devel/doc/functions.md shows it as an included fjnction

this is supported and works when i was testing.
=log(3)

CalculateHelper.cs has it in the regex.

I'm on the variant of main for 0.53 build but this should "just work" in 0.51.1

this is supported and works when i was testing.
=log(3)

CalculateHelper.cs has it in the regex.

As mentioned in my post, log(x) returns the natural log of x, which is log base e. I'm asking for a log base 10 command.

this is supported and works when i was testing.
=log(3)

CalculateHelper.cs has it in the regex.

As mentioned in my post, log(x) returns the natural log of x, which is log base e. I'm asking for a log base 10 command.

I can reproduce this. It returns ln(x) and not log(x).

 Yeah, we will fix it

I’ll try and get it in for 0.53

Just created a pull request.  It is an underlying calculator engine quirk

This is fixed in 0.53, get it at https://aka.ms/installPowerToys

Disclaimer: I don't use this tool as a calculator.
In most programming languages, log defaults to the natural logarithm. Though it should be noted that Google's calculator uses base 10 by default. log10 seems to be favoured among programming languages.
In mathematics, different notations are in use, but in some regions it's common to use lg for base 10: https://en.wikipedia.org/wiki/Logarithm#Particular_bases.
You can also write log(x)/log(b) to get the logarithm in base b, regardless of what base log uses.
I don't think this should be reverted or changed, but worth pointing out in case it gets revised later.

I think we should solve for the normal expectation which is base 10

There is no normal expectation, log by itself ambiguous and depends on the region and field of the user. E.g. in Russian and Germany literature, lg is the common notation for log_10 (https://mathworld.wolfram.com/Lg.html).
base 2: log2, lb, ld, lg
base e: loge, ln, log
base 10: log10, lg, log

In the end it doesn't really matter, since most formulas use a ratio of logarithms anyway, so the base becomes irrelevant.


Document PreReview workflow

Have asked Sean Wiseman for an example notification

Draft here based on Sean's example.
https://notify.coar-repositories.org/catalogue/workflows/repository-prereview-1/
There are a couple of issues with this so have emailed Sean about this

Have asked Sean to review this for us.

Sean agrees and is talking to Ted (?) from BioArxiv about correcting this. In the meantime, I have published the documented workflow here:
https://notify.coar-repositories.org/catalogue/workflows/repository-prereview-1/


Cross compiling with static linking fails on first compile due to Cmake configuration errors

Steps to reproduce
rustup target add x86_64-unknown-linux-musl
sudo apt install musl-tools
git clone git@github.com:aldanor/hdf5-rust.git --recurse
cd hdf5-rust
RUSTFLAGS="--codegen target-feature=+crt-static" \
    cargo run --example simple --features hdf5-sys/static,hdf5-sys/zlib,lzf,blosc --target "x86_64-unknown-linux-musl"

Relevant part (whole output in the bottom of the issue)
  --- stderr
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     TEST_LFS_WORKS_RUN (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     H5_PRINTF_LL_TEST_RUN (advanced)
     H5_PRINTF_LL_TEST_RUN__TRYRUN_OUTPUT (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     H5_LDOUBLE_TO_LONG_SPECIAL_RUN (advanced)
     H5_LDOUBLE_TO_LONG_SPECIAL_RUN__TRYRUN_OUTPUT (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     H5_LONG_TO_LDOUBLE_SPECIAL_RUN (advanced)
     H5_LONG_TO_LDOUBLE_SPECIAL_RUN__TRYRUN_OUTPUT (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     H5_LDOUBLE_TO_LLONG_ACCURATE_RUN (advanced)
     H5_LDOUBLE_TO_LLONG_ACCURATE_RUN__TRYRUN_OUTPUT (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     H5_LLONG_TO_LDOUBLE_CORRECT_RUN (advanced)
     H5_LLONG_TO_LDOUBLE_CORRECT_RUN__TRYRUN_OUTPUT (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     H5_DISABLE_SOME_LDOUBLE_CONV_RUN (advanced)
     H5_DISABLE_SOME_LDOUBLE_CONV_RUN__TRYRUN_OUTPUT (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     H5_NO_ALIGNMENT_RESTRICTIONS_RUN (advanced)
     H5_NO_ALIGNMENT_RESTRICTIONS_RUN__TRYRUN_OUTPUT (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  thread 'main' panicked at /home/marcbk/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5:

Simply invoking the same build command again completes the compilation and produces a statically linked binary
ldd target/x86_64-unknown-linux-musl/debug/examples/simple
# 	statically linked

Proposed solution
Let user define the missing defines that generate the errors.
Simply adding the following to https://github.com/aldanor/hdf5-rust/blob/master/hdf5-src/build.rs allows the build to continue with no errors on first try (while running cargo clean prior):
cfg.define("TEST_LFS_WORKS_RUN", "OFF");
cfg.define("H5_PRINTF_LL_TEST_RUN", "OFF");
cfg.define("H5_PRINTF_LL_TEST_RUN__TRYRUN_OUTPUT", "OFF");
cfg.define("H5_LDOUBLE_TO_LONG_SPECIAL_RUN", "OFF");
cfg.define("H5_LDOUBLE_TO_LONG_SPECIAL_RUN__TRYRUN_OUTPUT", "OFF");
cfg.define("H5_LONG_TO_LDOUBLE_SPECIAL_RUN", "OFF");
cfg.define("H5_LONG_TO_LDOUBLE_SPECIAL_RUN__TRYRUN_OUTPUT", "OFF");
cfg.define("H5_LDOUBLE_TO_LLONG_ACCURATE_RUN", "OFF");
cfg.define("H5_LDOUBLE_TO_LLONG_ACCURATE_RUN__TRYRUN_OUTPUT", "OFF");
cfg.define("H5_LLONG_TO_LDOUBLE_CORRECT_RUN", "OFF");
cfg.define("H5_LLONG_TO_LDOUBLE_CORRECT_RUN__TRYRUN_OUTPUT", "OFF");
cfg.define("H5_DISABLE_SOME_LDOUBLE_CONV_RUN", "OFF");
cfg.define("H5_DISABLE_SOME_LDOUBLE_CONV_RUN__TRYRUN_OUTPUT", "OFF");
cfg.define("H5_NO_ALIGNMENT_RESTRICTIONS_RUN", "OFF");
cfg.define("H5_NO_ALIGNMENT_RESTRICTIONS_RUN__TRYRUN_OUTPUT", "OFF");

Full compiler failure output
warning: `hdf5-sys` (build script) generated 1 warning
The following warnings were emitted during compilation:

warning<EMAIL_ADDRESS>Compiler family detection failed due to error: ToolNotFound: Failed to find tool. Is `musl-g++` installed?

error: failed to run custom build command for `hdf5-src v0.8.1 (/home/marcbk/test/hdf5-rust/hdf5-src)`

Caused by:
  process didn't exit successfully: `/home/marcbk/test/hdf5-rust/target/debug/build/hdf5-src-f7da7ad76cdac73e/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-changed=build.rs
  cargo:zlib_header=/home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/libz-sys-4a798fdc2f99a2cc/out/include/zlib.h
  cargo:zlib=z
  CMAKE_TOOLCHAIN_FILE_x86_64-unknown-linux-musl = None
  CMAKE_TOOLCHAIN_FILE_x86_64_unknown_linux_musl = None
  TARGET_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_x86_64-unknown-linux-musl = None
  CMAKE_GENERATOR_x86_64_unknown_linux_musl = None
  TARGET_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  cargo:warning=Compiler family detection failed due to error: ToolNotFound: Failed to find tool. Is `musl-g++` installed?
  CMAKE_PREFIX_PATH_x86_64-unknown-linux-musl = None
  CMAKE_PREFIX_PATH_x86_64_unknown_linux_musl = None
  TARGET_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_x86_64-unknown-linux-musl = None
  CMAKE_x86_64_unknown_linux_musl = None
  TARGET_CMAKE = None
  CMAKE = None
  running: cd "/home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build" && CMAKE_PREFIX_PATH="" "cmake" "/home/marcbk/test/hdf5-rust/hdf5-src/ext/hdf5" "-DHDF5_NO_PACKAGES=ON" "-DBUILD_SHARED_LIBS=OFF" "-DBUILD_TESTING=OFF" "-DHDF5_BUILD_TOOLS=OFF" "-DHDF5_BUILD_EXAMPLES=OFF" "-DHDF5_BUILD_JAVA=OFF" "-DHDF5_BUILD_FORTRAN=OFF" "-DHDF5_BUILD_CPP_LIB=OFF" "-DHDF5_BUILD_UTILS=OFF" "-DHDF5_ENABLE_PARALLEL=OFF" "-DHDF5_ENABLE_DEPRECATED_SYMBOLS=OFF" "-DHDF5_ENABLE_THREADSAFE=OFF" "-DALLOW_UNSUPPORTED=OFF" "-DHDF5_BUILD_HL_LIB=OFF" "-DHDF5_ENABLE_Z_LIB_SUPPORT=ON" "-DH5_ZLIB_HEADER=/home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/libz-sys-4a798fdc2f99a2cc/out/include/zlib.h" "-DZLIB_STATIC_LIBRARY=z" "-DCMAKE_SYSTEM_NAME=Linux" "-DCMAKE_SYSTEM_PROCESSOR=x86_64" "-DCMAKE_INSTALL_PREFIX=/home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64 -static" "-DCMAKE_C_COMPILER=/usr/bin/musl-gcc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64 -static" "-DCMAKE_CXX_COMPILER=musl-g++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64 -static" "-DCMAKE_ASM_COMPILER=/usr/bin/musl-gcc" "-DCMAKE_BUILD_TYPE=Debug"
  -- The C compiler identification is GNU 11.4.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/musl-gcc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- SOVERSION: 103.3.0
  -- SOVERSION_TOOLS: 102.0.0
  -- SOVERSION_CXX: 103.3.0
  -- SOVERSION_F: 102.1.0
  -- SOVERSION_HL: 100.1.4
  -- SOVERSION_HL_CXX: 100.1.5
  -- SOVERSION_HL_F: 100.0.6
  -- SOVERSION_JAVA: 100.5.0
  -- Looking for include file sys/file.h
  -- Looking for include file sys/file.h - found
  -- Looking for include files sys/file.h, sys/ioctl.h
  -- Looking for include files sys/file.h, sys/ioctl.h - found
  -- Looking for 3 include files sys/file.h, ..., sys/resource.h
  -- Looking for 3 include files sys/file.h, ..., sys/resource.h - found
  -- Looking for 4 include files sys/file.h, ..., sys/socket.h
  -- Looking for 4 include files sys/file.h, ..., sys/socket.h - found
  -- Looking for 5 include files sys/file.h, ..., sys/stat.h
  -- Looking for 5 include files sys/file.h, ..., sys/stat.h - found
  -- Looking for 6 include files sys/file.h, ..., sys/time.h
  -- Looking for 6 include files sys/file.h, ..., sys/time.h - found
  -- Looking for 7 include files sys/file.h, ..., sys/types.h
  -- Looking for 7 include files sys/file.h, ..., sys/types.h - found
  -- Looking for 8 include files sys/file.h, ..., features.h
  -- Looking for 8 include files sys/file.h, ..., features.h - found
  -- Looking for 9 include files sys/file.h, ..., dirent.h
  -- Looking for 9 include files sys/file.h, ..., dirent.h - found
  -- Looking for 10 include files sys/file.h, ..., setjmp.h
  -- Looking for 10 include files sys/file.h, ..., setjmp.h - found
  -- Looking for 11 include files sys/file.h, ..., stddef.h
  -- Looking for 11 include files sys/file.h, ..., stddef.h - found
  -- Looking for 12 include files sys/file.h, ..., stdint.h
  -- Looking for 12 include files sys/file.h, ..., stdint.h - found
  -- Looking for 13 include files sys/file.h, ..., unistd.h
  -- Looking for 13 include files sys/file.h, ..., unistd.h - found
  -- Looking for 14 include files sys/file.h, ..., io.h
  -- Looking for 14 include files sys/file.h, ..., io.h - not found
  -- Looking for 14 include files sys/file.h, ..., winsock2.h
  -- Looking for 14 include files sys/file.h, ..., winsock2.h - not found
  -- Looking for 14 include files sys/file.h, ..., globus/common.h
  -- Looking for 14 include files sys/file.h, ..., globus/common.h - not found
  -- Looking for 14 include files sys/file.h, ..., pdb.h
  -- Looking for 14 include files sys/file.h, ..., pdb.h - not found
  -- Looking for 14 include files sys/file.h, ..., pthread.h
  -- Looking for 14 include files sys/file.h, ..., pthread.h - found
  -- Looking for 15 include files sys/file.h, ..., srbclient.h
  -- Looking for 15 include files sys/file.h, ..., srbclient.h - not found
  -- Looking for 15 include files sys/file.h, ..., string.h
  -- Looking for 15 include files sys/file.h, ..., string.h - found
  -- Looking for 16 include files sys/file.h, ..., strings.h
  -- Looking for 16 include files sys/file.h, ..., strings.h - found
  -- Looking for 17 include files sys/file.h, ..., stdlib.h
  -- Looking for 17 include files sys/file.h, ..., stdlib.h - found
  -- Looking for 18 include files sys/file.h, ..., memory.h
  -- Looking for 18 include files sys/file.h, ..., memory.h - found
  -- Looking for 19 include files sys/file.h, ..., dlfcn.h
  -- Looking for 19 include files sys/file.h, ..., dlfcn.h - found
  -- Looking for 20 include files sys/file.h, ..., inttypes.h
  -- Looking for 20 include files sys/file.h, ..., inttypes.h - found
  -- Looking for 21 include files sys/file.h, ..., netinet/in.h
  -- Looking for 21 include files sys/file.h, ..., netinet/in.h - found
  -- Looking for 22 include files sys/file.h, ..., netdb.h
  -- Looking for 22 include files sys/file.h, ..., netdb.h - found
  -- Looking for 23 include files sys/file.h, ..., arpa/inet.h
  -- Looking for 23 include files sys/file.h, ..., arpa/inet.h - found
  -- Looking for 24 include files sys/file.h, ..., stdbool.h
  -- Looking for 24 include files sys/file.h, ..., stdbool.h - found
  -- Looking for include file quadmath.h
  -- Looking for include file quadmath.h - found
  -- Looking for ceil in m;
  -- Looking for ceil in m; - found
  -- Looking for dlopen in dl;m
  -- Looking for dlopen in dl;m - found
  -- Looking for WSAStartup in ws2_32;m;dl
  -- Looking for WSAStartup in ws2_32;m;dl - not found
  -- Looking for gethostbyname in wsock32;m;dl
  -- Looking for gethostbyname in wsock32;m;dl - not found
  -- Looking for gethostname in ucb;m;dl
  -- Looking for gethostname in ucb;m;dl - not found
  -- Performing Other Test STDC_HEADERS - Success
  -- Performing TEST_LFS_WORKS... no
  -- Performing Other Test HAVE_OFF64_T - Failed
  -- Looking for fseeko
  -- Looking for fseeko - found
  -- Performing Test HAVE_STAT64_STRUCT
  -- Performing Test HAVE_STAT64_STRUCT - Failed
  -- Looking for sys/types.h
  -- Looking for sys/types.h - found
  -- Looking for stdint.h
  -- Looking for stdint.h - found
  -- Looking for stddef.h
  -- Looking for stddef.h - found
  -- Check size of char
  -- Check size of char - done
  -- Check size of short
  -- Check size of short - done
  -- Check size of int
  -- Check size of int - done
  -- Check size of unsigned
  -- Check size of unsigned - done
  -- Check size of long
  -- Check size of long - done
  -- Check size of long long
  -- Check size of long long - done
  -- Check size of __int64
  -- Check size of __int64 - failed
  -- Check size of float
  -- Check size of float - done
  -- Check size of double
  -- Check size of double - done
  -- Check size of long double
  -- Check size of long double - done
  -- Check size of int8_t
  -- Check size of int8_t - done
  -- Check size of uint8_t
  -- Check size of uint8_t - done
  -- Check size of int_least8_t
  -- Check size of int_least8_t - done
  -- Check size of uint_least8_t
  -- Check size of uint_least8_t - done
  -- Check size of int_fast8_t
  -- Check size of int_fast8_t - done
  -- Check size of uint_fast8_t
  -- Check size of uint_fast8_t - done
  -- Check size of int16_t
  -- Check size of int16_t - done
  -- Check size of uint16_t
  -- Check size of uint16_t - done
  -- Check size of int_least16_t
  -- Check size of int_least16_t - done
  -- Check size of uint_least16_t
  -- Check size of uint_least16_t - done
  -- Check size of int_fast16_t
  -- Check size of int_fast16_t - done
  -- Check size of uint_fast16_t
  -- Check size of uint_fast16_t - done
  -- Check size of int32_t
  -- Check size of int32_t - done
  -- Check size of uint32_t
  -- Check size of uint32_t - done
  -- Check size of int_least32_t
  -- Check size of int_least32_t - done
  -- Check size of uint_least32_t
  -- Check size of uint_least32_t - done
  -- Check size of int_fast32_t
  -- Check size of int_fast32_t - done
  -- Check size of uint_fast32_t
  -- Check size of uint_fast32_t - done
  -- Check size of int64_t
  -- Check size of int64_t - done
  -- Check size of uint64_t
  -- Check size of uint64_t - done
  -- Check size of int_least64_t
  -- Check size of int_least64_t - done
  -- Check size of uint_least64_t
  -- Check size of uint_least64_t - done
  -- Check size of int_fast64_t
  -- Check size of int_fast64_t - done
  -- Check size of uint_fast64_t
  -- Check size of uint_fast64_t - done
  -- Check size of size_t
  -- Check size of size_t - done
  -- Check size of ssize_t
  -- Check size of ssize_t - done
  -- Check size of ptrdiff_t
  -- Check size of ptrdiff_t - done
  -- Check size of off_t
  -- Check size of off_t - done
  -- Check size of off64_t
  -- Check size of off64_t - failed
  -- Check size of time_t
  -- Check size of time_t - done
  -- Check size of _Bool
  -- Check size of _Bool - done
  -- Performing Other Test DEV_T_IS_SCALAR - Success
  -- Looking for CLOCK_MONOTONIC
  -- Looking for CLOCK_MONOTONIC - not found
  -- Performing Test H5_HAVE_TM_GMTOFF
  -- Performing Test H5_HAVE_TM_GMTOFF - Success
  -- Performing Test H5_HAVE___TM_GMTOFF
  -- Performing Test H5_HAVE___TM_GMTOFF - Success
  -- Performing Test H5_TIME_WITH_SYS_TIME
  -- Performing Test H5_TIME_WITH_SYS_TIME - Success
  -- Performing Test H5_HAVE_STRUCT_TIMEZONE
  -- Performing Test H5_HAVE_STRUCT_TIMEZONE - Failed
  -- Looking for gettimeofday
  -- Looking for gettimeofday - found
  -- Performing Other Test GETTIMEOFDAY_GIVES_TZ - Success
  -- Performing Other Test HAVE_TM_ZONE - Failed
  -- Performing Other Test HAVE_STRUCT_TM_TM_ZONE - Failed
  -- Performing Other Test HAVE_TIMEZONE - Success
  -- Performing Test H5_HAVE_STAT_ST_BLOCKS
  -- Performing Test H5_HAVE_STAT_ST_BLOCKS - Success
  -- Looking for ioctl
  -- Looking for ioctl - found
  -- Performing Test H5_HAVE_STRUCT_VIDEOCONFIG
  -- Performing Test H5_HAVE_STRUCT_VIDEOCONFIG - Failed
  -- Performing Test H5_HAVE_STRUCT_TEXT_INFO
  -- Performing Test H5_HAVE_STRUCT_TEXT_INFO - Failed
  -- Looking for _getvideoconfig
  -- Looking for _getvideoconfig - not found
  -- Looking for gettextinfo
  -- Looking for gettextinfo - not found
  -- Looking for _scrsize
  -- Looking for _scrsize - not found
  -- Looking for GetConsoleScreenBufferInfo
  -- Looking for GetConsoleScreenBufferInfo - not found
  -- Looking for TIOCGWINSZ
  -- Looking for TIOCGWINSZ - found
  -- Looking for TIOCGETD
  -- Looking for TIOCGETD - found
  -- Looking for getpwuid
  -- Looking for getpwuid - found
  -- Looking for alarm
  -- Looking for alarm - found
  -- Looking for fcntl
  -- Looking for fcntl - found
  -- Looking for flock
  -- Looking for flock - found
  -- Looking for fork
  -- Looking for fork - found
  -- Looking for frexpf
  -- Looking for frexpf - found
  -- Looking for frexpl
  -- Looking for frexpl - found
  -- Looking for gethostname
  -- Looking for gethostname - found
  -- Looking for getrusage
  -- Looking for getrusage - found
  -- Looking for llround
  -- Looking for llround - found
  -- Looking for llroundf
  -- Looking for llroundf - found
  -- Looking for lround
  -- Looking for lround - found
  -- Looking for lroundf
  -- Looking for lroundf - found
  -- Looking for lstat
  -- Looking for lstat - found
  -- Looking for pread
  -- Looking for pread - found
  -- Looking for pwrite
  -- Looking for pwrite - found
  -- Looking for rand_r
  -- Looking for rand_r - found
  -- Looking for random
  -- Looking for random - found
  -- Looking for round
  -- Looking for round - found
  -- Looking for roundf
  -- Looking for roundf - found
  -- Looking for setsysinfo
  -- Looking for setsysinfo - not found
  -- Looking for signal
  -- Looking for signal - found
  -- Looking for longjmp
  -- Looking for longjmp - found
  -- Looking for setjmp
  -- Looking for setjmp - found
  -- Looking for siglongjmp
  -- Looking for siglongjmp - found
  -- Looking for sigsetjmp
  -- Looking for sigsetjmp - found
  -- Looking for sigprocmask
  -- Looking for sigprocmask - found
  -- Looking for snprintf
  -- Looking for snprintf - found
  -- Looking for srandom
  -- Looking for srandom - found
  -- Looking for strdup
  -- Looking for strdup - found
  -- Looking for strtoll
  -- Looking for strtoll - found
  -- Looking for strtoull
  -- Looking for strtoull - found
  -- Looking for symlink
  -- Looking for symlink - found
  -- Looking for system
  -- Looking for system - found
  -- Looking for tmpfile
  -- Looking for tmpfile - found
  -- Looking for asprintf
  -- Looking for asprintf - found
  -- Looking for vasprintf
  -- Looking for vasprintf - found
  -- Looking for waitpid
  -- Looking for waitpid - found
  -- Looking for vsnprintf
  -- Looking for vsnprintf - found
  -- Performing Other Test VSNPRINTF_WORKS - Success
  -- Performing Other Test HAVE_ATTRIBUTE - Success
  -- Performing Other Test HAVE_C99_FUNC - Success
  -- Performing Other Test HAVE_FUNCTION - Success
  -- Performing Other Test HAVE_C99_DESIGNATED_INITIALIZER - Success
  -- Performing Other Test SYSTEM_SCOPE_THREADS - Success
  -- Performing Other Test HAVE_SOCKLEN_T - Success
  -- Performing Other Test HAVE_INLINE - Success
  -- Performing Other Test HAVE___INLINE__ - Success
  -- Performing Other Test HAVE___INLINE - Success
  -- Checking for appropriate format for 64 bit long:
  -- Width test failed with result: PLEASE_FILL_OUT-FAILED_TO_RUN
  -- Checking for appropriate format for 64 bit long: not found
  -- Looking for difftime
  -- Looking for difftime - found
  -- Looking for clock_gettime
  -- Looking for clock_gettime - found
  -- Looking for clock_gettime in rt
  -- Looking for clock_gettime in rt - found
  -- Looking for clock_gettime in posix4
  -- Looking for clock_gettime in posix4 - not found
  -- Check size of __float128
  -- Check size of __float128 - done
  -- Check size of _Quad
  -- Check size of _Quad - failed
  -- Checking IF your system converts long double to (unsigned) long values with special algorithm... no
  -- Checking IF your system can convert (unsigned) long to long double values with special algorithm... no
  -- Checking IF correctly converting long double to (unsigned) long long values... no
  -- Checking IF correctly converting (unsigned) long long to long double values... no
  -- Checking IF the cpu is power9 and cannot correctly converting long double values... no
  -- Checking IF alignment restrictions are strictly enforced... no
  -- Warnings Configuration: default:  -std=c99  -ffunction-sections -fdata-sections -fPIC -m64 -static :   -ffunction-sections -fdata-sections -fPIC -m64 -static
  -- CMAKE_C_FLAGS_GENERAL=-Wall;-Wcast-qual;-Wconversion;-Wextra;-Wfloat-equal;-Wformat=2;-Winit-self;-Winvalid-pch;-Wmissing-include-dirs;-Wno-c++-compat;-Wno-format-nonliteral;-Wshadow;-Wundef;-Wwrite-strings;-pedantic
  -- Filter ZLIB is ON
  -- Configuring incomplete, errors occurred!
  See also "/home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/CMakeFiles/CMakeOutput.log".
  See also "/home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/CMakeFiles/CMakeError.log".

  --- stderr
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     TEST_LFS_WORKS_RUN (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     H5_PRINTF_LL_TEST_RUN (advanced)
     H5_PRINTF_LL_TEST_RUN__TRYRUN_OUTPUT (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     H5_LDOUBLE_TO_LONG_SPECIAL_RUN (advanced)
     H5_LDOUBLE_TO_LONG_SPECIAL_RUN__TRYRUN_OUTPUT (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     H5_LONG_TO_LDOUBLE_SPECIAL_RUN (advanced)
     H5_LONG_TO_LDOUBLE_SPECIAL_RUN__TRYRUN_OUTPUT (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     H5_LDOUBLE_TO_LLONG_ACCURATE_RUN (advanced)
     H5_LDOUBLE_TO_LLONG_ACCURATE_RUN__TRYRUN_OUTPUT (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     H5_LLONG_TO_LDOUBLE_CORRECT_RUN (advanced)
     H5_LLONG_TO_LDOUBLE_CORRECT_RUN__TRYRUN_OUTPUT (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     H5_DISABLE_SOME_LDOUBLE_CONV_RUN (advanced)
     H5_DISABLE_SOME_LDOUBLE_CONV_RUN__TRYRUN_OUTPUT (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
     H5_NO_ALIGNMENT_RESTRICTIONS_RUN (advanced)
     H5_NO_ALIGNMENT_RESTRICTIONS_RUN__TRYRUN_OUTPUT (advanced)
  For details see /home/marcbk/test/hdf5-rust/target/x86_64-unknown-linux-musl/debug/build/hdf5-src-e377d37d8310b99c/out/build/TryRunResults.cmake
  thread 'main' panicked at /home/marcbk/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5:

  command did not execute successfully, got: exit status: 1

  build script failed, must exit now

Is it safe to add these config options when not cross compiling?

So I forked the repo and added the above config options to the build.rs file and it works fine when building in the repo but the configuration options seem to have no affect when I patch my project with my fork for hdf5-src, do you know what may be going on?

@Isaac-Leonard Does it work if you use the fork  (hdf5-metno-src) at https://github.com/metno/hdf5-rust?

FYI: I'm following this issue but won't be contributing much to the discussion due to my very limited HDF5 knowledge.

I couldn't get it to work unfortunately though I think due to dependency conflicts rather then because of the code.
I've solved the issue for now though by building on an older version of a platform that works without cross compiling.


[Bug] SGLang's OpenAI interface fails with Llama-3.2-1B due to missing chat template

Checklist

[ ] 1. I have searched related issues but cannot get the expected help.
[ ] 2. The bug has not been fixed in the latest version.
[ ] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
[ ] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
[ ] 5. Please use English, otherwise it will be closed.

Describe the bug
The official SGLang OpenAI interface example works without chat template, but when using Llama-3.2-1B, it throws ValueError requiring chat template. This seems inconsistent with the documentation.
Error: "Cannot use chat template functions because tokenizer.chat_template is not set"
Reproduction
(camel-ai-py3.10) mi@mi:~/gitlab/ml_explore/camel$ cat test/test_openai_sglang.py
from sglang.utils import (
execute_shell_command,
wait_for_server,
terminate_process,
)
server_process = execute_shell_command(
"python -m sglang.launch_server --model-path meta-llama/Llama-3.2-1B --port 30000 --host <IP_ADDRESS>"
)
wait_for_server("http://localhost:30000")
from openai import OpenAI
client = OpenAI(base_url="http://<IP_ADDRESS>:30000/v1", api_key="None")
response = client.chat.completions.create(
model="meta-llama/Llama-3.2-1B",
messages=[
{"role": "user", "content": "List 3 countries and their capitals."},
],
temperature=0,
max_tokens=64,
)
print(response)
terminate_process(server_process)
(camel-ai-py3.10) mi@mi:~/gitlab/ml_explore/camel$ python3 test/test_openai_sglang.py
[2024-12-10 21:23:27] server_args=ServerArgs(model_path='meta-llama/Llama-3.2-1B', tokenizer_path='meta-llama/Llama-3.2-1B', tokenizer_mode='auto', skip_tokenizer_init=False, load_format='auto', trust_remote_code=False, dtype='auto', kv_cache_dtype='auto', quantization=None, context_length=None, device='cuda', served_model_name='meta-llama/Llama-3.2-1B', chat_template=None, is_embedding=False, host='<IP_ADDRESS>', port=30000, mem_fraction_static=0.88, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=2048, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, tp_size=1, stream_interval=1, random_seed=32666879, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http=None, log_requests=False, show_time_cost=False, enable_metrics=False, decode_log_interval=40, api_key=None, file_storage_pth='SGLang_storage', enable_cache_report=False, dp_size=1, load_balance_method='round_robin', dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, lora_paths=None, max_loras_per_batch=8, attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', disable_radix_cache=False, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, disable_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=False, enable_dp_attention=False, enable_torch_compile=False, torch_compile_max_bs=32, cuda_graph_max_bs=4, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, num_continuous_decode_steps=1, delete_ckpt_after_loading=False)
[2024-12-10 21:23:32 TP0] Init torch distributed begin.
[2024-12-10 21:23:33 TP0] Load weight begin. avail mem=23.32 GB
INFO 12-10 21:23:36 weight_utils.py:243] Using model weights format ['*.safetensors']
INFO 12-10 21:23:36 weight_utils.py:288] No model.safetensors.index.json found in remote.
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  4.22it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  4.21it/s]
[2024-12-10 21:23:37 TP0] Load weight end. type=LlamaForCausalLM, dtype=torch.bfloat16, avail mem=20.45 GB
[2024-12-10 21:23:37 TP0] Memory pool end. avail mem=1.66 GB
[2024-12-10 21:23:37 TP0] Capture cuda graph begin. This can take up to several minutes.
[2024-12-10 21:23:38 TP0] max_total_num_tokens=578385, max_prefill_tokens=16384, max_running_requests=2260, context_len=131072
[2024-12-10 21:23:39] INFO:     Started server process [2393459]
[2024-12-10 21:23:39] INFO:     Waiting for application startup.
[2024-12-10 21:23:39] INFO:     Application startup complete.
[2024-12-10 21:23:39] INFO:     Uvicorn running on http://<IP_ADDRESS>:30000 (Press CTRL+C to quit)
[2024-12-10 21:23:39] INFO:     <IP_ADDRESS>:37630 - "GET /v1/models HTTP/1.1" 200 OK
[2024-12-10 21:23:40] INFO:     <IP_ADDRESS>:37642 - "GET /get_model_info HTTP/1.1" 200 OK
[2024-12-10 21:23:40 TP0] Prefill batch. #new-seq: 1, #new-token: 7, #cached-token: 0, cache hit rate: 0.00%, token usage: 0.00, #running-req: 0, #queue-req: 0
[2024-12-10 21:23:40] INFO:     <IP_ADDRESS>:37646 - "POST /generate HTTP/1.1" 200 OK
[2024-12-10 21:23:40] The server is fired up and ready to roll!
<IPython.core.display.HTML object>
[2024-12-10 21:23:44] INFO:     <IP_ADDRESS>:37648 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
[2024-12-10 21:23:44] ERROR:    Exception in ASGI application
Traceback (most recent call last):
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
result = await app(  # type: ignore[func-returns-value]
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in call
return await self.app(scope, receive, send)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in call
await super().call(scope, receive, send)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/starlette/applications.py", line 113, in call
await self.middleware_stack(scope, receive, send)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/starlette/middleware/errors.py", line 187, in call
raise exc
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/starlette/middleware/errors.py", line 165, in call
await self.app(scope, receive, _send)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in call
await self.app(scope, receive, send)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in call
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 715, in call
await self.middleware_stack(scope, receive, send)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 735, in app
await route.handle(scope, receive, send)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 288, in handle
await self.app(scope, receive, send)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 76, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 73, in app
response = await f(request)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/fastapi/routing.py", line 301, in app
raw_response = await run_endpoint_function(
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
return await dependant.call(**values)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/sglang/srt/metrics/func_timer.py", line 74, in async_wrapper
return await func(*args, **kwargs)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/sglang/srt/server.py", line 318, in openai_v1_chat_completions
return await v1_chat_completions(tokenizer_manager, raw_request)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/sglang/srt/openai_api/adapter.py", line 1100, in v1_chat_completions
adapted_request, request = v1_chat_generate_request(all_requests, tokenizer_manager)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/sglang/srt/openai_api/adapter.py", line 885, in v1_chat_generate_request
prompt_ids = tokenizer_manager.tokenizer.apply_chat_template(
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1617, in apply_chat_template
chat_template = self.get_chat_template(chat_template, tools)
File "/home/mi/.cache/pypoetry/virtualenvs/camel-ai-34ULexV3-py3.10/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1785, in get_chat_template
raise ValueError(
ValueError: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation at https://huggingface.co/docs/transformers/main/en/chat_templating
Environment
(camel-ai-py3.10) mi@mi:~/gitlab/ml_explore/camel$ python3 -m sglang.check_env
Python: 3.10.15 (main, Oct  3 2024, 07:27:34) [GCC 11.2.0]
CUDA available: True
GPU 0: NVIDIA GeForce RTX 3090
GPU 0 Compute Capability: 8.6
CUDA_HOME: None
PyTorch: 2.5.1+cu124
sglang: 0.3.6.post2
flashinfer: 0.1.6+cu124torch2.4
triton: 3.1.0
transformers: 4.47.0
torchao: 0.6.1
numpy: 1.26.4
aiohttp: 3.11.10
fastapi: 0.115.6
hf_transfer: 0.1.8
huggingface_hub: 0.26.5
interegular: 0.3.3
modelscope: 1.20.1
orjson: 3.10.12
packaging: 23.2
psutil: 6.1.0
pydantic: 2.9.2
multipart: 0.0.19
zmq: 26.2.0
uvicorn: 0.32.1
uvloop: 0.21.0
vllm: 0.6.4.post1
openai: 1.57.1
anthropic: 0.29.2
decord: 0.6.0
NVIDIA Topology:
GPU0	CPU Affinity	NUMA Affinity	GPU NUMA ID
GPU0	 X 	0-23	0		N/A
Legend:
X    = Self
SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
PIX  = Connection traversing at most a single PCIe bridge
NV#  = Connection traversing a bonded set of # NVLinks
ulimit soft: 1024

@NeilJohnson0930 Thanks for point this out. As you can see in our docs, our documents are complied in our CI and can run successfully with the results shown in the docs directly.
What version of SGLang you are using?

I rerun your codes in v0.3.6 and have the same error. Could you plz update your SGLang to the latest version v0.4? The problem is solved on it.
from sglang.utils import (
    execute_shell_command,
    wait_for_server,
    terminate_process,
    print_highlight,
)

import requests

server_process = execute_shell_command(
    """
python3 -m sglang.launch_server --model-path meta-llama/Llama-3.2-1B --port=30010
"""
)

wait_for_server("http://localhost:30010")

Hi Chenyang, upgrading SGLang to 0.4.0 doesn’t seem any help, probelm still.

PS: @NeilJohnson0930 I found that meta-llama/Meta-Llama-3.1-8B also lost its chat_template 😓

hi chenyang, I tested the *-Instruct model and confirmed that it works properly. We will use this model to bypass the issue. Thanks a lot!


LPA-3420 ECS Scaling on Demand for Make a LPA

Purpose
Use CPU and Memory metrics tracking to scale ECS services to meet demand (inferred by the load on services).
Fixes LPA-3420
Approach
Create an IAM role for ECS Autoscaling
Given the duplication of resources required to set up autoscaling for all ECS services, resources will be defined once using a terraform module, and use that module for each service.
Learning
Checklist

[x] I have performed a self-review of my own code
[x] I have updated documentation (Confluence/GitHub wiki/tech debt doc) where relevant
[ ] I have added tests to prove my work
[x] The product team have tested these changes

TF workspace env needs clearing.

spotted and fixed :D

Scaling occurred during testing with cpu and memory usage targets

Scaling for front end remained at max of 5 due memory usage
No errors during testing with autoscaling

Auto Scaling set up for each service that requires scaling (admin does not)


Click probe not producing results until around 1810

I think this is a problem on the clientside. This request is returning results but they aren't showing in the search results.


Consider dropping initServiceWorkerMessageEvent()

ServiceWorkerMessageEvent has initServiceWorkerMessageEvent() [1], but I think we should drop this method because as the note[2] says, initEvent() method only exists for historical reason and we can use constructor to initialize events.
[1] https://slightlyoff.github.io/ServiceWorker/spec/service_worker/#serviceworkermessage-event-initserviceworkermessageevent-method
[2] https://dom.spec.whatwg.org/#dom-event-initeventtype-bubbles-cancelable

Agreed. @jakearchibald @annevk Can there be any concern about dropping this method considering devs historically having used the initMessageEvent()? (Although they're now defined as unrelated interfaces.)

+1, new events should not have this

Thanks!


fix: generate correct auth mode for admin queries in amplifyconfiguration.json

Updated the fronted config files creator to have the correct authorization type for Admin Queries. The earlier implementation had AWS_IAM hardcoded. The updated config looks similar to this
{
  "api": {
        "plugins": {
                "AdminQueries": {
                    "endpointType": "REST",
                    "endpoint": "https://zzzz.execute-api.us-west-2.amazonaws.com/dev",
                    "region": "us-west-2",
                    "authorizationType": "AMAZON_COGNITO_USER_POOLS"
                }
            }
        }
    }
}

fix #6983

Description of changes

Issue #, if available

Description of how you validated changes
Checklist

[x] PR description included
[x] yarn test passes
[ ] Tests are changed or added
[ ] Relevant documentation is changed or added (and PR referenced)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

For existing projects, what's the plan for mitigation? Will we auto-add the authorization type in amplify-meta.json?

For existing projects, what's the plan for mitigation? Will we auto-add the authorization type in amplify-meta.json?

We are not doing any migration work as the Admin Queries never worked in that case. The fix would be to remove the admin queries and add it back again to get the fix.

@yuth not a great solution, I updated my admin queries functions to have some custom code so removing and re-adding isn't a fun time.

👋 Hi, this issue was referenced in the v4.50.2 release!
Check out the release notes here https://github.com/aws-amplify/amplify-cli/releases/tag/v4.50.2.


Styled compoents way doesn't seem to work correctly.

Yesterday I've setup some templates which all worked fine a fresh qwik project. Now I'm trying it again and it doesn't seem to be correctly set up as the component get's literally inserted into html.

If I do the vanilla-extract way of inserting the object into the class field it works normally however.
This is the code:

It's a component, you have to capitalize the name, so "StyledMessage". If you start the name with lowercase, the JSX transform considers it a regular element like "div".
Enjoy :)

@wmertens Is there any place I can consider asking styled-vanilla-extract questions, that may not require an entire issue here?
I'm like constantly running into small things I can't find any info about.

@TheAyes come over to the qwik discord, ask in the help channel. I'm often there or others can help too
https://qwikcommunity.com/

@TheAyes come over to the qwik discord, ask in the help channel. I'm often there or others can help too
https://qwikcommunity.com/


GDM missing icons

Describe the bug
GDM is missing all its icons (network, battery, user profile, back button, ...). See screenshot.
git bisect says it's https://github.com/NixOS/nixpkgs/commit/b1e73fa2e086f1033a33d93524ae2a1781d12b95
See also related PR https://github.com/NixOS/nixpkgs/pull/163623
Steps To Reproduce
Used this config (based on the GNOME ISO) for bisecting with VMs. The mkOverrides are there to reduce the build time a little bit.
{ lib, modulesPath, ... }:

with lib;

{
  imports = [ "${modulesPath}/installer/cd-dvd/installation-cd-graphical-base.nix" ];

  isoImage.edition = "gnome";

  services.xserver.desktopManager.gnome = {
    # Add Firefox and other tools useful for installation to the launcher
    favoriteAppsOverride = ''
      [org.gnome.shell]
      favorite-apps=[ 'firefox.desktop', 'nixos-manual.desktop', 'org.gnome.Terminal.desktop', 'org.gnome.Nautilus.desktop', 'gparted.desktop' ]
    '';
    enable = true;
  };

  documentation.enable = lib.mkOverride 9 false;
  documentation.man.enable = lib.mkOverride 9 false;
  documentation.doc.enable = lib.mkOverride 9 false;
  documentation.info.enable = lib.mkOverride 9 false;
  documentation.nixos.enable = lib.mkOverride 9 false;
  services.gnome.core-utilities.enable = lib.mkOverride 9 false;
  services.xserver.displayManager = {
    gdm = {
      enable = true;
      # autoSuspend makes the machine automatically suspend after inactivity.
      # It's possible someone could/try to ssh'd into the machine and obviously
      # have issues because it's inactive.
      # See:
      # * https://github.com/NixOS/nixpkgs/pull/63790
      # * https://gitlab.gnome.org/GNOME/gnome-control-center/issues/22
      autoSuspend = false;
    };
  };

}

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots

Notify maintainers
$ nix eval gnome.gdm.meta.maintainers --apply "map (x: x.github)"
[ "bobby285271" "hedning" "jtojnar" "dasj19" "maxeaubrey" ]

@bobby285271
@hedning
@jtojnar
@dasj19
@maxeaubrey
PR author: @ilya-fedin
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.16.20, NixOS, 22.05 (Quokka), 22.05.20220426.e10da1c`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix Super) 2.8.0pre20220415_0883a09`
 - channels(root): `""`
 - channels(max): `""`
 - nixpkgs: `/etc/nix/flake-channels/nixpkgs`

I also have the same problem.

We should probably add adwaita-icon-theme to gdm. Probably in the module like we did for the a11y icon https://github.com/NixOS/nixpkgs/commit/87d18174d317e78fdfd9ab1fd35b685deb812aa8. Maybe a wrapper in gnome-shell or gdm would be a better place but I am currently not sure which is more appropriate.


Update Suppression File

Update the base suppression file from the generatedSuppressions branch to prepare for the 7.3.1 release.

Delineator added to generatedSuppressions.xml:  https://github.com/jeremylong/DependencyCheck/pull/new/updateSuppressions
Intent is to delete these entries from the generated suppressions once 7.3.1 is no longer supported.

The purpose of this is to keep the base suppression file current - even when users do not utilize the hosted suppression file (some run ODC in an offline mode).

@aikebah the intent is to ship 7.3.1 this week.


Replace /ping command with something more informative

Maybe something like /info that would give a summary of the server, with information such as:

uptime of the server
number of claims
unique players with claims
time since the last flush

Will need to add a way to record flushes before being able to acheive that last point

[ ] Keep an audit of flushes


cli: improve new stack command DX

Update DX and prompts in the nitric stack new command. Let's update it to match the new DX of the new project command, with clear prompts for each step and inline input validation to minimize errors causing a restart of the process.

last few things:

[ ] prevent the pulumi login state check, this isnt required for creating a stack file
[ ] create the vhs tapes

More context:
The current output is inconsistent, the output has mixed letter casing and while the provider says "which do you wish.." the region said "select" when they're both dropdown selections. I think we could make some small changes to improve this output.
Current:
nitric stack new
? What do you want to call your new stack? dev
? Which Cloud do you wish to deploy to? aws
? select the region us-east-1

Suggest updating to something consistent:
nitric stack new
? What do you want to name this stack? dev
? Which provider do you want to deploy with? aws
? Which region should the stack deploy to? us-east-1

or something concise:
nitric stack new
? Stack name: dev
? Provider: aws
? Region: us-east-1

Once we decide and implement this we should update relevant docs/examples where we can.

docs are here: https://github.com/nitrictech/docs/pull/488


community[feat]: Add Apache Opendal S3 document loader

Description: Add Apache Opendal S3 document loader

OpenDAL represents Open Data Access Layer. Our vision is to access data freely.It can serve as a unifying layer for object storage and file storage.I will improve other types of storage in subsequent PRs.
https://opendal.apache.org/

@baskaryan, @efriis, @eyurtsev, @ccurme, @vbarda, @hwchase17.

@baskaryan PTAL

hey there! sorry for the delay on this one. Not mergable as-is, and 1 question and 1 suggestion

question: what is the difference between this and an S3 loader?
suggestion: implement this as a BaseBlobLoader instead, which can be used with unstructured as a blobparser

let me know if that works for you!

hey there! sorry for the delay on this one. Not mergable as-is, and 1 question and 1 suggestion

question: what is the difference between this and an S3 loader?
suggestion: implement this as a BaseBlobLoader instead, which can be used with unstructured as a blobparser

let me know if that works for you!

Thank you for taking the time to review my PR.
OpenDAL is an Open Data Access Layer that enables seamless interaction with diverse storage services.OpenDAL is an Open Data Access Layer that enables seamless interaction with diverse storage services.
In fact, I want to use opendal to unify all my storage access.

sure but nothing is passed into Operator that's unrelated to unstructured and S3, so I'm curious what this does differently than just loading these files from S3 and passing them to unstructured
answer to the previous question was a bit vague, and without clarity on this I'll close the PR later this week. Let me know if there's something else that I'm missing!

sure but nothing is passed into Operator that's unrelated to unstructured and S3, so I'm curious what this does differently than just loading these files from S3 and passing them to unstructured
answer to the previous question was a bit vague, and without clarity on this I'll close the PR later this week. Let me know if there's something else that I'm missing!

Metrics, logging, tracing, can be added through opendal's layers.
https://opendal.apache.org/docs/vision#4-extensible-architecture
I don't know if this is an advantage or not.

Got it. This adds a net-new community integration or feature, which has been replaced by dedicated integration packages. I'll close this PR, and would recommend reopening with just docs updates, as well as registering your package in libs/packages.yml! We'll be able to review simple PRs that only modify these two things much faster :)
Here's the guide, and if you have questions, feel free to leave them in the comments on those pages so others can see them! https://python.langchain.com/docs/contributing/how_to/integrations/
Before doing so, I would highly recommend looking into implementing the BaseBlobLoader abstraction instead of requiring unstructured for pdf parsing. From what I understand, opendal should be used to load bytes, and there's no reason it should be tied to unstructured for parsing. Folks can use BaseBlobParser objects with the GenericLoader that pairs one of each.


Convert CocoaPods libraries to Frameworks

We should get some dogfooding done.

Moving to #455 & #429.


dpkg: fix the build on Darwin

Borrowed some hints from homebrew's formula.

build failure is unrelated to this change (it's devscripts - I'll fix that too)

This PR now supersedes #10935 as well

@benley: I tried to merge it, but it appears the packages have been updated to newer versions in the meantime.
I think the only useful bit remaining in this PR is this part from dpkg:
+     (stdenv.lib.optionalString stdenv.isDarwin "--disable-linker-optimisations")
+     (stdenv.lib.optionalString stdenv.isDarwin "--disable-start-stop-daemon")

(Although I don't know how important that part is.)
Please have a look and either rebase/close/open new PR.

Thanks for reminding me - I'll take another look at this PR and either update or close it.

@benley ping :)

revived! I hella forgot about this again.