Court field not appearing in Better BibTeX exports

[X] I have verified in the Zotero addons screen that I have the latest release of BBT (https://github.com/retorquere/zotero-better-bibtex/releases/latest) and Zotero installed
[X] I am posting a single question, bug or feature request
[X] I have included a descriptive subject of the problem
[X] I am available for follow-up questions and testing
[X] If I am posting an export or key-generation related bug report, I have included an error-report by selecting the problematic reference(s), right-clicking, and submitting an BBT error report from that popup menu -

Bug classification
Exporting references
Exporting BibTeX - Error ID = CA23UIRE-euc
Thank you for this great tool. I just started using Zotero for legal citations and have set it up to auto-export a .bib file with Better BibTeX syntax. However, not every field with data is being exported into the .bib file. In particular, Zotero's "Court" field for legal cases (which maps onto the CSL "authority" field) is not being exported into the .bib file.
Here is an example. This is a standard citation that appears when I drag-and-drop the item into this text box:
Lake River Corp. v. Carborundum Co., 769 F.2d 1284 (7th Cir. 1985). 7th Cir. is the court that decided the case.
Here is what the automatically exported .bib file shows for the item with the export format set to "Better BibTeX"; the "court" field is missing:
@misc{LakeRiverCorp1985,
  title = {Lake {{River Corp}}. v. {{Carborundum Co}}.},
  volume = {769},
  year = {1985},
  keywords = {liquidated damages},
  pages = {1284},
  note = {mlzsync1:0075\{"extrafields":\{"jurisdiction":"005us:c7United States|US|Seventh Circuit"\}\}}
}

What the item looks like in Zotero:

Is this a bug, or do I need to switch export formats? I want a simple, standard format that I can use in Emacs, Pandoc, and Sublime Text.
Thank you,
Ryan

Which field would you like it to show up in in the case of bibtex? For biblatex (which you get by exporting as "Better BibLaTeX" rather than "Better BibTeX"), it is exported as institution.
If your main target is Pandoc for anything but PDF output, I'd recommend "Better CSL" instead. Unless you use Pandoc to output PDF, it converts things back to CSL internally before it does anything else, and the conversion Zotero -> BibTeX -> Pandoc CSL is (necessarily) more lossy than just Zotero -> Pandoc CSL. BBT supports both pandoc-supported CSL formats (JSON and YAML).

Thank you for your response. I was hoping the BibTeX entries would show a "court" field. The BibTeX specification says that unknown tags (fields) "are ignored by BibTeX, thus can be used to store additional information without interfering with the final outcome of a document." With that in mind, I hoped that my automatic export would recreate all of the completed fields in the exported .bib file--including, for some legal cases, "court."
I am new to BibTeX so I could be wrong, but I assumed BibTeX was a standard and would be a good portable format for my citation library. But perhaps the Pandoc CSL is better. Since Zotero does not seem to have good bulk operations (adding or removing tags, changing item type, etc.), I would ideally like to be able to manipulate the parent citation items in a text editor, then reimport them into Zotero.
Thank you for your help.

Yeah but stuffing data in non-standard fields means you also need a custom bibtex style to use it at all, and I can't cater to every custom bibtex style.
If you can use a custom bibtex style, you can probably also use biblatex, and a Better BibLaTeX export will include the court. BibTeX should really only be used if your publisher forces you to -- it's really outdated at this point and superseded by biblatex.
Mind that bibtex (or biblatex) are really not good formats for round tripping data in Zotero. The export and import cannot roundtrip cleanly. Also, if I would stick the court into the court field, it would not be imported back because it's non-standard. Also, if you edit and re-import, you'll end up with duplicates. But if you're looking to do this anyhow, you may want to look at the "BetterBibTeX JSON" format. I use it internally for my tests but it gets as close to roundtripping cleanly as is possible in Zotero.
The CSL formats are great for Pandoc (unless you're using Pandoc to output PDF) but also don't roundtrip cleanly.

That is helpful, thank you. Keep up the great work.


template auto completion for customize view helper

hi,
if i have a customize view helper staticSite()
how to completion auto template like $this->paginationcontrol() in phtml files

IIRC IDEs pull those auto-completions from the docblock on the PhpRenderer (here).  You can't do the same for your custom helpers without hacking core, unfortunately.  If your IDE supports definiing custom typehinting maps you may be able to tell it to recognize PhpRenderer::staticSite as a known method.
Beyond that there's nothing we here can do to help.


Revise README after following for local execution

Includes the following:

Renames an osde2e flag that was updated recently (destroy-cluster)
Updates the steps to run locally to include passing secret location to allow test to pass

/lgtm
/approve


How to display the pointcloud?

when I run the "./demo [*.pcd]" in the directory of cuda-icp, just got the result:
GPU has cuda devices: 1
----device id: 0 info----
GPU : Xavier
Capbility: 7.2
Global memory: 31918MB
Const memory: 64KB
SM in a block: 48KB
warp size: 32
threads in a block: 1024
block dim: (1024,1024,64)
grid dim:<PHONE_NUMBER>,65535,65535)
Loaded 7000 data points for P with the following fields: x y z
Loaded 7000 data points for Q with the following fields: x y z
iter.Maxiterate 0
iter.threshold 1e-12
iter.acceptrate 1
Target rigid transformation : cloud_in -> cloud_icp
Rotation matrix :
| 0.923880 -0.382683 0.000000 |
R = | 0.382683 0.923880 0.000000 |
| 0.000000 0.000000 1.000000 |
Translation vector :
t = < 0.000000, 0.000000, 0.200000 >
------------checking CUDA ICP(GPU)----------------
CUDA ICP by Time: 0.797888 ms.
CUDA ICP fitness_score: 0.777453
matrix_icp calculated Matrix by Class ICP
Rotation matrix :
| 1.000000 0.000000 -0.000000 |
R = | -0.000000 1.000000 0.000000 |
| -0.000000 0.000000 1.000000 |
Translation vector :
t = < -0.000000, 0.000000, -0.000000 >
------------checking PCL ICP(CPU)----------------
PCL icp.align Time: 38.2758 ms.
has converged: 1 score: 0.651369
CUDA ICP fitness_score: 0.651369
transformation_matrix:
0.999905  0.00279406   0.0134922   0.0161865
-0.00265722    0.999945   -0.010151  0.00527596
-0.0135198   0.0101141    0.999858   0.0133578
0           0           0           1
------------checking PCL GICP(CPU)----------------
PCL Gicp.align Time: 144.663 ms.
has converged: 1 score: 0.541552
transformation_matrix:
0.99874  0.00468762   0.0499603  -0.0427716
-0.00344507    0.999683  -0.0249281   0.0265501
-0.0500613   0.0247246     0.99844    0.148036
0           0           0           1
so,I really want to know how to display the result,just like https://developer.nvidia.com/zh-cn/blog/cuda-pcl-1-0-jetson/

There will be three *.pcd: cuda.pcd, ICP.pcd, GICP.pcd in the same dir after "./demo".
You can use the command below to install "pcl_viewer" which can be used to show *.pcd
sudo apt-get install  pcl-tools

There will be three *.pcd: cuda.pcd, ICP.pcd, GICP.pcd in the same dir after "./demo".
You can use the command below to install "pcl_viewer" which can be used to show *.pcd
sudo apt-get install  pcl-tools

when I run the "./demo [*.pcd]" in the directory of cuda-icp, just got the result:
GPU has cuda devices: 1
----device id: 0 info----
GPU : Xavier
Capbility: 7.2
Global memory: 31918MB
Const memory: 64KB
SM in a block: 48KB
warp size: 32
threads in a block: 1024
block dim: (1024,1024,64)
grid dim:<PHONE_NUMBER>,65535,65535)
Loaded 7000 data points for P with the following fields: x y z
Loaded 7000 data points for Q with the following fields: x y z
iter.Maxiterate 0
iter.threshold 1e-12
iter.acceptrate 1
Target rigid transformation : cloud_in -> cloud_icp
Rotation matrix :
| 0.923880 -0.382683 0.000000 |
R = | 0.382683 0.923880 0.000000 |
| 0.000000 0.000000 1.000000 |
Translation vector :
t = < 0.000000, 0.000000, 0.200000 >
------------checking CUDA ICP(GPU)----------------
CUDA ICP by Time: 0.797888 ms.
CUDA ICP fitness_score: 0.777453
matrix_icp calculated Matrix by Class ICP
Rotation matrix :
| 1.000000 0.000000 -0.000000 |
R = | -0.000000 1.000000 0.000000 |
| -0.000000 0.000000 1.000000 |
Translation vector :
t = < -0.000000, 0.000000, -0.000000 >
------------checking PCL ICP(CPU)----------------
PCL icp.align Time: 38.2758 ms.
has converged: 1 score: 0.651369
CUDA ICP fitness_score: 0.651369
transformation_matrix:
0.999905 0.00279406 0.0134922 0.0161865
-0.00265722 0.999945 -0.010151 0.00527596
-0.0135198 0.0101141 0.999858 0.0133578
0 0 0 1
------------checking PCL GICP(CPU)----------------
PCL Gicp.align Time: 144.663 ms.
has converged: 1 score: 0.541552
transformation_matrix:
0.99874 0.00468762 0.0499603 -0.0427716
-0.00344507 0.999683 -0.0249281 0.0265501
-0.0500613 0.0247246 0.99844 0.148036
0 0 0 1
so,I really want to know how to display the result,just like https://developer.nvidia.com/zh-cn/blog/cuda-pcl-1-0-jetson/

There will be three *.pcd: cuda.pcd, ICP.pcd, GICP.pcd in the same dir after "./demo".
You can use the command below to install "pcl_viewer" which can be used to show *.pcd
sudo apt-get install pcl-tools

when I run the "./demo [*.pcd]" in the directory of cuda-icp, just got the result:
GPU has cuda devices: 1
----device id: 0 info----
GPU : Xavier
Capbility: 7.2
Global memory: 31918MB
Const memory: 64KB
SM in a block: 48KB
warp size: 32
threads in a block: 1024
block dim: (1024,1024,64)
grid dim:<PHONE_NUMBER>,65535,65535)
Loaded 7000 data points for P with the following fields: x y z
Loaded 7000 data points for Q with the following fields: x y z
iter.Maxiterate 0
iter.threshold 1e-12
iter.acceptrate 1
Target rigid transformation : cloud_in -> cloud_icp
Rotation matrix :
| 0.923880 -0.382683 0.000000 |
R = | 0.382683 0.923880 0.000000 |
| 0.000000 0.000000 1.000000 |
Translation vector :
t = < 0.000000, 0.000000, 0.200000 >
------------checking CUDA ICP(GPU)----------------
CUDA ICP by Time: 0.797888 ms.
CUDA ICP fitness_score: 0.777453
matrix_icp calculated Matrix by Class ICP
Rotation matrix :
| 1.000000 0.000000 -0.000000 |
R = | -0.000000 1.000000 0.000000 |
| -0.000000 0.000000 1.000000 |
Translation vector :
t = < -0.000000, 0.000000, -0.000000 >
------------checking PCL ICP(CPU)----------------
PCL icp.align Time: 38.2758 ms.
has converged: 1 score: 0.651369
CUDA ICP fitness_score: 0.651369
transformation_matrix:
0.999905 0.00279406 0.0134922 0.0161865
-0.00265722 0.999945 -0.010151 0.00527596
-0.0135198 0.0101141 0.999858 0.0133578
0 0 0 1
------------checking PCL GICP(CPU)----------------
PCL Gicp.align Time: 144.663 ms.
has converged: 1 score: 0.541552
transformation_matrix:
0.99874 0.00468762 0.0499603 -0.0427716
-0.00344507 0.999683 -0.0249281 0.0265501
-0.0500613 0.0247246 0.99844 0.148036
0 0 0 1
so,I really want to know how to display the result,just like https://developer.nvidia.com/zh-cn/blog/cuda-pcl-1-0-jetson/

There will be three *.pcd: cuda.pcd, ICP.pcd, GICP.pcd in the same dir after "./demo".
You can use the command below to install "pcl_viewer" which can be used to show *.pcd
sudo apt-get install pcl-tools


are there plans to support distributed scheduling?

No description provided.

Yes, while this lib is basically stable (probably v0.3.5), I will start developing a distributed scheduling system.
Containing a web admin panel, with multiple nodes distributed across the machines.

sounds good, what can i do that help you  reach stablity quickly ?

sounds good, what can i do that help you reach stablity quickly ?

#4 Thank you very much, I don't know how well you have mastered the asynchrony of rust, whether it is possible to encapsulate a state machine in order to make it easier for the user to dynamically cancel the running instances generated by the task through a handle, and expose the events that occur inside the specified task.
If you have some experience in this area, I can work with you to accomplish this. ：)
https://github.com/BinChengZhao/delay-timer/issues/4#issuecomment-767381469

trytry

trytry

// Here is an expected implementation for DelayTimer::add_task , which is not yet determined.
**Do you have a better opinion?**

///```
// let delay_timer = DelayTimer::default();
//
// let join_handle = delay_timer.add_task(_).unwrap();
//
// let peek : Option<Peek<&Instance>> = join_handle.peek().await;
// let peek : Result<Option<Peek<&Instance>>> join_handle.try_peek();
//
// let instance : Option<Intance> =  join_handle.next().await.unwrap();
// let instance : Result<Option<Intance>> =  join_handle.try_next().unwrap();
//
// instance.cancel();
//
///```

i spent some time on the delay_timer these days，now i hava general idea of it.

do you want use native async or build it yourself (e.g. executor、reactor etc.)

haha，holidy is coming soon   do it

i spent some time on the delay_timer these days，now i hava general idea of it.
i think you gave me a good plan.

Thank you, we can work together in one piece, so the energy will be great.

delicate  Distributed scheduling platform, the first release has been stabilized, V1.1.0 will be released soon.    :)


bug: VPC flow logs documentation link is broken

          ID aws-ec2-require-vpc-flow-logs-for-all-vpcs
      Impact Without VPC flow logs, you risk not having enough information about network traffic flow to investigate incidents or identify security issues.
  Resolution Enable flow logs for VPC

  More Information
  - https://aquasecurity.github.io/tfsec/v1.28.1/checks/aws/ec2/require-vpc-flow-logs-for-all-vpcs/

That link lands you on a 404 page.

The link is still broken

The link is still broken. Please remove stalebot.

Go away stalebot.

I also have same issue.

Same issue here, is this being looked at?

Probably not.

Our efforts have moved to Trivy and we now have AVD that documents all such misconfigurations in much more detail. You can find an example here https://avd.aquasec.com/misconfig/aws/ec2/vpc-flow-logs-enabled/


responses_to_action_page GET endpoint

This issue turned out to be not needed so we will repurpose it later.

Implemented as part of the demo on 4/22/2021.


Add README.md for Calendar cog

Is your feature request related to a problem? Please describe.
The Overlord Bot as a whole is not documented enough.
Describe the solution you'd like
Add a README.md file inside the folder for the cog when you get the chance.
Describe the cog's functions and how to use them.

README.md was made and resolved issue


View Character route and page

This page needs to display the character that was just created and include a back button to return to home page.

Taking this


e.preventDefault() doesn't work for 'select' event

Pretty easy to reproduce:
table.on('select', function (e, dt, type, indexes) {
    e.preventDefault();
});

My use case is that I need to prevent certain rows from being selectable. My work around has been to manually remove the selected class from each  that should be disabled after the select() event fires. The problem is that in the background DataTables will still display that a number of rows are selected when they really aren't.

You are correct - that event doesn't operate with preventDefault I'm afraid.
The way to stop the user from selecting rows is the user-select event.
Another option is to use the select.selector option to refine the selection to just the parts of the table you want.


Refactored server/grpc to use the sidecar runtime structure

Refactored slightly to separate the start/stop and the authorization and authentication
logic from the handlers.
Rewrote existing unit tests and added additional tests in server/grpc to improve code coverage.
Addresses https://github.com/kubernetes-csi/external-snapshot-metadata/issues/15.

/release-note-none

Improved unit test coverage of the server/grpc package:
Running tool: /usr/local/go/bin/go test -timeout 30s -coverprofile=/tmp/vscode-go49z0Hd/go-code-cover github.com/kubernetes-csi/external-snapshot-metadata/pkg/internal/server/grpc -v

=== RUN   TestNewServer
=== RUN   TestNewServer/sanity-tls-generator
=== RUN   TestNewServer/tls-load-error
=== RUN   TestNewServer/listen-error
E0725 00:57:11.617245 2384055 server.go:109] failed to start grpc server: listen tcp: address -1: invalid port
=== RUN   TestNewServer/start-stop
I0725 00:57:11.619816 2384055 server.go:116] GRPC server started listening on port 5001
I0725 00:57:11.631408 2384055 server.go:135] Starting a graceful shutdown on signal 'terminated' (15)
I0725 00:57:11.631696 2384055 server.go:139] Shutdown complete
--- PASS: TestNewServer (0.02s)
    --- PASS: TestNewServer/sanity-tls-generator (0.00s)
    --- PASS: TestNewServer/tls-load-error (0.00s)
    --- PASS: TestNewServer/listen-error (0.00s)
    --- PASS: TestNewServer/start-stop (0.01s)
=== RUN   TestSnapshotMetadata_GetMetadataDeltaInvalidRequest
--- PASS: TestSnapshotMetadata_GetMetadataDeltaInvalidRequest (0.00s)
=== RUN   TestSnapshotMetadata_GetMetadataAllocatedInvalidRequest
--- PASS: TestSnapshotMetadata_GetMetadataAllocatedInvalidRequest (0.00s)
PASS
coverage: 88.2% of statements
ok  	github.com/kubernetes-csi/external-snapshot-metadata/pkg/internal/server/grpc	0.042s	coverage: 88.2% of statements

/assign @PrasadG193

/lint

Latest push put copyrights into the new files.

Corrected handler test function names.

/approve

/lgtm


Projection Spike

This is here strictly to gather feedback and provide a discussion forum

Hey folks, I’m working a bit on the projections support today to try to get the event store half of things moving. For the first wave of things, I’m thinking that this is the functionality:

Start with .Net projections first, then go onto the equivalent functionality w/ PLV8 in the database
Support projections calculated either “Inline” during event capture, “Async” by some kind of async daemon that you would need to leave running in your application, or “Live” that’s can be calculated on the fly at any time from the latest data.
Leave partial document updates for a later phase
Support a single event to a single document projection (per Event)
Support an aggregation over all the events in a Stream. This would later get reused as the mechanism to create aggregate snapshots

For aggregates, we can do things in a couple different ways. We can:

Allow you to define and plug in a bunch of Func<TAggregate, TEvent, TAggregate> functions that would transform an aggregate model
Allow you to define a concrete type with Apply(Event1), Apply(Event2) methods that mutate itself based on event types. My thought here is to generate the Func’s from the preceeding item behind the scenes for this approach. I was iffy on this the first time I saw this pattern, but it's nice to have everything segregated.
We’ll have a high level aggregator interface you could use to do something completely different.
Dunno how this will go over, but we could also have an approach to say IEvent<TAggregate>, where that interface has a void Apply(TAggregate) function on it so that the event class itself would "know" how to transform the aggregate.

I'm not thrilled with (4) - events should be pocos with no behavior.

@jeffdoolittle I did that on a previous system. Makes the codebase very easy to extend because you don't need any kind of duplicated hierarchy of "one class for this, one other matching class for that."

of course I'm open to learning new things. :) but when I think of an event, I think of something you might be publishing to another service where having behavior on the event class would be odd/weird.

Yeah, not fond of 4 either. Might accuse you of staring at NServiceBus code for too long. :smile:

I prefer a dedicated class per Projection which and the output a separated POCO with several IHandleEvent. There is no need to encapsulate the data here so I prefer to have the responsibilities separated.

But I also used the self mutating projection on a project which also worked out fine. Maybe we can support both. In the end this is the same as the first option but only with return this; as output.

Is some situation it might be useful to have a projection that subscribes to all the events.

And I also find it useful to be able to restart a projection from scratch a. This will probably be difficult with the sycnprojections .

For event sourced aggregates I currently use something with AggregateEvent objects. I like that the publish and subscribe of an event are combined in a single object. The wiring is a bit more complicated but there is no need for a special base class of to inject a service to publish the events. (Gist: https://gist.github.com/tim-cools/694ba34dbb93b7a0cdf710886aa29275)

4 might work if you have one projection per event, but I guess it becomes messy when you have multiple projections per event. I prefer to group the handler per projection type instead of per event type.

To make the point clear that I didn't before, none of those options are mutually exclusive and everything would boil down to IAggregator.Apply(TAggregate, TEvent) behind the scenes.
I get the theoretical objections to "OMG, putting code into an Event class", but the reality is that that creates a very easily extensible mechanism for modeling state changes with fewer moving parts.

:scream: methods on events! ;)

Closing this as the work is already under way and usable.


PCUICWcbvEval.eval is incomplete?

Hi all,
After looking at the PCUICWcbvEval.eval relation I noticed the following:

it looks like cases for atoms (tFix, tContruct, tInd) are missing. These cases probably should be added in the same way as for WcbvEval for MetaCoq terms.

I don't know how to derive the following:

Inductive blah :=
  Blah : nat -> nat -> blah.

Compute ((fun x : nat => Blah x) 0 0).

In PCUIC terms it will look like this:
Module Pcbv := PCUICWcbvEval.
Notation "Σ ;;; Γ |- t1 ⇓ t2 " := (Pcbv.eval Σ Γ t1 t2) (at level 50).

Example PCUICWcbvEval_app Σ Γ :
  Σ ;;; Γ |-
mkApps (tLambda (BasicAst.nNamed "x") (tInd ty_blah []) (tApp (tConstruct ty_blah 0 []) (tRel 0))) [Zero;Zero] ⇓ mkApps (tConstruct ty_blah 0 []) [Zero;Zero].
Proof.
  eapply Pcbv.eval_app_constr.
  (* stuck! *)
Abort.

With this (sketch) of the evaluation relation it is possible to derive this example: https://gist.github.com/annenkov/5b2af547fb7ee018dc18161068071dc7

You're absolutely right, the wcbvEval relations are all wrong! We're working on them right now. I think we will go with a version closer to CertiCoq where for applications we have:
| wAppLam: forall (fn bod a1 a1' s:Term) (nm:name),
    WcbvEval p fn (TLambda nm bod) ->
    WcbvEval p a1 a1' ->
    WcbvEval p (whBetaStep bod a1') s ->
    WcbvEval p (TApp fn a1) s
| wAppFix mfix idx args args' narg fn res :
      unfold_fix mfix idx = Some (narg, fn) ->
      All2 eval args args' ->
      is_constructor narg args' = true ->
      eval (mkApps fn args') res ->
      eval (mkApps (tFix mfix idx) args) res
| wAppFixStuck: forall dts m n (fn arg s x:Term),
      unfold_fix mfix idx = Some (narg, fn) ->
      All2 eval args args' ->
      is_constructor narg args' = false ->
      eval (mkApps (tFix mfix idx) args) (mkApps (tFix mfix idx) args')
| wAppCong: forall fn fn' arg arg',
    WcbvEval p fn fn' ->
    (~ isLambda fn' /\ ~ isFix fn') ->
    WcbvEval p arg arg' ->
    WcbvEval p (TApp fn arg) (TApp fn' arg')

Then atoms reduce to themselves and application congruence can be applied to them, as well as in the case where the head of an application reduces to an atom, as you propose. Do you think that version will be equivalent to the one you have, or a simple extension of it? I'm a bit unsure about the stuck fixpoint case, but I believe we should have a rule allowing to produce stuck fixpoint applications as values, for separate compilation in particular.
The  (~ isLambda fn' /\ ~ isFix fn') ensures that we do not miss redexes at the head of the term, but that we can accumulate evaluated arguments on constructors, inductive types, evars and (non-let-bound) variables.

CC @yforster this will impact extraction too

@mattam82 ah, great! This version of relation indeed looks close to what I had in mind. I think it should be equivalent to my attempt (but without converting values back to terms, as I have in my version). Evaluation of application in the wAppCong rule now follows the syntactic structure of the term, no mkApps magic involved, I like it :) Looking forward to having this relation in MetaCoq!
The wAppFix might again cause problems when inverting it, but, I guess, it is necessary to have mkApps there because fixpoints might be defined by recursion on nth argument (but if we allow only recursion on the first argument, it could be simplified as well).
I don't think I have anything analogous to wAppFixStuck. I guess, the distinction is in is_constructor narg args' = false, so it probably will be ruled out by the guard condition in my case.

Indeed, the wAppFixStuck case is delicate. In the case of MetaCoq, the evaluation rule disappears by extraction time and is replaced by an unrestricted wAppFix evaluation rule for the extracted code: we know by typing that when we evaluate a closed term of inductive type containing a fixpoint that the fixpoint must be fully applied (to produce a term in the inductive type) and that its recursive argument must evaluate to a constructor (or a Box) as this argument is a closed term of inductive type itself.

For the time being, I only updated the evaluation for erasure here: https://github.com/MetaCoq/metacoq/blob/ce689415dbbde05b203a6d018b84b1fae51f2b31/extraction/theories/EWcbvEval.v in PR #239 . Please have a look and tell me if somethings wrong from your side. I'll pull back these changes to PCUIC and template-coq's wcbveval definitions next.

I think it looks good. I guess mkApps will creep back in the tFix case in PCUIC eval, because we need to check the guard condition. If so, wouldn't it then overlap with the eval_app_cong rule?

Yes, we will need an mkApps there in case there is at least one argument, but it should only be a matter of an inversion proof to show that it will be the only rule to evaluate fixpoints. Maybe we can just forget about suspended fixpoints whose structural argument does not evaluate to a constructor altogether to avoid an apparent critical pair. One will not be able to start an application congruence with a Lambda or Fix at the head, still. And for our purposes in CertiCoq we don’t need that I think. I’m more worried for you and other users who will want to produce WcbvEval derivations in PCUIC, wouldn’t you need that? Nothing prevents a user from writing a program with a partially applied fix somewhere.

Could you please clarify what is a fully applied fixpoint? For example, for the usual definition of addition, add 1 1 is fully applied, right? But what if the fixpoint returns a lambda:
Definition add' :=
  fix add (n : nat) {struct n} : nat -> nat :=
    match n with
    | 0 => fun m => m
    | S p => fun m => S (add p m)
    end.

What would be a fully applied fix in this case?

I mean you can have add : nat -> nat -> nat defined by recursion on the 2nd argument and see add 1 in your program, that's a partially applied fixpoint.

Ok, that's what I thought. So, basically, a fully applied fix is a fix applied up to its recursive argument.
In my case this should not be a problem, since I only have one argument fix. So, it's either fully applied, or not applied at all. In both cases it is possible to build a derivation.

Cool! I'll get on pulling things back then.

@mattam82 just an after-thought: maybe it's worth adding an explicit constraint on the length of the arguments in the rule for fix application? So that #|args| = narg. Otherwise, it would be possible to apply wAppFix to add' n m' where add' is from my example above, or it would be also possible to first apply 'eval_beta` rule, which give us an overlap of the two rules.

Indeed, you are right, it's an unnecessary overlap.

It seems that the case for tInd is missing from the evaluation relation. I guess, tInd should be an atom as well.

Right

Great! Thanks, @mattam82


misc changes

add log output to console for config and admin service
remove dev, lpt, tooling logic
add default 2 seconds of initial delay to long polling

Coverage decreased (-0.07%) to 50.032% when pulling a99894a7454430b3b5a683b0a9217a4cd6916c74 on nobodyiam:misc-change-merge into 58bd26e300a0bacd5941b7ee4bddb3c55b1edfad on ctripcorp:master.

Codecov Report

Merging #638 into master will decrease coverage by 0.03%.
The diff coverage is 31.25%.

@@             Coverage Diff             @@
##             master    #638      +/-   ##
===========================================
- Coverage     46.54%   46.5%   -0.04%
- Complexity     1471    1472       +1
===========================================
  Files           342     342
  Lines          9381    9390       +9
  Branches        927     926       -1
===========================================
+ Hits           4366    4367       +1
- Misses         4682    4693      +11
+ Partials        333     330       -3

Impacted Files
Coverage Δ
Complexity Δ

.../apollo/internals/RemoteConfigLongPollService.java
78.37% <16.66%> (-2.61%)
23 <0> (ø)

...va/com/ctrip/framework/apollo/util/ConfigUtil.java
46.49% <40%> (+0.54%)
22 <1> (ø)
:arrow_down:

...work/apollo/biz/message/DatabaseMessageSender.java
44.68% <0%> (-6.39%)
6% <0%> (ø)

...ervice/service/ReleaseMessageServiceWithCache.java
86.9% <0%> (+1.19%)
25% <0%> (+1%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 58bd26e...a99894a. Read the comment docs.

Coverage increased (+0.3%) to 50.415% when pulling f30ab9c98ea743ac92097de6ec84bf57af4ac703 on nobodyiam:misc-change-merge into 58bd26e300a0bacd5941b7ee4bddb3c55b1edfad on ctripcorp:master.


Enhancement and bugfix of fragment related stuffs.

Maintain orders of labels as user specified.
Bugfix of AddEdges, do property reassignment of the offset array.
Validate the schema that properties with the same name must have the same type.
Add primary keys when necesssary.

Fixes #297.
/hold until the CI in GraphScope turned green.

Please sign-off you commit @siyuan0322.


Inferred files from VS Code propagate through intellisense.

Issue Description
Hi all, happy new year. I was hitting a strange issue today and it had a very funny resolution. Basically, I forgot to add .cs to one of my files after renaming. What appears to have happened was that the extensionless file was being inferred as a csharp file by VS Code's intellisense, and its namespace and other metadata were loaded into intellisense. When I opened other files which referenced that file (which was technically not a csharp file), they appeared to work correctly and performing actions like "Go to Definition" went to the extensionless file.
However, there were errors on Build because the project was unable to find a csharp file that had the code that intellisense was picking up. After taking a break and eating lunch, I came back and easily noticed the missing extension, but VS code made it a little harder to detect by auto-inferring the C#-ness of the extensionless file and making it appear normal.
This is a bit of Quality of Life more than a bug, but it might be nice to avoid combining inferred code into the intellisense of real code. Or at least display a warning like "This appears to be a csharp file but will not work in the project due to lack of a .cs extension."
Steps to Reproduce

Create a project
Remove CSharp extension from a file.
Go to another file referencing code in that file. There will be no errors / missing references.
Receive an error on build, missing references.

Expected Behavior
Print errors that references are missing, or warn that the file cannot be referenced by a project when it has an incorrect extension.
Actual Behavior
Intellisense does not warn or error, but build produces an error.
Logs
OmniSharp log
Post the output from Output-->OmniSharp log here
C# log
Post the output from Output-->C# here
Environment information
VSCode version: 1.63.2
C# Extension: 1.23.17
Dotnet Information
.NET SDK (reflecting any global.json):
 Version:   6.0.101
 Commit:    ef49f6213a
Runtime Environment:
OS Name:     Windows
OS Version:  10.0.19042
OS Platform: Windows
RID:         win10-x64
Base Path:   C:\Program Files\dotnet\sdk\6.0.101\
Host (useful for support):
Version: 6.0.1
Commit:  3a25a7f1cc
.NET SDKs installed:
2.2.402 [C:\Program Files\dotnet\sdk]
3.0.103 [C:\Program Files\dotnet\sdk]
3.1.416 [C:\Program Files\dotnet\sdk]
5.0.404 [C:\Program Files\dotnet\sdk]
6.0.101 [C:\Program Files\dotnet\sdk]
.NET runtimes installed:
Microsoft.AspNetCore.All 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.0.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.22 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.22 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.0.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.22 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.13 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
To install additional .NET runtimes or SDKs:
https://aka.ms/dotnet-download

Visual Studio Code Extensions

Extension
Author
Version

csharp
ms-dotnettools
1.23.17

dotnet-test-explorer
formulahendry
0.7.7

mdx
silvenon
0.1.0

php-debug
felixfbecker
1.22.0

phpcs
ikappas
1.0.5

powershell
ms-vscode
2021.12.0

prettier-vscode
esbenp
9.1.0

vscode-intelephense-client
bmewburn
1.8.0

vscode-mdx-preview
xyc
0.3.3

A lot has changed with O# code analysis since this issue was opened. If you are still having problems, please open a new issue at https://github.com/omnisharp/omnisharp-roslyn/issues

Thanks, I don't see this issue anymore.


Internet header: add dark mode styling

For now the internet header remains white on dark mode

Closing as out of scope


nan in loss with those molecules

basically due to the     features.GasteigerCharge() that can be null few molecules with like whit  "P" atoms will produce nans during training.
Can you replace the nan features for those molecules by 0.0 in such case ?

This is the current implementation:
@atom_features.register(name='gasteiger_charge')
class GasteigerCharge(Feature):
    def __call__(self, atom: Chem.Atom) -> float:
        mol = atom.GetOwningMol()
        rdPartialCharges.ComputeGasteigerCharges(mol)
        val = atom.GetDoubleProp('_GasteigerCharge')
        return val if val is not None else 0.0

Isn't it already taking care of it? Can you give me an example molecule that causes the issue and I'll look into it.

this is a molecule that failed : ['OCPH(CO)CO']
I add GasteigerCharges and it goes to nan I remove it and it works. I think the vales are not None but Nan.

smiles = 'OCPH(CO)CO'
mol = molgraph.chemistry.molecule_from_string(smiles)

ValueError: Inputted `molecule` (None) is invalid

I fix the smiles in github issue page see upper


Add a rowIndex prop for keyboard navigation

I have a requirement that the tree should support keyboard navigation. But with scrollToId is not enough to do it.
There is a call back function named onRowsRendered in react-virtualized and it can be used to fetch the startIndex and stopIndex.  Reference link: https://github.com/bvaughn/react-virtualized/issues/581
Wrapper component can pass a ref to get these 2 properties.
Looking forward for feedbacks.
I'll use the forked repo from my own temporarily.
And sorry for the version change in the package.json. That can be ignored.

Coverage decreased (-0.6%) to 93.93% when pulling 7fdae6c211445b7ac2e106e369a7af44ea837b2d on RyanZhu1024:master into dfaaf7402d56a3eaf1034fbebfc86770d41f3700 on diogofcunha:master.


Cocreation Editors add Filename

Describe the bug
It's unclear what file is opened in the editors (form editor, process modeler)
Expected behavior
Show the filename somewhere in the editors
See https://git.muenchen.de/digitalisierung/digiwf-support/-/issues/14

Done


fix: add og:type for twitter card generation

Can't check on review app because of robot.txt but I think it should do it :crossed_fingers:


beta-estradiol 3-sulfate

beta-estradiol 3-sulfate
known to KEGG
http://www.genome.ad.jp/dbget-bin/www\_bget?cpd:C08357
Reported by: deustp01

thanks www.movied.org


Problems with inserted semicolons in generated JS

This Roy code:
let f x =
  let a = 2
  a * x

console.log (f 4)

Generates this JS code:
var f = function(x) {
    console.log(1);
    var a = 2;;
    return a * x;
}
console.log((f(4)))

The double semicolons are just an aesthetic problem, but the lack of semicolons in single-line expressions can lead to syntactically incorrect JS.

Pushed a change. The input now compiles to this:
var f = function(x) {
    var a = 2;
    return a * x;
};
console.log((f(4)));

swell, thanks.


update to Rust master

Vec::{remove,swap_remove} now return T instead of Option<T>
Vec::from_fn was deprecated

Thanks!

Thanks!

That Option change flew right under my radar, I think I'll want to discuss that with aturon...

That Option change flew right under my radar, I think I'll want to discuss that with aturon...


Grab the envVars from the run

@ydubreuil I don't have an instance of Jenkins to test on, will this work though to just pass on the variables from the trigger?

@ydubreuil check my updates to your comment


Uvcvideo patching issue

Required Info

Camera Model
SR300

Operating System & Version
Ubuntu 16.04

Kernel Version (Linux Only)
4.8.0-36-generic

Build System
VS2013, XCode or Makefile

I installed Ubuntu on my mac as a dual system. When I run the script ./scripts/patch-realsense-ubuntu-xenial.sh I got the error :
modprobe: ERROR: could not insert 'uvcvideo': Unknown symbol in module, or unknown parameter (see dmesg)
Failed to insert the patched module. Operation is aborted, the original module is restored
Verify that the current kernel version is aligned to the patched module version
dmesg : [ 3233.439976] uvcvideo: Unknown symbol media_entity_create_link (err 0)
[ 3233.440057] uvcvideo: Unknown symbol media_entity_init (err 0)
[ 3233.440100] uvcvideo: Unknown symbol media_entity_cleanup (err 0)
I got the same error when I tried to install it on a windows/Ubuntu dual system machine. I solved it by disable secure boost with command below:
sudo apt install mokutil
sudo mokutil --disable-validation
It worked. However, it does not work on my mac. When I tried to run the commands. It says:
This system does't support Secure Boot
What should I do to solve this ?
Thank you!

Hi, you've got Ubuntu LTS with kernel 4.8, probably '16.04.2' that was released this month.
The patch, however,  support LTS kernels 4.4.0-xx only (4.4.0-63 the latest) .
Support for kernel 4.8 is on the TODO list, but at this point you should either downgrade Ubuntu to kernel 4.4, or modify the patches on your own to make this work.

Is there an ETA for kernel 4.8 support?

I'm also interested in ETA for 4.8 support. Will downgrade kernel until then.

We have tried downgrading to 4.4 on an Intel Aero board but do unfortunately not have any success with enumerating the camera.
Looking in dmesg shows that the USB 3.0 port is started up and detected but fails afterwards why the port become "deregistered".
[    7.013643] xhci-hcd xhci-hcd.3.auto: xHCI Host Controller
[    7.013662] xhci-hcd xhci-hcd.3.auto: new USB bus registered, assigned bus number 3
[    7.048224] xhci-hcd xhci-hcd.3.auto: Host not halted after 16000 microseconds.
[    7.048233] xhci-hcd xhci-hcd.3.auto: can't setup: -110
[    7.048315] xhci-hcd xhci-hcd.3.auto: USB bus 3 deregistered
[    7.048349] xhci-hcd: probe of xhci-hcd.3.auto failed with error -110

Any idea?

Requesting update! 16.04.2 is the mainline LTS released version.  Running 14.04 doesn't work for many robotics applications.
We mark realsense as non-op/no-use until this is fixed.

Someone found the way to fix this issue yet?

I have the same problem with uvcvideo for Ubuntu 16.04lts, kernel 4.7. Is the proposal "disable secure boost" the final official solution? Can anyone confirm it?

I have modified the script to support new kernel(4.8.0-xx).
link: https://github.com/JJHu1993/librealsense-patch.git
Patched kernel module created successfully

Unloading module uvcvideo
Applying patched module uvcvideo
Inserting uvcvideo succeeded

Script has completed successfully. Please consult the installation guide for further instruction.

jjhu@jjhu-computer:~/projects/librealsense$ uname -r
4.8.0-52-generic

@Ryanliyikun @hiptechboy @jamessergeant @mindThomas @tdnet12434 @Thanh-Binh

Hello @JJHu1993 please commit the file patch-utils.sh . Thanks!

Hello @JJHu1993 ,
Thanks a lot for working on the patch.
Could you please push the file patch-utils.sh as well? Thanks again.

@thesidjway You can replace that patch to old one in https://github.com/IntelRealSense/librealsense/blob/master/scripts/patch-realsense-ubuntu-xenial.sh
Thank you @JJHu1993 it worked for me :)

@fxprime Thanks a lot!

Hmm tried this, the script said it succeeded,
but the result of
sudo dmesg | tail -n 50
results in:
[ 6861.673072] uvcvideo: Unknown symbol media_entity_create_link (err 0)
[ 6861.673128] uvcvideo: Unknown symbol media_entity_init (err 0)
[ 6861.673162] uvcvideo: Unknown symbol media_entity_cleanup (err 0)
[ 8490.871704] uvcvideo: Unknown symbol media_entity_create_link (err 0)
[ 8490.871773] uvcvideo: Unknown symbol media_entity_init (err 0)
[ 8490.871806] uvcvideo: Unknown symbol media_entity_cleanup (err 0)
[ 8919.318059] uvcvideo: Unknown symbol media_entity_create_link (err 0)
[ 8919.318201] uvcvideo: Unknown symbol media_entity_init (err 0)
[ 8919.318285] uvcvideo: Unknown symbol media_entity_cleanup (err 0)
for the uvcvideo module
I guess it didn't work, even though the script succeeded?

It worked for me and people around me whose kernel version is 4.8.xx
发自我的 iPhone

在 2017年6月4日，上午11:29，Ender1618<EMAIL_ADDRESS>写道：
Hmm tried this, the script said it succeeded,
but the result of
sudo dmesg | tail -n 50
results in:
[ 6861.673072] uvcvideo: Unknown symbol media_entity_create_link (err 0)
[ 6861.673128] uvcvideo: Unknown symbol media_entity_init (err 0)
[ 6861.673162] uvcvideo: Unknown symbol media_entity_cleanup (err 0)
[ 8490.871704] uvcvideo: Unknown symbol media_entity_create_link (err 0)
[ 8490.871773] uvcvideo: Unknown symbol media_entity_init (err 0)
[ 8490.871806] uvcvideo: Unknown symbol media_entity_cleanup (err 0)
[ 8919.318059] uvcvideo: Unknown symbol media_entity_create_link (err 0)
[ 8919.318201] uvcvideo: Unknown symbol media_entity_init (err 0)
[ 8919.318285] uvcvideo: Unknown symbol media_entity_cleanup (err 0)
for the uvcvideo module
I guess it didn't work, even though the script succeeded?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

It worked for me and people around me whose kernel version is 4.8.xx
see pictures
result

@Ender1618

Hello,
I have an Intel Aero setup with the vision accessory kit. I'm trying to run the camera on ROS (Ubuntu 16.04, Linux 4.4.0-78). For me, the 4.4 patch finishes successfully but neither am I able to see a /dev/video device connected to my system, nor do any of the example codes in the librealsense directory work. Has anyone of you had success using the realsense vision accessory kit with Ubuntu?
https://software.intel.com/en-us/aero/vision-kit
@JJHu1993 @fxprime @ev-mp @mindThomas

Hi, is the R200 properly connected ? There are no UVC-related info in the screenshot.
Additionally, please specify "lsusb" log

@JJHu1993 - I just merged your revised patch script scripts/patch-realsense-ubuntu-xenial.sh to the librealsense repository locally on my machine. It worked! (just run an example cpp tutorial 2 stream script with the R200 plugged in.)
Log:
Kernal 4.8.x...
johnny@johnny-XPS-8700:~/repos/librealsense$ uname -r
4.8.0-54-generic

last few lines says new driver registered...
johnny@johnny-XPS-8700:~/repos/librealsense$ sudo dmesg | tail -n 50
.... (bla bla bla) ...
[  484.018926] media: Linux media interface: v0.10
[  484.038903] Linux video capture interface: v2.00
[  484.061164] usbcore: registered new interface driver uvcvideo
[  484.061165] USB Video Class driver (1.1.1)

Plugged the R200 camera in and run an example stream script - worked! :)
johnny@johnny-XPS-8700:/usr/local/bin$ ./cpp-tutorial-2-streams
There are 1 connected RealSense devices.

Using device 0, an Intel RealSense R200
    Serial number:<PHONE_NUMBER>
    Firmware version: <IP_ADDRESS>

(Here is the revised librealsense branch with your revised script - https://github.com/Atlas7/librealsense/tree/patch-JJHu1993/scripts)
I guess you are good to go raising the pull request to merge to master?
Thanks a lot again!
Cheers
Johnny

Thanks for notice. I will write a script to support both kernel versions and pull it to master.

在 2017年6月13日，上午4:26，Johnny Chan<EMAIL_ADDRESS>写道：
@JJHu1993 - I just merged your revised patch script scripts/patch-realsense-ubuntu-xenial.sh to the librealsense repository locally on my machine. It worked! (just run an example cpp tutorial 2 stream script with the R200 plugged in.)
Log:
Kernal 4.8.x...
johnny@johnny-XPS-8700:~/repos/librealsense$ uname -r
4.8.0-54-generic
last few lines says new driver registered...
johnny@johnny-XPS-8700:~/repos/librealsense$ sudo dmesg | tail -n 50
.... (bla bla bla) ...
[  484.018926] media: Linux media interface: v0.10
[  484.038903] Linux video capture interface: v2.00
[  484.061164] usbcore: registered new interface driver uvcvideo
[  484.061165] USB Video Class driver (1.1.1)
Plugged the R200 camera in and run an example stream script - worked! :)
johnny@johnny-XPS-8700:/usr/local/bin$ ./cpp-tutorial-2-streams
There are 1 connected RealSense devices.
Using device 0, an Intel RealSense R200
Serial number:<PHONE_NUMBER>
Firmware version: <IP_ADDRESS>
(Here is the revised librealsense branch with your revised script - https://github.com/Atlas7/librealsense/tree/patch-JJHu1993/scripts)
I guess you are good to go raising the pull request to merge to master?
Thanks a lot again!
Cheers
Johnny
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

So it's not clear from closing comment...   as of 7/5/2017 has this been pulled into master? or the best place to get this (working) is from @Atlas7's repo?

As of Aug 10th, the librealsense patch is updated to support Ubuntu LTS 16.04.1 and 16.04.2 with kernels  4.4, 4.8* and 4.10.
*Starting from Aug 2nd, Ubuntu 16.04.2 users will receive kernel 4.10 version on their next system upgrade. After the upgrade it is important to reapply the UVC driver patch as kernel 4.10 implements a more strict policy on handling UN-authorized kernel modules


Missing Windows support

Currently, the library uses precompiled Linux binaries. I am unsure how compatible these are with standard PyTorch installs on Windows. It might be that the binaries need to be compiled against mingw32/64 to create functional binaries for Windows.
The most helpful would be a case where a person is able to compile from source and use the library. This will require altering the Makefile file. If this works, we can add instructions on compiling for Windows as a first step before doing a full-scale Windows deployment of binaries on pip.
Since I do not have a Windows machine, any help is wanted on this!

I am able to compile the csrc part on Windows but fail to link (Visual Studio)
FYI, this is the error message. I'm no expert on cuda, so I'm not quite sure where it goes wrong.

To successfully compile on windows MSVC, some parts need to be patched as following:

rename pythonInterface.c to pythonInterface.cpp, or visual studio will try using a C compiler for it.

add one missing template instantiation like this: (in SIMD.h)

get unistd.h and getopt.h for windows

get pthread for windows

finally, this is just a build test, so I'm not using the Makefile.
That's all for now.
Tested on CUDA toolkit 11.6, windows 11

After some really tedious debugging and tackling various hidden problems, I managed to successfully compile the whole module.
This is the end result:
https://github.com/DeXtmL/bitsandbytes-win-prebuilt
The binaries are compiled against CUDAToolkit 11.6 and Visual Studio 2022.
I am able to make inferences nearly identical to "normal" fp16 version. So this is kind of a confirmation for "it works" shout-out. No vigorous testing was conducted though. @TimDettmers
Finally, the "cuda_setup" part of the source code is entirely incompatible with Windows, there are loads of hardcoded routines; so I used a quick makeshift patch instead of making it proper, that's also why I'm not posting my changes or making PR for now. If you are eager to test:
in cuda_setup/main.py:
make evaluate_cuda_setup() always return "libbitsandbytes_cuda116.dll"
in ./cextension.py:
change ct.cdll.LoadLibrary(binary_path) to ct.cdll.LoadLibrary(str(binary_path))

That should do the trick.
Hopefully this can help someone in the Windows territory; let's hope the official windows support come fast.

Thank you. This might be the key.

After some really tedious debugging and tackling various hidden problems, I managed to compile the whole module. This is the end result: https://github.com/DeXtmL/bitsandbytes-win-prebuilt
The binaries are compiled against CUDAToolkit 11.6 and Visual Studio 2022. I am able to make inferences nearly identical to "normal" fp16 version. So this is kind of a confirmation for "it works" shout-out. No vigorous testing was conducted though. @TimDettmers Finally, the "cuda_setup" part of the source code is entirely incompatible with Windows, there are loads of hardcoded routines; so I used a quick makeshift patch instead of making it proper, that's also why I'm not posting my changes or making PR for now. If you are eager to test:
in cuda_setup/main.py:
make evaluate_cuda_setup() always return "libbitsandbytes_cuda116.dll"
in ./cextension.py:
change ct.cdll.LoadLibrary(binary_path) to ct.cdll.LoadLibrary(str(binary_path))

That should do the trick.
Hopefully this can help someone in the Windows territory; let's hope the official windows support come fast.

could you provide a makefile for this?

After some really tedious debugging and tackling various hidden problems, I managed to compile the whole module. This is the end result: https://github.com/DeXtmL/bitsandbytes-win-prebuilt
The binaries are compiled against CUDAToolkit 11.6 and Visual Studio 2022. I am able to make inferences nearly identical to "normal" fp16 version. So this is kind of a confirmation for "it works" shout-out. No vigorous testing was conducted though. @TimDettmers Finally, the "cuda_setup" part of the source code is entirely incompatible with Windows, there are loads of hardcoded routines; so I used a quick makeshift patch instead of making it proper, that's also why I'm not posting my changes or making PR for now. If you are eager to test:
in cuda_setup/main.py:
make evaluate_cuda_setup() always return "libbitsandbytes_cuda116.dll"
in ./cextension.py:
change ct.cdll.LoadLibrary(binary_path) to ct.cdll.LoadLibrary(str(binary_path))

That should do the trick.
Hopefully this can help someone in the Windows territory; let's hope the official windows support come fast.

Where do you put the pre-built file to activate adam?

After some really tedious debugging and tackling various hidden problems, I managed to compile the whole module. This is the end result: https://github.com/DeXtmL/bitsandbytes-win-prebuilt
The binaries are compiled against CUDAToolkit 11.6 and Visual Studio 2022. I am able to make inferences nearly identical to "normal" fp16 version. So this is kind of a confirmation for "it works" shout-out. No vigorous testing was conducted though. @TimDettmers Finally, the "cuda_setup" part of the source code is entirely incompatible with Windows, there are loads of hardcoded routines; so I used a quick makeshift patch instead of making it proper, that's also why I'm not posting my changes or making PR for now. If you are eager to test:
in cuda_setup/main.py:
make evaluate_cuda_setup() always return "libbitsandbytes_cuda116.dll"
in ./cextension.py:
change ct.cdll.LoadLibrary(binary_path) to ct.cdll.LoadLibrary(str(binary_path))

That should do the trick.
Hopefully this can help someone in the Windows territory; let's hope the official windows support come fast.

An easy way to always return libbitsandbytes_cuda116.dll would be to insert
if torch.cuda.is_available(): return 'libbitsandbytes_cuda116.dll', None, None, None, None
under
119: if not torch.cuda.is_available(): return 'libsbitsandbytes_cpu.so', None, None, None, None

After some really tedious debugging and tackling various hidden problems, I managed to compile the whole module. This is the end result: https://github.com/DeXtmL/bitsandbytes-win-prebuilt
The binaries are compiled against CUDAToolkit 11.6 and Visual Studio 2022. I am able to make inferences nearly identical to "normal" fp16 version. So this is kind of a confirmation for "it works" shout-out. No vigorous testing was conducted though. @TimDettmers Finally, the "cuda_setup" part of the source code is entirely incompatible with Windows, there are loads of hardcoded routines; so I used a quick makeshift patch instead of making it proper, that's also why I'm not posting my changes or making PR for now. If you are eager to test:
in cuda_setup/main.py:
make evaluate_cuda_setup() always return "libbitsandbytes_cuda116.dll"
in ./cextension.py:
change ct.cdll.LoadLibrary(binary_path) to ct.cdll.LoadLibrary(str(binary_path))

That should do the trick.
Hopefully this can help someone in the Windows territory; let's hope the official windows support come fast.

Where do you put the pre-built file to activate adam?

You put them in site-packages\bitsandbytes

Appreciate you doing this work, helped unblock me in a big way. I hope bitsandbytes supports Windows directly sooner rather than later, but this is a great stopgap.

@PinPointPing Thanks a lot, it just worked like a charm

Same issue for me, if someone can compile cuda 11.8 binaries for me i can test them :)
using StableDiffusion-WebUI + Dreambooth, would love to give Adam a spin !
btw under Windows, the environment variable is CUDA_PATH and CUDA_PATH_V11_8
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
For effortless bug reporting copy-paste your error into this form: https://docs.google.com/forms/d/e/1FAIpQLScPB8emS3Thkp66nvqwmjTEgxp8Y9ufuWTzFyr9kJ5AoI47dQ/viewform?usp=sf_link
================================================================================
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)!
CUDA SETUP: Loading binary G:\Visions of Chaos\MachineLearning\Text To Image\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so...
Exception importing 8bit adam: argument of type 'WindowsPath' is not iterable
 Scheduler Loaded
 Allocated: 2.3GB
 Reserved: 2.4GB

after following some advice and making the edits in this thread, i got Adam to run on my CUDA 11.8 setup/ it does indeed work for me :)

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
For effortless bug reporting copy-paste your error into this form: https://docs.google.com/forms/d/e/1FAIpQLScPB8emS3Thkp66nvqwmjTEgxp8Y9ufuWTzFyr9kJ5AoI47dQ/viewform?usp=sf_link
================================================================================
CUDA SETUP: Loading binary G:\Visions of Chaos\MachineLearning\Text To Image\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda116.dll...
 Scheduler Loaded
 Allocated: 0.3GB
 Reserved: 0.4GB

would you mind sharing your cuda 11.8 binary

would you mind sharing your cuda 11.8 binary

using the libs cuda 11.6 from [(https://github.com/DeXtmL/bitsandbytes-win-prebuilt)]

@TimDettmers any updates on this ? 😄

I just got this building with cmake.
First thing I did was make a directory called dependencies , then download https://github.com/GerHobbelt/pthread-win32 and extract the whole thing there so it's /project_root/dependencies/pthread-win32-main/pthread.h
(I lied, I didn't really do that first, did that last but I suggest anyone following this do it first to avoid the error)
I did apply the patch above by @DeXtmL (Does not compile without it, missing vec_t error).
As for unistd.h/getopt, I literally just commented out the #include <unistd.h> in the file it gave the error on and it now fully compiles so not sure anything relies on unistd.h (At least in windows with VC2019.. ), we'll see when I go to test the .dll (Didn't give any errors about getopt)
CMakeLists.txt
CMakeLists.txt
The first CMakeLists.txt is in the root, the second in csrc.
I do:
cd /project_root
mkdir build
cd build
cmake ..

Then can open the .sln, right click bitsandbytes hit build and it goes to town.
I imagine for linux the same cmake file would actually work just fine. Only thing I did special for windows was add the include path for the pthreads, adding an include path to a folder that doesn't exist probably won't hurt right?
GL

Quick final follow up, it built fine in Debug with the above, i actually don't know quite how. For release mode I did actually have to build the pthread library (just another mkdir build, cmake .., open solution, build all in release mode), then slightly modify the cmake file.
I don't know why it worked before in debug mode at all, because I had link_libraries wrong (don't have the -l in cmake in front), and for release mode had to fix that and include pthreadVC3.lib .. here's the final final from me CMakeLists.txt for the csrc folder
CMakeLists.txt

Instructions

You are a hero, thank you so much for posting this!

Quick final follow up...

Any chance you could hang some instructions somewhere to help others replicate the process? I'm trying to follow along but have run into some issues. A step-by-step would be awesome.

From my not so great memory it's something like:
To build the bitsandbytes project for Windows, you will need two programs: cmake and nvcc. You can a build environment such as Visual Studio and Miniconda.
Open the command line interface (CLI) for your build environment. (Start menu/visual studio/ one of them consoles)
Activate your chosen environment(Miniconda) and install necessary packages. ( cuda-nvcc iirc? Probably a cuda environment like https://pytorch.org/get-started/locally/ )
Place cmake files in the right location.
Build pthreads (if necessary) using cmake (The same commands as below).
Run the following commands: (Note -j4 means use 4 cores to build. If you don't have 4 cores, or you have a lot more, change that number.)
mkdir build
cd build
cmake ..
cmake --build ./ -j4 --config Release
The resulting dll file will be in build/csrc/Release/bitsandbytes.dll.

Is it possible to run bitsandbytes with 2060RTX 6GB on Windows 10?

Is it possible to run bitsandbytes with 2060RTX 6GB on Windows 10?

I don't see why it wouldn't run on a 2060, just be aware it doesn't eliminate vram requirements, just reduces them. Still wouldn't be able to run chatgpt for example with it's 800GB+ 32-bit precision vram requirement (if had access to that). Any model that takes <24gb of vram in 32bit, or <12gb of vram in 16bit mode, should be able to fit in 6gb at 8bit.

@centerionware
Thanks for the answer.
I think I get confused about bitsandbytes. I was thinking it only works on Linux and repository such as kohya-ss/sd-scripts or bmaltais/kohya_ss won't work on Windows, because of the lack of compatibility and that is the reason I am not able to run them. But this is not true, as I manage to make them work on my machine eventually.

After some really tedious debugging and tackling various hidden problems, I managed to compile the whole module. This is the end result: https://github.com/DeXtmL/bitsandbytes-win-prebuilt
The binaries are compiled against CUDAToolkit 11.6 and Visual Studio 2022. I am able to make inferences nearly identical to "normal" fp16 version. So this is kind of a confirmation for "it works" shout-out. No vigorous testing was conducted though. @TimDettmers Finally, the "cuda_setup" part of the source code is entirely incompatible with Windows, there are loads of hardcoded routines; so I used a quick makeshift patch instead of making it proper, that's also why I'm not posting my changes or making PR for now. If you are eager to test:
in cuda_setup/main.py:
make evaluate_cuda_setup() always return "libbitsandbytes_cuda116.dll"
in ./cextension.py:
change ct.cdll.LoadLibrary(binary_path) to ct.cdll.LoadLibrary(str(binary_path))

That should do the trick.
Hopefully this can help someone in the Windows territory; let's hope the official windows support come fast.

This solution is still valid and the linked binaries work with cuda 11.7 as well (at least for Adam 8 bit and on win 10). But the location of ct.cdll.LoadLibrary changed and it can now be found in ./cuda_setup/main.py as well and not in ./cextension.py
just replace both occurrences of self.lib = ct.cdll.LoadLibrary(binary_path)
with self.lib = ct.cdll.LoadLibrary(str(binary_path))

Let me share my bits - users of kohya_ss doing something similiar https://github.com/bmaltais/kohya_ss/tree/master/bitsandbytes_windows. And steps of how it must work on windows machines simple and clear.
cp .\bitsandbytes_windows\*.dll <place-for-python-or-venv>\Lib\site-packages\bitsandbytes\
cp .\bitsandbytes_windows\cextension.py <place-for-python-or-venv>\Lib\site-packages\bitsandbytes\cextension.py
cp .\bitsandbytes_windows\main.py <place-for-python-or-venv>\Lib\site-packages\bitsandbytes\cuda_setup\main.py

To use this with facebook-research/LLaMA-7b within text-generation-webui on windows 11:

install text-generation-webui
download HuggingFace converted model weights for LLaMA, or convert them by yourself from the original weights
put libbitsandbytes_cuda116.dll in C:\Users\xxx\miniconda3\envs\textgen\lib\site-packages\bitsandbytes\
edit \bitsandbytes\cuda_setup\main.py:

search for:
if not torch.cuda.is_available(): return 'libsbitsandbytes_cpu.so', None, None, None, None
replace with:
if torch.cuda.is_available(): return 'libbitsandbytes_cuda116.dll', None, None, None, None
search for this twice:
self.lib = ct.cdll.LoadLibrary(binary_path)
replace with:
self.lib = ct.cdll.LoadLibrary(str(binary_path))

I've tried all of the above and I'm still getting an error
... bitsandbytes\libbitsandbytes_cuda116.dll...
[WinError 193] %1 is not a valid Win32 application
Any ideas?  I thought the whole point of this .dll was that it is a windows version?

That sounds like an error with a script, not an error with the DLL.
On Wed, Mar 15, 2023 at 5:00 PM testFaze @.***> wrote:

I've tried all of the above and I'm still getting an error
... bitsandbytes\libbitsandbytes_cuda116.dll...
[WinError 193] %1 is not a valid Win32 application
Any ideas? I thought the whole point of this .dll was that it is a windows
version?
—
Reply to this email directly, view it on GitHub
https://github.com/TimDettmers/bitsandbytes/issues/30#issuecomment-1471001081,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AM3LWCXG5CYX53ELQBR4FQDW4JJZBANCNFSM6AAAAAAQFJ4QZQ
.
You are receiving this because you commented.Message ID:
@.***>

--
Tony Beeman
he/him or they/them
Assistant Artistic Director | Unexpected Productions
http://unexpectedproductions.org/

Thankyou.
TIL (1) - if you right click/save a filename from GitHub, while it appears you are saving the target file, you're not.  The file that appears in the target folder has the same name but is gibberish.
TIL (2) - I am not very good at IT.
All working fine now.

I've got this compiling under CUDA 11.7 with CMake if y'all are interested. I DID NOT RUN ANY TESTS yet, it is too late
Prototype CMAKE file, it is missing functionality of the makefile. It is usable to target a single config, and does not bring in /dependencies/cub.
https://github.com/acpopescu/bitsandbytes/tree/cmake_windows
To change for your CUDA version:

look in CMakelists.txt
change libbitsandbytes_cuda${CMAKE_CUDA_STANDARD}7 to whatever you have
change set(CMAKE_CUDA_STANDARD 11) to whatever you have
change set(CMAKE_CUDA_ARCHITECTURES 75 80 86) to whatever you have
change in cuda_setup/main.py CUDA_RUNTIME_LIB: str = "libcudart.so" if not IS_WINDOWS_PLATFORM else "cudart64_110.dll". Yes windows requires the version to be specified there.

Still WIP.
To Deploy copy build/Release/. to ./bitsandbytes
For reference and diff - https://github.com/TimDettmers/bitsandbytes/pull/229

Automated tests look promising on 11.7, at least on GPU :) https://github.com/TimDettmers/bitsandbytes/pull/229#issuecomment-1486142982

I've got this compiling under CUDA 11.7 with CMake if y'all are interested. I DID NOT RUN ANY TESTS yet, it is too late in the day
Prototype CMAKE file, it is missing functionality of the makefile. It is usable to target a single config, and does not bring in /dependencies/cub.
https://github.com/acpopescu/bitsandbytes/tree/cmake_windows Still WIP.
To Deploy copy build/Release/*.* to ./bitsandbytes/
For reference and diff - #229

Did you look at https://github.com/TimDettmers/bitsandbytes/pull/127  by any chance?

@centerionware no, I couldn't find that referenced anywhere. I was looking directly at the issues.

I'm very glad you're taking it further than I wanted to and getting the entire wheel outputting. Thank you! On Mar 27, 2023 10:28 PM, Adrian Popescu @.***> wrote:
@centerionware no, I couldn't find that referenced anywhere. I was looking directly at the issues.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

Hey, yeah I need to change the cmake to add the _noblaslt for that option.
Happy to hear that the resulted DLL works by renaming, I did not test it for noblaslt btw.

@km19809  the build script should be fixed, renamed properly and copied in the right location

Compiled a release here, against 11.6 and 11.7 - Feature level 11x with and without cublast:
https://github.com/acpopescu/bitsandbytes/releases/tag/v0.37.2-win.0

@acpopescu
Thank you so much! I'll try it.

Hmm...
The current wheel throws symbol not found.
CUDA SETUP: CUDA runtime path found: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin\cudart64_110.dll
CUDA SETUP: Highest compute capability among GPUs detected: 6.1
CUDA SETUP: Detected CUDA version 117
D:\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\cuda_setup\main.py:141: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!
  warn(msg)
CUDA SETUP: Loading binary D:\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda117_nocublaslt.dll...
Error named symbol not found at line 508 in file F:\dev\AI\BLOOM\bitsandbytes\csrc\ops.cu

However, I have no F: at all...
Although I've searched all the files in the repo to find "F:" or "BLOOM", they did not appear.
I cannot understand why this happens.
I should compile this again by myself.

@acpopescu Now it works well. Thank you a lot!

AttributeError: function 'clion32bit_g32' not found
how can i do? i am crying...

@acpopescu I did an update to your fork with newer bitsandbytes version plus applied one fix for qlora on Windows, maybe you would wish to apply same changes to your branch - https://github.com/stoperro/bitsandbytes_windows/tree/cmake_windows .

@stoperro  I'll take a look when I can (hopefully soon) :)

I am able to compile the csrc part on Windows but fail to link (Visual Studio) FYI, this is the error message. I'm no expert on cuda, so I'm not quite sure where it goes wrong.
To successfully compile on windows MSVC, some parts need to be patched as following:

rename pythonInterface.c to pythonInterface.cpp, or visual studio will try using a C compiler for it.
add one missing template instantiation like this: (in SIMD.h)

get unistd.h and getopt.h for windows
get pthread for windows

finally, this is just a build test, so I'm not using the Makefile. That's all for now. Tested on CUDA toolkit 11.6, windows 11

Thanks for your work, by referencing your idea, I am able to buid it on Windows11, CUDA 12.1, VS 2022 Community!
I have forked it with my procedure and code changes: https://github.com/ShanGor/bitsandbytes-windows
Who wants to run it on Windows with insufficient GPU memory can try it.

Error Message: ModuleNotFoundError: No module named 'transformers_modules.'
I encountered the following error message while running your bitsandbytes code in a Windows environment:
bin D:\sj\project\python\test\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda117.dll
CUDA SETUP: CUDA runtime path found: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin\cudart64_110.dll
CUDA SETUP: Highest compute capability among GPUs detected: 8.6
CUDA SETUP: Detected CUDA version 117
CUDA SETUP: Loading binary D:\sj\project\python\test\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda117.dll...
D:\sj\project\python\test\venv\lib\site-packages\bitsandbytes\cuda_setup\main.py:156: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('D:/yy/寰\ue1bb俊web寮�鍙戣�呭伐鍏穃dll'), WindowsPath('C:/Users/aoguai/anaconda3/Library/usr/bin')}
  warn(msg)
Traceback (most recent call last):
  File "D:\sj\project\python\test\core\deeplearning\test.py", line 13, in <module>
    tokenizer = AutoTokenizer.from_pretrained(
  File "D:\sj\project\python\test\venv\lib\site-packages\transformers\models\auto\tokenization_auto.py", line 676, in from_pretrained
    tokenizer_class = get_class_from_dynamic_module(class_ref, pretrained_model_name_or_path, **kwargs)
  File "D:\sj\project\python\test\venv\lib\site-packages\transformers\dynamic_module_utils.py", line 443, in get_class_from_dynamic_module
    return get_class_in_module(class_name, final_module.replace(".py", ""))
  File "D:\sj\project\python\test\venv\lib\site-packages\transformers\dynamic_module_utils.py", line 164, in get_class_in_module
    module = importlib.import_module(module_path)
  File "D:\yy\Python\Python38\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'transformers_modules.'

I have already installed the transformers4.30.2. The issue occurred when I tried to use the AutoTokenizer.from_pretrained() function from bitsandbytes. Specifically, the error message indicates that it cannot find the module named transformers_modules.
I have attempted to check my environment settings, including CUDA configuration and path setup. However, these attempts did not resolve the issue.
I would appreciate your assistance in resolving this problem. Thank you for developing this library, and I appreciate your help!

The bitsandbytes library is currently only supported on Linux distributions. Windows is not supported at the moment.
https://github.com/acpopescu/bitsandbytes/tree/cmake_windows
搞了半天，不支持windows

why we still cant use on windows
this is 2023

@FurkanGozukara  we can use it on Windows as many people do (me including https://github.com/stoperro/bitsandbytes_windows) with this community effort. It's just less convenient as support is not yet merged into official branch and you need to compile it yourself.
Note that even Microsoft doesn't care about Windows support of their own AI tools (https://github.com/microsoft/DeepSpeed/issues/2427), so having some support here when authors don't necessarily have Windows machine is heartening.

Using Windows is frustrating....

@FurkanGozukara we can use it on Windows as many people do (me including https://github.com/stoperro/bitsandbytes_windows) with this community effort. It's just less convenient as support is not yet merged into official branch and you need to compile it yourself.
Note that even Microsoft doesn't care about Windows support of their own AI tools (microsoft/DeepSpeed#2427), so having some support here when authors don't necessarily have Windows machine is heartening.

I know that branch really good but really older commit
I am making tutorials for regular people. It is not an option for them to "compile it yourself"
I hope windows support gets added

If anyone else is still searching for a Windows solution and doesn't want to lose a few hours to the same issue just use this repo:
https://github.com/jllllll/bitsandbytes-windows-webui
The README even includes a pip install command and (as of 0.41.1) installs the newest version of bitsandbytes... only difference is that it's compatible with windows. Includes .dll instead of .so and cuda_setup\main.py works with us

If anyone else is still searching for a Windows solution ~and doesn't want to lose a few hours to the same issue~ just use this repo: https://github.com/jllllll/bitsandbytes-windows-webui
The README even includes a pip install command and (as of 0.41.1) installs the newest version of bitsandbytes... only difference is that it's compatible with windows. Includes .dll instead of .so and cuda_setup\main.py works with us

wow very nice

If anyone else is still searching for a Windows solution ~and doesn't want to lose a few hours to the same issue~ just use this repo: https://github.com/jllllll/bitsandbytes-windows-webui
The README even includes a pip install command and (as of 0.41.1) installs the newest version of bitsandbytes... only difference is that it's compatible with windows. Includes .dll instead of .so and cuda_setup\main.py works with us

Legend it worked for me I was one of those looking for hours xDD

Bitsandbytes was not supported windows before, but my method can support windows.（yuhuang）
1  open folder J:\StableDiffusion\sdwebui，Click the address bar of the folder and enter CMD
or WIN+R, CMD  。enter，cd /d J:\StableDiffusion\sdwebui
2  J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes
3 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes-windows
4 J:\StableDiffusion\sdwebui\py310\python.exe -m pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.1-py3-none-win_amd64.whl
Replace your SD venv directory file（python.exe Folder） here（J:\StableDiffusion\sdwebui\py310）

pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.1-py3-none-win_amd64.whl

Excellent! On Anaconda same thing applies - has to be root of the venv


Update list of available feeds after adding a feed

When adding a new feed, the feeds list in the Manage Feeds dialog doesn't immediately update.

Resolved in 05515bf


Could DCN be used only by CPU

i want to run the demo.py(r-fcn) only by CPU，so i want to ask whether it support ONLY-CPU

We didn't write CPU code, so you need to write the CPU code by yourself.


[CORE][DOC][MINOR] Remove incorrect scaladoc

What changes were proposed in this pull request?
It removes a sentence about SparkListener being internal and may change in the future yet the class is @DeveloperApi that may or may not say the same. If it does not, it's clearly incorrect. If it says what @DeveloperApi is for, it's a duplication (and/or would require other @DeveloperApi to have it, too).
How was this patch tested?
manual build

Test build #59724 has started for PR 13432 at commit 306e60e.

Before @srowen jumps on this ... can you batch your fixes?

Test build #59724 has finished for PR 13432 at commit 306e60e.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59724/
Test FAILed.

Also I'm not sure if this is incorrect. Even though it's @DeveloperApi it's still mainly for internal use and we don't guarantee backward compatibility with these APIs.

@jaceklaskowski Can you close this PR?


How can I fine tune Llama 3 on my own data?

Hi,
I have been looking for documentation on how to add my own data set into your config, but without success. If it's possible, could anyone provide a small guide on what needs to be done in order to:

Fine tune on csv data.
or
Arbitrary Hugging Face data.

Much appreciated!

Hi @agutell, great questions! I think both of these use-cases are covered by this tutorial: https://pytorch.org/torchtune/main/tutorials/chat.html.
If you have follow-up questions though, please let us know!

Thank you! I'll check it out. One thing though, that page does not seem to show when you enter the documentation from https://pytorch.org/torchtune/stable/index.html

Thank you! I'll check it out. One thing though, that page does not seem to show when you enter the documentation from pytorch.org/torchtune/stable/index.html

So we have a stable version of the documentation that is tied to our code at the v0.1.1 release - this is the one you see immediately if you navigate to the torchtune docs from the web. However, we also update our docs constantly as we contribute more code to the repository. This code lives under the dropdown on the corner of the website under "main". You can also access it through pytorch.org/torchtune/main/index.html.
Because we're developing so quickly on torchtune, it's best to check the "main" documentation.

Ok, thanks! Thats very helpful :)
About the earlier message, regarding the custom data. I have now implemented message_converter and custom_dataset. However, the config demands a "dotted path" to the custom_dataset. I am working in a notebook (google colab) and I have not found a way to create a dotted path that the function  "_get_component_from_path(path: str)" will accept. I keep getting  the error:
raise InstantiationError(
f"Error loading '{path}':\n{repr(exc_import)}"
+ f"\nAre you sure that module '{part0}' is installed?"
I there an easier way to get around this problem, instead of trying to define a module and then give the dotted path? It would be preferable if one could just specify the function name "custom_data" in the config, when working from a notebook, and the function "custom_data" is just defined in a cell.
Thanks for all your work! :)

I there an easier way to get around this problem, instead of trying to define a module and then give the dotted path? It would be preferable if one could just specify the function name "custom_data" in the config, when working from a notebook, and the function "custom_data" is just defined in a cell.

Hi @agutell, this is a good question. I think to have arbitrary functions/modules importable by torchtune from cells is a bit tricky as there is not really a consistent way to define this in a globally unique fashion. We had discussed using a registry, but ultimately found it was harder to scale than the current approach based on _component_. For this particular case, can you use an editable install of torchtune in Colab and add your own dataset there?
E.g. for install run
!git clone https://github.com/pytorch/torchtune.git
!pip install -e torchtune

Then from the file tree you can navigate to torchtune/datasets and create a new file e.g. my_custom_dataset.py containing the function custom_data. Then you should be able to override from the CLI via
!tune run <recipe> --config <config> dataset=torchtune.datasets.my_custom_dataset.custom_data ...

(Or you can do the same thing by modifying whatever config file you're using like this:)
dataset:
  _component_: torchtune.datasets.my_custom_dataset.custom_data

Also tagging @RdoubleA for any thoughts on this.

Thanks for raising this @agutell, this is an excellent case we hadn't fully considered when designing the configs. It would be ideal to support both in code custom datasets and ad-hoc in notebook code that's not directly importable.
What @ebsmothers suggested is a good approach. Actually you don't need to modify the torchtune internals directly (we should prefer user code as an entry point instead of having to modify torchtune internals), you can create a separate .py file in the directory of your notebook with your custom dataset and converters. As long as this is importable in the notebook, it should work as a module path in the config.
I'll put up an issue for how to make this easier in a notebook environment.


add: ssl context to transport.

Added SSL context to transport on Httpx Client.

Thank you for your contribution!


feat: use request histories for the history panel

Fixes #46, #61
This PR uses request histories for the history panel.

Currently, I have no idea what content should be displayed in the detail panel. I think it would be nice to display request data or errors.

This PR requires refactoring, but I'll merge this and work on it later.


Add membership proof to nipost and remove members list from poet proof

Closes #3929, see the issue for motivation and explanation.

bors try

bors try

just in case if it will be useful - previous failure

bors try

bors try

bors try

bors try

bors try

bors merge


JSX Changes

This is a cross-repo change and so I want to have a place here to comment/get feedback from folks.
In particular I want to remember to take a look at all of the issues/prs tagged html attribute change #38 will land since it's a bug but still needs to be ported over.

Is the other repo the syntax repo?

Yup! Here's a (very rough) draft of changes I've chatted about. Needs some cleanup before it's ready for prime time.

@Kingdutch DOM props are now defined as JsxDOM.domProps here in the compiler repo: https://github.com/rescript-lang/rescript-compiler/blob/10.1_release/jscomp/others/jsxDOM.res.
@rickyvetter Closing this issue, as the JSX 4 changes are well underway / almost finished and documented here: https://github.com/rescript-lang/syntax/blob/master/cli/JSXV4.md


BulkInsertAll "recursive" entities with Many-to-Many relationships seems to repeatedly insert into related entity table.

Basically when running a Sql Profiler I saw SQL that was inserting into my UnitAmenities over and over. The main entity I am trying to insert is called Units. I also have a join table called Units_UnitAmenities. Primary keys are all Guids.
Here is the SQL that was running over and over. I was hoping the recursion would insert them all at once (bulk) like it normally does on Units without recursion.
INSERT INTO [dbo].[UnitAmenities] ([UnitAmenityId],[Name])                              SELECT [UnitAmenityId],[Name]                               FROM tempdb..#b249e294eb5e413e8561a4faf2a3cd9f AS [t0]                              WHERE NOT EXISTS (                                 SELECT [UnitAmenityId]                                 FROM [dbo].[UnitAmenities] AS [t1]                                 WHERE [t0].[UnitAmenityId] = [t1].[UnitAmenityId]                              )

Thanks for the feedback.  Interesting error. I have successfully bulk inserted whole hierarchies with join tables but apparently there is still a way to make that not work so well. If you could provide some sample code illustrating what you are doing and how you have configured your entities that would be a huge help.

Oh. Also. I actually fixed something that might be related to your problem recently. If you have not already, please try the latest beta and see if the error is still there.

I did try out both betas. The following strange error only occurs with the betas in my case:
threw exception:
Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: Operator '==' cannot be applied to operands of type 'System.Guid' and 'int'
    at CallSite.Target(Closure , CallSite , Object , Int32 )
   at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.DoBulkInsertAll(DbContext ctx, IList`1 entities, SqlTransaction transaction, Boolean recursive, Boolean allowNotNullSelfReferences, TimeSpan commandTimeout, Dictionary`2 savedEntities, Dictionary`2 mappingsByType, BulkInsertResponse response)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.BulkInsertAll[T](DbContext ctx, BulkInsertRequest`1 request)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.BulkInsertAll[T](DbContext ctx, IList`1 entities, SqlTransaction transaction, Boolean recursive)

My project is a bit large. I hope I can find time to create some to create some simplified examples.

Thank you for the revisions! I gave it a shot and got a different error when trying with recursive.
threw exception:
System.Data.SqlClient.SqlException: An object or column name is missing or empty. For SELECT INTO statements, verify each column has a name. For other statements, look for empty alias names. Aliases defined as "" or [] are not allowed. Change the alias to a valid name.
Database name 'tempdb' ignored, referencing object in tempdb.
Database name 'tempdb' ignored, referencing object in tempdb.
    at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlCommand.RunExecuteNonQueryTds(String methodName, Boolean async, Int32 timeout, Boolean asyncWrite)
   at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
   at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.CreateTempTable(SqlConnection connection, SqlTransaction transaction, TableName tableName, String[] columnNames, Boolean includeRowNumber)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.FillTempTable(SqlConnection conn, IList entities, TableName tableName, Dictionary`2 columnMappings, TableColumnMapping[] keyColumnMappings, TableColumnMapping[] nonKeyColumnMappings, SqlTransaction sqlTransaction)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.DoBulkCopy(DbContext ctx, IList entities, Type t, Mappings mappings, SqlTransaction transaction, Boolean allowNotNullSelfReferences, TimeSpan commandTimeout, BulkInsertResponse response)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.DoBulkInsertAll(DbContext ctx, IList`1 entities, SqlTransaction transaction, Boolean recursive, Boolean allowNotNullSelfReferences, TimeSpan commandTimeout, Dictionary`2 savedEntities, Dictionary`2 mappingsByType, BulkInsertResponse response)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.BulkInsertAll[T](DbContext ctx, BulkInsertRequest`1 request)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.BulkInsertAll[T](DbContext ctx, IList`1 entities, SqlTransaction transaction, Boolean recursive)

Quick notes about my setup:

The Unit entity list I am seeding has a column for example UnitTypeId but I didn't populate the UnitType entity as I didn't want to recursively insert the UnitType entities. I have a few similar Id columns with either Guid or Int. Not sure if that causes an issue.
With the latest beta I tried the BulkInsertAll with recursive = false for my Unit list and UnitAmenities list and had no issues. But without recursive the join table won't be bulk inserted. :-(

Would it be possible for you to send me the class definitions and the EF Configuration? You can email directly to<EMAIL_ADDRESS>Do you have any nullable properties not configured as optional?

Hi!
I solved the immediate problem. For some reason the join table table name
was not in the same place as for non edmx configs but I got that fixed (no
new beta yet though). However. There is another bug related to having Guid
primary keys that are generated in code rather than by the database for
join tables and the tables they "join". An interesting and very useful
setup you have! :-) You managed to punch holes in my little program just
where it hurts!
I'll get back to you as soon as I have fixed it.
/Måns
Den sön 3 feb. 2019 kl 07:25 skrev Steven<EMAIL_ADDRESS>
I made a quick unit test project and database project that recreates my
last mentioned error. I am not sure about email file size limitations but,
looks like it fits on the github upload. I am using EF6 database first
approach so I am not sure about your 'configure as optional' question.
TestingRecursiveBulkInsert.zip
https://github.com/mtanneryd/ef6-bulk-operations/files/2825237/TestingRecursiveBulkInsert.zip
Restating the error:
threw exception:
System.Data.SqlClient.SqlException: An object or column name is missing or empty. For SELECT INTO statements, verify each column has a name. For other statements, look for empty alias names. Aliases defined as "" or [] are not allowed. Change the alias to a valid name.
Database name 'tempdb' ignored, referencing object in tempdb.
Database name 'tempdb' ignored, referencing object in tempdb.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction)    at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at System.Data.SqlClient.SqlCommand.RunExecuteNonQueryTds(String methodName, Boolean async, Int32 timeout, Boolean asyncWrite)
at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)    at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()    at Tanneryd.BulkOperations.EF6.DbContextExtensions.CreateTempTable(SqlConnection connection, SqlTransaction transaction, TableName tableName, String[] columnNames, Boolean includeRowNumber)    at Tanneryd.BulkOperations.EF6.DbContextExtensions.FillTempTable(SqlConnection conn, IList entities, TableName tableName, Dictionary2 columnMappings, TableColumnMapping[] keyColumnMappings, TableColumnMapping[] nonKeyColumnMappings, SqlTransaction sqlTransaction)
at Tanneryd.BulkOperations.EF6.DbContextExtensions.DoBulkCopy(DbContext ctx, IList entities, Type t, Mappings mappings, SqlTransaction transaction, Boolean allowNotNullSelfReferences, TimeSpan commandTimeout, BulkInsertResponse response)
at Tanneryd.BulkOperations.EF6.DbContextExtensions.DoBulkInsertAll(DbContext ctx, IList1 entities, SqlTransaction transaction, Boolean recursive, Boolean allowNotNullSelfReferences, TimeSpan commandTimeout, Dictionary2 savedEntities, Dictionary2 mappingsByType, BulkInsertResponse response)    at Tanneryd.BulkOperations.EF6.DbContextExtensions.BulkInsertAll[T](DbContext ctx, BulkInsertRequest1 request)
at Tanneryd.BulkOperations.EF6.DbContextExtensions.BulkInsertAll[T](DbContext ctx, IList`1 entities, SqlTransaction transaction, Boolean recursive)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/mtanneryd/ef6-bulk-operations/issues/13#issuecomment-460026904,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADQZRannxHP4q0nsUf-LAaGzWpF83Zlhks5vJoDFgaJpZM4afG2P
.

I ran a quick test on 1.2.3-beta4 with my sample project posted previously.
I found that it only worked without the transaction. When I ran with the transaction that I had in the upload I get the following error:
Test method TestingRecursiveBulkInsert.BulkRecusiveTest.TestMethod1 threw exception:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.InvalidOperationException: ExecuteNonQuery requires the command to have a transaction when the connection assigned to the command is in a pending local transaction.  The Transaction property of the command has not been initialized.
    at System.Data.SqlClient.SqlCommand.ValidateCommand(String method, Boolean async)
   at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
   at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.CreateTempTable(SqlConnection connection, SqlTransaction transaction, TableName tableName, String[] columnNames, IncludeRowNumber includeRowNumber)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.DoBulkSelectNotExisting[T1,T2](DbContext ctx, BulkSelectRequest`1 request)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.BulkSelectNotExisting[T1,T2](DbContext ctx, BulkSelectRequest`1 request)
--- End of inner exception stack trace ---
    at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.BulkSelectNotExisting(DbContext ctx, Type t, IList entities, TableColumnMapping[] pkColumnMappings)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.DoBulkCopy(DbContext ctx, IList entities, Type t, Mappings mappings, SqlTransaction transaction, Boolean allowNotNullSelfReferences, TimeSpan commandTimeout, BulkInsertResponse response)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.DoBulkInsertAll(DbContext ctx, IList`1 entities, SqlTransaction transaction, Boolean recursive, Boolean allowNotNullSelfReferences, TimeSpan commandTimeout, Dictionary`2 savedEntities, Dictionary`2 mappingsByType, BulkInsertResponse response)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.BulkInsertAll[T](DbContext ctx, BulkInsertRequest`1 request)
   at Tanneryd.BulkOperations.EF6.DbContextExtensions.BulkInsertAll[T](DbContext ctx, IList`1 entities, SqlTransaction transaction, Boolean recursive)
   at TestingRecursiveBulkInsert.BulkRecusiveTest.TestMethod1() in C:\Users\StevenLyon\source\repos\TestingRecursiveBulkInsert\TestingRecursiveBulkInsert\BulkRecusiveTest.cs:line 64

I also noticed BulkInsertAll() stopped Updating existing rows for me. Could that be an added option?

Sorry about that. I forgot to pass on the transaction when calling the new BulkSelectNotExisting using runtime binding to generic methods. Tested it with your project and it works now. About the updating of existing rows I think that was an unintentional side effect that might have gone now. Implementing it for real has been on my backlog for a while. Adding it as an option would be the right way to go. I'll look into it. There is yet another beta available now. Sorry about the poor quality of my updates and thanks a lot for helping me out finding bugs!

Thanks so much for the updates. I did have trouble with the Recursive option. I scaled up my test to around 10,000 "Units" and 25 unique "UnitAmenities" joined in. I found that the completion time came to a crawl and my sql profile was repeating the following highlighted section tons of times. (Attached screenshot)

Very interesting. Thanks for your feedback! I really appreciate it.

Good catch! Usually when we recursively insert entities with fk relationships we're being reasonably smart about it. But, for join tables the situation is a little different and as it turns out we're not being smart at all from a performance perspective. I found the problem and will fix it as soon as I can.

There is another beta out 1.2.3-beta6. I have not been able to test it enough yet but initial tests indicate that it solves the performance problem, at least. If you have the time, feel free to check it out.

As far as I can tell this now works. Please repoen the issue if the problems reappear.


Segmentation fault (core dumped)

The code crashed at last line in 'metrics/mAP_1000.py', when I run with cuda11+pytorch1.8.0.
"Segmentation fault (core dumped)" arise, when "return np.mean(avg_r_precisions)"

Hey unfortunately a segmentation fault can have multiple issues, especially as I havent tested the code with both Cuda 11 and PyTorch 1.8.0. If you could test and run with CUDA <= 10.1 and PyTorch 1.6.0, I could offer better help.
Have you checked what avg_r_precisions looks like?

Closed it for now, feel free to reopen it if more detailed issues arise :).