Add minimal pen tablet support

Hi, I would like to propose this minimal pen tablet support (wacom etc) using wintab32 for win32.
With support for x, y, z, pressure, pitch, yaw, roll, cursor (pen, eraser etc) and proximity.
@elmindreda, I have something similar for cocoa and x11 but I wanted to check first if you are ok with this.
I tested variants of these over the years by modifying glfw's code for in-house painting tools, mainly on mac and windows.

This looks very promising. Will review after 3.3 is out. Two issues remaining now.

If you are ok with the structure of it, I can do the same for X11. What do you think ? (The issues reported by Travis should be fixed now).

I can't review it in detail right now but at minimum it looks like a very solid foundation. The cursor and proximity events and their parameters could perhaps do with a bit more documentation.

Ok, thanks, I'll detail the doc.

I added XInput2 basic support. Handling the "Coordinate Transformation Matrix" property was a bit tricky but this way we can get X and Y at full precision. I didn't find how to read Z coordinate though. I tested it on Linux Mint with a Graphire2 and an Intuos4 but it should be tested on other systems.

Saw your tweet @anael-seghezzi and thought I'd let you know that the Z coordinate isn't reported by the X driver. The only way to get it on Linux is through the Wayland tablet protocol or directly through the kernel. Also looks like you're properly using the min/max data of the valuators, so it should be 8K-safe :+1:

Another thing I don't know how to handle correctly on x11 linux is to catch the tablet being unplugged en route and disable it. Right now it causes a crash.

I have an Intuos on loan now and have tested this PR on Windows, X11 and macOS. It worked well on all platforms. Will try to help out with the X11 disconnect issue.
Definitely want this in 3.4. Thank you!

@anael-seghezzi I'm not sure if you're already handling the HierarchyChanged events that are sent when devices are enabled/disabled/added/removed, but that might be the key if you aren't.

EVENT type 12 (PropertyEvent)
     property: 166 'Device Enabled'
     changed: modified
EVENT type 11 (HierarchyChanged)
    Changes happened:        [device disabled]
[...]
    device 10 [slave pointer (2)] is disabled
    changes:        [device disabled]
[... Lots of "EVENT type 12 (PropertyEvent)" as device props are deleted ...]
EVENT type 11 (HierarchyChanged)
    Changes happened:    [slave removed]
[...]
    device 10 [<undefined> (0)] is disabled
    changes:    [slave removed]
[... Repeat the sequence of "HierarchyChanged" and "PropertyEvent" events for all the other disconnected devices ...]

Thank you jigpu I'll have a look at it.

heya, just wanted to say thanks for the pull request and that i'm successfully using it. to resolve build/merge problems, i had to add this commit:
https://github.com/hanatos/glfw/commit/709d88f5d87d12c7b2b58e601526a35fd00508a3

What is going on with this branch, are we just waiting for review?

@hanatos Hey there's something very screwy going on with your repo.  Like I think the git history is corrupted or something, if I look at your repo on github, I see case WT_PROXIMITY: in the correct place, but if I clone your repo, case WT_PROXIMITY: appears out of order and causes \hanatos_glfw\src\win32_window.c(550): error C2046: illegal case

@Cazadorro ah nice catch. i don't normally have access to macintosh or windows computers, so i just didn't notice. i mean i can try and fix this syntactically (it's a mixture of cascaded if and switch statements, probably should be switch all the way) but i won't be able to thoroughly test on windows computers.

but yes, it would be great to have this merged! let me know if i can help. i could offer rebasing to current master, testing the linux version and i might figure out a way to test windows too.

@hanatos You can use github CI for free to test these compilation issues on windows with out using a windows computer yourself, the fixes appear to have gotten the vulkan example to work in glfw.

fair point. pushed to the master branch in my repo now, so it will run the checks. it's also rebased to current upstream/master which was surprisingly simple. i tested drawing on linux and can confirm this still works.

@anael-seghezzi There needs to be the ability to set a user pointer for use in the callbacks.  Every single other GLFW API callback function has this capability.  Because there's apparently no object assigned to this (unlike monitor or joystick, or no associated window ala with GLFWWindow), I'm not sure how that would work, maybe just have void* user_ptr be pass in as the last parameter to the function.
The reason user_ptr is so important is that it allows wrappers to use arbitrary std::function objects, which may capture state, by delegating calls to the user_ptr instead of the actual function pointer with a wrapper function.  This also allows lambdas which capture variables to be passed in as callbacks.

@anael-seghezzi @hanatos Confirms this does not work on windows.  My huion tablet can correctly be used as a mouse cursor, but no tablet callbacks are triggered when running tests.  I've never had issues with this in any other program.  It's possible that wintab32 is just not the correct API to use on windows period, as even Wacom recommends against using it, instead opting for Windows Ink.

I'm investigating what needs to happen to get Windows Ink API to work.  there may be many touch devices, and I think a GLFWmonitor like API may be needed for this.  This repo seems to have a pretty minimalistic easy example of how to use the API on windows: https://github.com/glynskyi/stylus_api/blob/main/windows/stylus_api_plugin.cpp

I got windows ink to work with GLFW (on the outside), but I found in the process that the wintab version of GLFW eats my inputs or does something else horribly wrong, windows ink will only work for my device if I don't use wintab.  This means that you will actually break peoples code with the current implementation.    I also tested Krita and realized that I had been using wintab32 the entire time with out problems, and can also use windows ink version entirely with out problems, which seems to indicate to me there's something weird going on with this library.  I have a feeling it has to do with initialization, in the fact that we suspiciously aren't passing any info on exactly what device we want to use, but it's hard to find API documentation for wintab to figure out if this is actually the case.

I have been using my original pull request fork on production for some time, tested on windows 7 and windows 10, on many wacom products. But when testing on XP-PEN Artist 22 Pro, I ran in some issues, you can see the type of fixes here https://github.com/anael-seghezzi/glfw/commit/8533f58c91f94421f6b7a6204ff9073959159695 and here https://github.com/anael-seghezzi/glfw/commit/213a2965ddc29dc31843524f5baab13926d5b5f3. It's possible that the Huion tablet trigger some specific issue similarly to the XP-PEN.


Fine tuning

How to fine tune your model?
I don't have sufficient data to retrain your model from scratch.I want to fine tune your model on my data which has only two classes ?

I included the train_val file and solver of the GoogleCAM. You just follow the standard fine-tuning procedure to finetune the googleCAM model.
If you are not familiar with fine-tuning in caffe, you could just google fine-tune in caffe, or take this page as example: http://caffe.berkeleyvision.org/gathered/examples/finetune_flickr_style.html


Sync speakers, talks and workshops from pretalx

Django management command for initial synchronization and full-updates.
Configure Django admin interface for speakers, talks and workshops.
Add Django admin actions for syncing of individual speakers/talks/workshops.
Add Django template tag for rendering markdown.


New about guides

Related to #152
Opening this PR up for some early review on the docs. I'd love to tackle the following:

[x]  Intro to Polaris / what is Polaris?
[x]    Getting started guide
[ ]    Full guide on Routing (similar to here)

cc: @TylerLeonhardt @jeremymcgee73 @ChendrayanV

@TylerLeonhardt - I think I got em all if you want to give this another look before I merge.


Substantial rewrite to support ND-LAr reco

This pull request would introduce a fairly large rewrite to ND_CAFMaker in order to support filling of ND-LAr reco branches into output CAFs.  What changed:

Reorganize the files so there's less clutter at top level
Introduce a FHICL-based configuration system so that parameters are more easily versioned and modified (instead of the command-line-switch-based system)
Move the pseudo-reco code out of the top-level makeCAF.cxx (see also below)
Support for / dependency on more "structured" StandardRecord from duneanaobj  ~requires its PR #1 to be merged first~ ✔
Introduce a base class IRecoBranchFiller and the skeleton for switching which concrete chain of reco objects actually gets filled (whether ND-LAr only, ND-LAr + TMS, ND-LAr + ND-GAr, ...)
Rearrange the "parameterized reco" code as a concrete IRecoBranchFiller, and leave it on by default (used unless an mlreco input file is passed)
Implement a concrete IRecoBranchFiller that reads the HD5 .h5 summary file outputs from the ND-LAr reco and fills the reco branches with them

CAVEATS

Adding the xsec weights to the new StandardRecord appears not to be fully working.  Many of them (all those directly copied from GENIE knobs?) seem to be basically ok, but some (I think only those directly implemented in nusystematics?) aren't producing the same values.  (I have a Python notebook that compares an old CAF and a new CAF made from the same GENIE input which can be shared.)  After discussions with Luke P., the interaction model weights have been disabled until the DIRT-II group finishes the model & uncertainty work.
Relatedly (?), the CAFAna code that automatically calculates a default central value weight (FixupRecord()) needs to be disabled to use these CAFs.  I have a branch of lblpwgtools feature/ndlar-cafs-wip that disables it that can be used until this can be resolved.

Most of the actual caf-making details are over my head, but I'm happy with the broad outline. It seems more likely someone else should look at this than it did for duneanaobj, but I'm not sure if it's actually necessary.

I believe @muether and @lcrem are responsible for figuring out who should officially review here.

As per discussion at the May 2022 collaboration meeting, I'm merging this now.  The previous version is tagged as last_unstructured_cafs.
Note that the nusystematics plugins are disabled for now until an expert can look at them


[PROPOSAL] Schemeless Dimensions Support

Motivation
Currently, when using schema-less ingestion, all dimensions will be ingested as String-typed dimensions [1]. However, one can define auto type detection for timestamp as follows:
"timestampSpec" : {
     "format" : "auto",
     "column" : "ts"
}

In similar manner, one cannot detect field type via parseSpec. i.e.:
{"ts":"2018-01-01T03:35:45Z","app_token":"guid1","eventName":"app-x","properties-key1":"123"}
{"ts":"2018-01-01T03:35:45Z","app_token":"guid2","eventName":"app-x","properties-key2":123}
Both properties-key1 and properties-key2 are indexed as String. It is expected to index properties-key2 as Integer or Long at Druid.
Auto field type detection should be implemented into Druid for a newly created field as similar to Solr Schemaless Mode
PS 1: I've started a conversation about this task at mail list and than created an issue for it: Druid Auto Field Type Detection
PS 2: I've created a discussion issue for possible solutions of this issue: https://github.com/apache/incubator-druid/issues/7027
[1] Druid Schema Design
Proposed changes
Current Implementation
IncrementalIndex.java has following code section:
if (desc != null) {
  capabilities = desc.getCapabilities();
} else {
  wasNewDim = true;
  capabilities = columnCapabilities.get(dimension);
  if (capabilities == null) {
    capabilities = new ColumnCapabilitiesImpl();
    // For schemaless type discovery, assume everything is a String for now, can change later.
    capabilities.setType(ValueType.STRING);
    capabilities.setDictionaryEncoded(true);
    capabilities.setHasBitmapIndexes(true);
    columnCapabilities.put(dimension, capabilities);
  }
  DimensionHandler handler = DimensionHandlerUtils.getHandlerFromCapabilities(dimension, capabilities, null);
  desc = addNewDimension(dimension, capabilities, handler);
}

Suggested Implementation
We can create a regex defined dimension for capturing non-defined dimensions to map types. Such configuration can be defined for this purpose:
{ "pattern": "*_l", "type": "long" },
{ "pattern": "*_f", "type": "float" }

Flow will be as described below:

If given dimension name is matched within defined schema, code will work as is.

If given dimension name is not matched within defined schema:

First pattern checks dimension and if it ends with _l maps the dimension into a Long type.

Second pattern check dimension with similar manner and if it ends with _f maps the dimension into a Float type.

If not any pattern matches, dimension is assigned into a String field as is.

Rationale
Other possible solutions are:

If value can be convertible to Float then convert to Float, otherwise try with Long, otherwise go with String.

If field name ends with _l, and is not defined at scheme, and a config value of schemeless mode is enabled then convert to Long, so on so forth.

However, both solutions are not generic and vulnerable to having null values at first row for particular dimensions. Defining a regex pattern for such purpose is the best choice among these.
Only open question is deciding the format of the regex pattern.
Operational impact
This improvement does not change existing code flow.
Test plan
Regular test codes will be implemented.
Future work
We should consider https://github.com/apache/incubator-druid/issues/658 when implementing this issue.

You are welcome to give feedback about proposed solution. I'll can improve my ongoing implementation according to given comments.

SGTM as a feature. In fact this doesn't exist only because non-String dimension columns were an afterthought.

If given dimension name is not matched within defined schema:

I wouldn't change the default behavior for backward compatibility, reasons using patterns is fine.
auto type-detection based on value observed might not work because you might determine type to be "long" on observing first couple of rows but then you see a non-long value at later time.

Only open question is deciding the format of the regex pattern.

Druid uses java regex pattern in different places so that would be a good choice.
From the proposal, It is not clear where the {pattern,type} blob would be written inside overall indexing task spec .

I'll implement this feature as suggested if there is not any other comments?

@kamaci
This is what I want
So, Have you implemented the feature? which version


Enhance load function to support model configuration editing

The current implementation of the load function lacks the flexibility to adjust specific configuration parameters. I propose enhancing the load function to support a model_config parameter, enabling users to modify these parameters dynamically upon loading a model. For example, this enhancement would allow increasing the context window with position interpolation or adjusting the base frequency of RoPE on the fly.
Recent advancements in language model research have introduced techniques that can significantly enhance the capabilities of base language models (LLMs) without requiring additional fine-tuning. By enabling dynamic model configuration adjustments with the model_config parameter, users can leverage these techniques seamlessly within our framework.
Related Pull Request: #770

Added in #770


WebRTC: Support FS MCU connects to SRS by WHIP for SIP clients.

If you got SIP clients to join a meeting, to communicate with WebRTC clients, how to do that?
Because SIP clients only support 1 video and 1 audio stream, some might suport 1 extra screen stream, so you should use MCU to merge all streams in a room.
FS or Freeswitch is a MCU for SIP clients and also supports WebRTC clients, so you can use FS instead.
Sometimes, the vast majority of rooms have no SIP clients, only a small group of rooms should support SIP client. In this situation, SFU is better solution, because MCU requires a huge resource of CPU for encoding.
If you get only one SIP client, others are WebRTC clients like Chrome browsers, you can also use FS as a SIP to WebRTC proxy to connect to SRS like a WebRTC client.
The bellow is the full architecture:

To do this, FS should support pulling WebRTC stream from SRS by WHIP protocol, please see Unity: Player. I think the workflow should be this:

Chrome A push WebRTC stream to SRS.
SRS call FS HTTP server by HTTP Callback.
FS pulls WebRTC stream from SRS by WHIP.
FS publish a mixed stream to SRS, which contains both WebRTC clients and SIP clients.
Chrome A pull the mixed stream from SRS.

Besides this solution, SRS also clould forward or push WebRTC stream to FS by WHIP, the workflow should be:

Chrome A push WebRTC stream to SRS.
A SIP client connect to FS, and notify your HTTP server about this event.
Your HTTP server notify SRS to start forwarding.
SRS forward the WebRTC stream to FS by WHIP.

Note: We highly recommend that MCU pull stream from SRS by WHIP or WHEP, which is a common solution for MCU and SFU architecture.

要点翻译下：
a. Chrome A 推流到SRS后，回调到FS。
b. FS通过recvonly WHIP从SRS拉Chrome A的流。
c. Chrome B和C和A一样，FS都会把流拉过去。
1.1. FS应该会有一个混流，通过sendonly WHIP送到SRS，这个不一定是同一个SRS，可以是另外一个SRS。
1.2. Chrome A/B/C可以拉这个混流，也可以互相拉流，看用户的策略了。
2.1 FS应该还可以送一个RTMP出来，这就是连麦的直播流了。

Hope the following diagrams help.


Alternate approach for named-units

Starting a new issue to not add another thread to #46.
@dmcclean wrote:

I forgot to comment on internationalisation. I think that it might be best served by going from UnitName to UCUM and allowing people to internationalize by coming back out again? Direct structural recursion on UnitName is another possibility, but our name and abbreviation choices for non-SI units are arguably less standard.

I don't follow exactly what you envisioned in the first suggestion above. But yesterday evening I also had an idea about structural recursion and have drafted the https://github.com/bjornbm/dimensional-dk/tree/typenames branch.
The most interesting modules are UnitNames and  I18N, but there are also some changes elsewhere including a change of the Dimensional type in DK.
Type level "canonical" (our arbitrary choice) unit names are attached as ASTs to Units. When the unit is to be shown the type level AST is converted to a value level AST. The mechanics for this is in UnitNames.
The string representations for units and prefixes in the value level AST can then be translated to other language by use of a Map. See I18N, which also has the non-internationalised showUnit and showIn.
If you load I18N you get the prelude too (right now) and can play around a little in ghci.
*Numeric.Units.Dimensional.DK.I18N> let x = 10 *~ (meter / second ^pos2)
*Numeric.Units.Dimensional.DK.I18N> x
10.0 m s^-2
*Numeric.Units.Dimensional.DK.I18N> showIn (kilo meter / second ^ pos2) x
"1.0e-2 km/s^2"
*Numeric.Units.Dimensional.DK.I18N> showIn (second ^ neg2 * kilo meter) x
"1.0e-2 s^-2 km"
*Numeric.Units.Dimensional.DK.I18N> long en_us (kilo meter / second ^ pos2)
"kilometer/second^2"
*Numeric.Units.Dimensional.DK.I18N> long en_gb (kilo meter / second ^ pos2)
"kilometre/second^2"
*Numeric.Units.Dimensional.DK.I18N> long sv_se (kilo meter / second ^ pos2)
"kilometer/sekund^2"
*Numeric.Units.Dimensional.DK.I18N> short sv_se (kilo meter / second ^ pos2)
"km/s^2"
*Numeric.Units.Dimensional.DK.I18N> showUnit (kilo meter / second * second)
"km/s*s"
*Numeric.Units.Dimensional.DK.I18N> showUnit (kilo meter / (second * second))
"km/(s s)"
*Numeric.Units.Dimensional.DK.I18N> showUnit (one * second)
"s"
*Numeric.Units.Dimensional.DK.I18N> showUnit (one / second)
"1/s"

I haven't thought through all corner cases there might be.
I haven't thought through naming very much, mostly just used what occurred to me first.
I've based the AST around your UnitName data type. The reason for Atomic and PrefixableAtomic eludes me but I kept them all the same as I'm sure you have thought things through. I didn't do any research either: perhaps someone (UCUM?) already has designed the perfect AST for describing unit presentation?
Most units and prefixes are missing from SIUnits, and even fewer (kilo, meter, second) are in I18N.
Does this approach have merit?
(The code could clearly use more documentation, and perhaps som restructuring in places. If it is too opaque for you to judge let me know and I'll try to document better.)

This I like very much.
Sticking points:

Our needing to choose "canonical" unit names, and the lack of solid grounds upon which to make such a choice once you get outside of the SI. This also raises the issue of accidental collision between units defined in different packages. (It's possible to avoid that by pairing names with Data.Uniques, but there's no good story for creating them because of the IO requirement, and people will be confused when units with the same apparent name don't cancel with each other.) (There are other possible ways to avoid this, that are nice, but they require TH at the location where new units are declared.)
A decision is needed on the prefixability question, and there are strong arguments on the merits on both sides, to the point where I don't know which way to turn.
UCUM is an attractive source of (atomic, canonical, human-language-independent) names. And it would certainly make building a UCUM parser easy. But their name grammar did make some questionable decisions, some mentioned above, namely:

allowing rational factors to appear in units
allowing "annotations" with application defined semantics (although they are segregated somewhat, and the standard does say that it's fine for applications to interpret them as semantically void)
They have "special units": "Those symbols that are used as units that imply a measurement on a scale other than a ratio scale (e.g., interval scale, logarithmic scale) are defined differently. These do not represent proper units as elements of the group (U,·). Therefore those special semantic entities are called special units, as opposed to proper units." (e.g. degC, dB). In our design these aren't units at all, because they don't measure quantities. (Of course, we have degC actually, but that's a different concept with the same name, it's the relative-only kind. UCUM models the non-zero-crossing and even non-linear "special units" in detail.)
They have CGS electrostatic units defined with the dimensions they take in the SI basis.
They have an ambiguous grammar caused by not having a delimiter between unit prefixes and the unit itself. Even with the canonical names they defined themselves, there are a few such overlaps, at least when people break the prefixability guidelines, but in the presence of user defined units obviously there could be more.

Addressing the simpler design decisions first:

Sigh. Return multiple results explicitly when parsing. In practice it doesn't come up, and it doesn't introduce much of an issue since you already need to deal with parse failures.

3 seems like an easy one. We don't do that. If we have a dimensional-dk-experimental module that is a UCUM parser, we have it return the UCUM AST. Our conversion function from UCUM AST to dynamic Unit returns Left something when passed such a beast.
4 I'm OK with. There's nothing saying we have to define these just because they did, but if we do we put them in a separate module.
2 is just a world of pain, because it's an even worse global namespace problem than the one with picking "canonical" names for user defined units. Having people pick different strings for "semantically" equivalent things is pretty much guaranteed (evidence: look at all the effort that's been spent on HL7 to try to tame it...). The names tend to name things that are logically dimensions but aren't statically available to the dimension checker. There are lots of other problems. Parse these, return them in the UCUM AST, and when converting to Unit just do what the standard condones and drop them on the floor.
Which brings us to 1, which might be the deal breaker.
(I will now advocate for a "learning to stop worrying and love the bomb" approach to number 1. I'm not as convinced of the correctness of what follows as I am of what has come so far.)
Arbitrary numbers in units aren't so bad. They still have names, so adopting a programmers hat you can still see how they are distinguished from quantities.
Moreover, and this is the key bit, they still have static values. In fact, while I continue to believe as I did upthread that statically tracking unit names isn't useful, statically tracking unit values might be awesome. Indeed, statically tracking unit values is the essence of fixed point arithmetic as it's applied to dimensional values.
Length Integer isn't a very useful type, on one view, because it breaks the dimensional abstraction. "Everyone knows" it's in meters (or, for more complicated dimensions, at least it's in siUnits...), but it's implied in a somewhat unhealthy way. On the other hand, with statically checked values using (the type-level version of...) the multiplication group from exact-pi package would be really nice. We would have machine-readable, statically checked, and dimensional types for things like "16 bit signed integer in milli-gee" and "32 bit unsigned integer in 2^-12 turns" and all the other stuff that comes up in contexts where dimensions and units are application critical but floating point isn't the right choice for the application, or wasn't the choice of legacy software.

Point 6: Their grammar doesn't exclude zero from the integer factors that are acceptable in units.
This is an intolerable atrocity, because it doesn't have a multiplicative inverse.
Filed http://unitsofmeasure.org/trac/ticket/172, since I couldn't find an existing one.

Thanks for the the in-depth analysis. I'm pretty much willing to trust your judgement on these questions, directed by your needs and keeping complexity a low as possible.

Sigh. Return multiple results explicitly when parsing. In practice it doesn't come up, and it doesn't introduce much of an issue since you already need to deal with parse failures.

I assume this comment was actually for 5?

statically tracking unit values might be awesome.

I can see how it could be. It is beyond my current needs but you are welcome to hash out what this could look like in code if you think it will pay off.

It was for 5, I messed up the markdown, which doesn't like starting ol's with numbers other than 1, apparently.

So using your type family concept above, I am making a named-units-2 branch (so I didn't have to delete my old one and can use it for comparison). It's working out pretty well, I think. Not quite ready for review but on the right track. It uses the UCUM-issued unit identifiers where they are available, our own where we have units they don't, and custom ones if end users need them. It has the ability to reduce it into units that exist in the UCUM before sending it over the wire, because it tracks who the issuing authority of the name was.
The exact values/fixed point thing is a whole other can of worms. Deserving of it's own thread if I take a pass at it. But right now I'm not going to, because type rationals are a prerequisite.


WIP [SCB-1577] Improve the transaction compensation mechanism for FSM

When the compensation method fails, only the output log without throwing an exception. Alpha does not know that the compensation method failed.

[x]  Modify the OnConnected method to bidirectional streaming RPC for Saga
This will be needed for the Step2
[ ] Custom compensation exception rules to distinguish between business compensation exceptions and framework compensation exceptions for FSM
After calling the compensation method, Alpha needs to wait for Omega to send a compensation result message. The message type is as follows

Successful compensation
Business compensation failure business code throws an exception, Do not switch Omega instance and retry
Omega compensation failure omega code throws an exception, switch other Omega instance and retry
Retry rule is implemented in the Step3

[ ] Added retry and timeout rule to compensate for failures for FSM
Exceeded the number of retries or timeout will change state to suspended

@coolbeevip This PR is good for me.
I think we can merge it,  do you mind remove the WIP perfix？You can send other PR for the implementation of next steps.

@coolbeevip This PR is good for me.
I think we can merge it, do you mind remove the WIP perfix？You can send other PR for the implementation of next steps.

OK, I will remove the WIP prefix and split the issue of JIRA

I created a new JIRA[1] and convert SCB-1577 into its sub-task
[1] https://issues.apache.org/jira/browse/SCB-1622


Not able to fietune on single node multiple gpu

System Info
PyTorch version: 2.0.1+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A
OS: Ubuntu 20.04.1 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: version 3.27.0
Libc version: glibc-2.31
Python version: 3.11.4 | packaged by conda-forge | (main, Jun 10 2023, 18:08:17) [GCC 12.2.0] (64-bit runtime)
Python platform: Linux-5.13.0-28-generic-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: 11.0.194
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA A100-SXM4-40GB
GPU 1: NVIDIA A100-SXM4-40GB
GPU 2: NVIDIA A100-SXM4-40GB
GPU 3: NVIDIA A100-SXM4-40GB
GPU 4: NVIDIA A100-SXM4-40GB
GPU 5: NVIDIA A100-SXM4-40GB
GPU 6: NVIDIA A100-SXM4-40GB
GPU 7: NVIDIA A100-SXM4-40GB
Nvidia driver version: 470.103.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   43 bits physical, 48 bits virtual
CPU(s):                          64
On-line CPU(s) list:             0-63
Thread(s) per core:              2
Core(s) per socket:              16
Socket(s):                       2
NUMA node(s):                    2
Vendor ID:                       AuthenticAMD
CPU family:                      23
Model:                           49
Model name:                      AMD EPYC 7282 16-Core Processor
Stepping:                        0
Frequency boost:                 enabled
CPU MHz:                         3195.310
CPU max MHz:                     2800.0000
CPU min MHz:                     1500.0000
BogoMIPS:                        5600.46
Virtualization:                  AMD-V
L1d cache:                       1 MiB
L1i cache:                       1 MiB
L2 cache:                        16 MiB
L3 cache:                        128 MiB
NUMA node0 CPU(s):               0-15,32-47
NUMA node1 CPU(s):               16-31,48-63
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full AMD retpoline, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sme sev sev_es
Versions of relevant libraries:
[pip3] numpy==1.25.1
[pip3] torch==2.0.1
[pip3] torch-tb-profiler==0.4.1
[pip3] torchvision==0.15.2
[conda] numpy                     1.25.1                   pypi_0    pypi
[conda] torch                     2.0.1                    pypi_0    pypi
[conda] torch-tb-profiler         0.4.1                    pypi_0    pypi
[conda] torchvision               0.15.2                   pypi_0    pypi
Information

[ ] The official example scripts
[ ] My own modified scripts

🐛 Describe the bug
Tried running torchrun for single node and multigpu but the process exists
Error logs
--> Running with torch dist debug set to detail
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 611719 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 611720 closing signal SIGTERM
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 2 (pid: 611721) of binary: /home/amit_g/scratch/env/llm/bin/python
Traceback (most recent call last):
File "/home/amit_g/scratch/env/llm/bin/torchrun", line 8, in
sys.exit(main())
^^^^^^
File "/home/amit_g/scratch/env/llm/lib/python3.11/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 346, in wrapper
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/home/amit_g/scratch/env/llm/lib/python3.11/site-packages/torch/distributed/run.py", line 794, in main
run(args)
File "/home/amit_g/scratch/env/llm/lib/python3.11/site-packages/torch/distributed/run.py", line 785, in run
elastic_launch(
File "/home/amit_g/scratch/env/llm/lib/python3.11/site-packages/torch/distributed/launcher/api.py", line 134, in call
return launch_agent(self._config, self._entrypoint, list(args))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/amit_g/scratch/env/llm/lib/python3.11/site-packages/torch/distributed/launcher/api.py", line 250, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
./llama_finetuning.py FAILED
Failures:
<NO_OTHER_FAILURES>
Root Cause (first observed failure):
[0]:
time      : 2023-08-09_02:46:25
host      : user
rank      : 2 (local_rank: 2)
exitcode  : -9 (pid: 611721)
error_file: <N/A>
traceback : Signal 9 (SIGKILL) received by PID 611721
Expected behavior
i expected it to train

@aaekay can you pls share the command/ setting (how many GPU/ what type) you are running. Also please note if you are using FSDP + PEFT you need to install PyTorch nightlies.

@HamidShojanazeri  and  after using the nightly pytorch , showing similar error:

"""
File "/home/amit_g/scratch/env/llm/bin/torchrun", line 33, in
sys.exit(load_entry_point('torch==2.1.0.dev20230808', 'console_scripts', 'torchrun')())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/amit_g/scratch/env/llm/lib/python3.11/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 346, in wrapper
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/home/amit_g/scratch/env/llm/lib/python3.11/site-packages/torch/distributed/run.py", line 806, in main
run(args)
File "/home/amit_g/scratch/env/llm/lib/python3.11/site-packages/torch/distributed/run.py", line 797, in run
elastic_launch(
File "/home/amit_g/scratch/env/llm/lib/python3.11/site-packages/torch/distributed/launcher/api.py", line 134, in call
return launch_agent(self._config, self._entrypoint, list(args))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/amit_g/scratch/env/llm/lib/python3.11/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
"""
I am using the below command
torchrun --nnodes 1
--nproc_per_node 3
--rdzv_endpoint=localhost:1800
./llama_finetuning.py
--enable_fsdp
--use_peft
--peft_method lora
--dataset inhouse_dataset
--batch_size_training 2
--num_epochs 10
--model_name ../llama/models_hf/70B
--pure_bf16
--output_dir ./tmp/70B

@aaekay you would a lot more compute resources to run the 70B model, at least 16 A100x80GB. Given enough compute,  please make use of this PR  to bypass the CPU OOM that you would potentially run into.

It seems be stale, closing this pls feel free to re-open if still seeing the issue.


[Question] Is import * as ... work as indended?

Description
For example, there is a TS file with some exported functions.
When I import it via import * as alphas from '...' and call a function (e.g. alphas.test()), I expect alphas.test() will be generated. However, the transpiled result is alphas:test().
Is that transpiling correct?
Docs (there are no : in function call): Lua module require
Example
See playground

This seems to work as intended. You should write declarations for the functions so TypeScript understands what you are importing: https://typescripttolua.github.io/docs/advanced/writing-declarations/
So in your project you would have a file beta.d.ts:
export declare function test(): void;

Part of the declarations is if function have a 'self parameter' or not, if they do, : will be used, . otherwise. See https://typescripttolua.github.io/docs/the-self-parameter for more information.
If you don't want the self parameter you can update the declarations to:
/** @noSelf */
export declare function test(): void;

Generally if you have an error in your TS (like the red line under beta), you should not expect tstl to produce correct output.

That works, thank you


Feature request: expand password length

Hi, I'm using your RuleBuilder plugin. And it's fantastic!
But I have some problems with the plugin. Is it possible to make password length longer? When using RuleBuilder window it's restricted to 99 chars max. But some of my passwords are longer and it's impossible to define rule for them.
It is easy to solve the problem using field editing: "just change the value "Length" in "Password Rule" field". But when I define password rule for a whole entry group -- it's impossible to edit this property manually.
Please make it possible to define password length longer than 99 chars. Thanks!

ADD.
I tried to solve the problem by myself. I have a several entries for one website (let's say example.com) and I want to define one password rule for them. But they are stored in different groups. Subsequently, it's impossible to define the one rule for them all using "Group Password Rule" function.
I tried to use KeePass references. I defined a special entry (for example, "Example.com Password Rule"), filled it's "Notes" field with "Password Rule" definition for RuleBuilder. And pasted a KeePass reference to the entry's "Notes" field in all of my "example.com" entries. It didn't work. Is it possbile to use KeePass references with RuleBuilder?

For both of those scenarios, I would suggest using a profile instead of a rule. You can define a profile that produces a password longer than 99 characters, and you can link an entry to a profile using the Rule Builder so that new passwords for that entry will be generated from that profile.

Thanks for your answer.
Yes, it is possible solution. But in a case of different PC's with different KeePass installations it's inconvenient. I have to sync password profile between KeePass installations manually. Maybe portable version will help, but I'm not sure about different OS (for example, KeePass for Win on my Home PC, and KeePass Mono on Ubuntu Work Laptop).
Suggestion in my previous message allows to define password rule as a part of KeePass Database. And consequently, the password rule becomes portable.
Anyway, thank you! For your answer, and for your work!


Allow only valid group by SQL^2 queries

SQL semantics allow for one of the following three entities to be selected in a query with a group by:
(1) the group key
(2) a reduction
(3) a constant value
For example
select count(pop), 42 as foo, state from zips group by state

is valid SQL while
select count(pop), 42 as foo, state, loc from zips group by state

is invalid SQL because zips.loc is not a group key (nor a reduction nor a constant).
Quasar should error in this case, giving a helpful message to the user. Currently this is allowed, and the query is evaluated by the connector.
EDIT: I originally stated we would implement this in the parser, but there is likely another better phase.

Why should we do this, since SQL2 has a sane semantic for it? It forbids useful classes of queries.
For example:
SELECT sales / SUM(sales) AS percent FROM orders

Also if we wanted to do this, I'd say the parser is probably not the best place for it, because it requires a dimensionality check. It would be better suited for an analysis phase on the parsed AST.

@jdegoes your example doesn't include an explicit grouping and so it wouldn't be affected by my proposal.
What is the sane semantic for the following query (which would be disallowed under my proposal):
SELECT City, Country from Customers group by Country;

The only possibly sane semantic that I see is to arbitrarily choose a value for City for each unique Country. If that value is truly arbitrary, then the same query could return different results each time, even when run on the same connector, unless we require connectors to implement this arbitrary value consistently.
This query is disallowed in SQL because it "does not include the specified expression City as part of an aggregate function". So this would be an extension of SQL^2, which is not in and of itself a problem. I'm just not convinced this is an extension that makes sense.

(Also, I agree the parser may not be the correct place to do this.)

@alissapajer This query is the same as SELECT City, Country from Customers. In QScript, you build an extra layer of dimensionality, but then post SELECT that ends up being stripped off as only values are returned.
Another interesting example is, SELECT City, SUM(Population) from Customers GROUP BY Country, which has as many results as Customers and with columns with a constant population per country. This query may or may not do what you want, but it's still sensible.

@jdegoes hm, right, so considering your example
SELECT City, SUM(Population) from Customers GROUP BY Country

we use the group key to compute the reduction, and then return each value City paired with the population of its Country. This does appear to be a useful query, and avoids the problem I had with selecting an arbitrary value for City.
Still thinking about if I'm convinced this approach (or any other) is consistent.

@jdegoes It's valid to define semantics for those sorts of invalidly grouped queries, but those semantics are definitely incompatible with SQL.  Also, because of the semantics of SQL, you can generate queries which have non-deterministic results if you choose this path.

@jdegoes consider
SELECT Country from Customers group by Country

If the grouping information is ignored entirely when we compute the select, wouldn't this query return a set the size of Customers instead of a single row per unique value of Country (which is what I think it should return).
Furthermore, given the query
SELECT Country, SUM(Population) from Customers group by Country

I would expect a single row per Country. But if we are only incorporating the grouping information to compute the reduction, then we must return results the size of the original Customers, with multiple rows per Country, each paired with that Country's population. To be clear, is this what you are proposing?
I don't see how selecting the group key (in this case Country) can in any way be "special" given your proposal. The grouping adds provenance information, but that's only used in the reduction. So do you think that selecting the group key should be treated that same as selecting any other non group key value?

An analogue would be SELECT foo[_] FROM bar. This query returns identical results to SELECT foo[*] FROM bar. Some constructs, including GROUP BY, add dimensional information that, if not utilized by subsequent reduction, is pretty useless.

I would expect a single row per Country. But if we are only incorporating the grouping information to compute the reduction, then we must return results the size of the original Customers, with multiple rows per Country, each paired with that Country's population. To be clear, is this what you are proposing?

If you utilize a naive translation into MRA, then yes, these are the semantics you get. Both Country and SUM(Population) are grouped by Country, so they auto-join on Country prior to the OBJECT_CONCAT that merges {"Country": ...} and {"1": ...}. Because there are as many {"Country": ...} entries as rows in the original data set, you would get results the size of the original Customers.
However, the logical plan compiler inserts ARBITRARY around Country because it sees the same projection is used as a group by key and a selected projection. So the query you actually get is the following:
SELECT ARBITRARY(Country), SUM(Population) FROM Customers GROUP BY Country

In other words, you get SQL semantics and not "MRA semantics" in this case purely because our logical plan compiler detects the pattern and inserts the ARBITRARY there.

I don't think we should use MRA semantics for group by queries, because the results will not be what the user expects and the user will be very confused. This would be a huge change that would break many existing customers queries. If we want MRA grouping, I think we should introduce second alternative grouping notion with different syntax that represents MRA grouping semantics. I see the value in MRA grouping semantics, but I think we also need to provide regular group by semantics that are equivalent to SQL's.

@alissapajer I am saying we already do use MRA semantics for GROUP BY and that the user only gets what they expect because of a special-case in the logical plan compiler.
If that code isn't broken, then it should yield the same result for any class of query accepted by SQL.

@jdegoes :+1: I had misunderstood and thought that we were discussing removing the Arbitrary special case so that SQL^2 would redefine group by entirely, rather than extend group by like we do now.
Let's keep this open for now as I (and maybe @djspiewak?) are still thinking about the details of the extension and are considering if there are any meaningful counterexamples.

@djspiewak I believe we have resolved this discussion as we've developed qsu, so I'm closing. Reopen if you disagree!


Review dungeon stars and normal stars new special chars (➍➎)

Stars now have new special chars:
➍➎ ... probably 1,2,3 too and idk what happens after 5 (crimson go up to 15 stars)
dump of items with stars from 15h:
§dAncient Necron's Leggings §6✪§6✪§6✪§6✪§6✪§c➎
§dAncient Necron's Chestplate §6✪§6✪§6✪§6✪§6✪§c➎
§dRenowned Warden Helmet
§dSuspicious Astraea §6✪§6✪§6✪§6✪§6✪§c➍
§6Odd Wither Cloak Sword §6✪§6✪§6✪§6✪§6✪
§6Necrotic Wither Goggles §6✪§6✪§6✪§6✪§6✪§c➎
§cAncient Diamond Necron Head §6✪§6✪§6✪§6✪§6✪§c➎
§dRenowned Werewolf Helmet §6✪§6✪§6✪§6✪§6✪
§dWise Young Dragon Helmet §6✪§6✪§6✪§6✪§6✪
§dGiant Young Dragon Helmet §6✪§6✪§6✪§6✪§6✪
§dLoving Storm's Chestplate §6✪§6✪§6✪§6✪§6✪§c➎
§dAncient Storm's Chestplate §6✪§6✪§6✪§6✪§6✪§c➍
§dRenowned Werewolf Chestplate §6✪§6✪§6✪§6✪§6✪
§dWise Young Dragon Chestplate §6✪§6✪§6✪§6✪§6✪
§dGiant Young Dragon Chestplate §6✪§6✪§6✪§6✪§6✪
§dNecrotic Storm's Leggings §6✪§6✪§6✪§6✪§6✪§c➎
§dAncient Storm's Leggings §6✪§6✪§6✪§6✪§6✪§c➎
§dRenowned Werewolf Leggings §6✪§6✪§6✪§6✪§6✪
§dWise Young Dragon Leggings §6✪§6✪§6✪§6✪§6✪
§dGiant Young Dragon Leggings §6✪§6✪§6✪§6✪§6✪
§dNecrotic Storm's Boots §6✪§6✪§6✪§6✪§6✪§c➎
§dAncient Storm's Boots §6✪§6✪§6✪§6✪§6✪§c➍
§dRenowned Werewolf Boots §6✪§6✪§6✪§6✪§6✪
§dWise Young Dragon Boots §6✪§6✪§6✪§6✪§6✪
§dGiant Young Dragon Boots §6✪§6✪§6✪§6✪§6✪
§dNecrotic Precursor Eye §6✪§6✪§6✪§6✪§6✪
§dRenowned Superior Dragon Helmet §6✪§6✪§6✪§6✪§6✪ ✦
§dAncient Frozen Blaze Chestplate §6✪§6✪§6✪§6✪§6✪
§dRenowned Superior Dragon Chestplate §6✪§6✪§6✪§6✪§6✪
§dAncient Frozen Blaze Leggings §6✪§6✪§6✪§6✪§6✪
§dRenowned Superior Dragon Leggings §6✪§6✪§6✪§6✪§6✪
§6Necron's Leggings §6✪§6✪§6✪
§6Maxor's Leggings §6✪
§dAncient Frozen Blaze Boots §6✪§6✪§6✪§6✪§6✪
§dRenowned Superior Dragon Boots §6✪§6✪§6✪§6✪§6✪
§6Storm's Boots §6✪
§dAncient Maxor's Boots §6✪§6✪§6✪§6✪§6✪§c➎
§6Goldor's Boots §6✪
Custom Resources loading done. (worker1)
custom_resources_w1: 9.989s
§dFabled Valkyrie §6✪§6✪§6✪§6✪§6✪§c➎
§dWithered Hyperion §6✪§6✪§6✪§6✪§6✪§c➍
§dDirty Soul Whip §6✪§6✪§6✪§6✪§6✪
§dWithered Flower of Truth §6✪§6✪§6✪§6✪§6✪
§9Heroic Frozen Scythe §6✪§6✪§6✪§6✪§6✪
§dHeroic Yeti Sword §6✪§6✪§6✪§6✪§6✪
§dFabled Aspect of the Dragons §6✪§6✪§6✪§6✪§6✪
§6Precise Sniper Bow §6✪§6✪§6✪§6✪§6✪
§dWithered Giant's Sword §6✪§6✪§6✪§6✪§6✪§c➍
§dWithered Livid Dagger §6✪§6✪§6✪§6✪§6✪
§dPrecise Mosquito Bow §6✪§6✪§6✪§6✪§6✪
§dPrecise Runaan's Bow §6✪§6✪§6✪§6✪§6✪
§6Treacherous Phantom Rod §6✪§6✪§6✪§6✪§6✪
§6Withered Reaper Scythe §6✪§6✪§6✪§6✪§6✪
§6Spiritual Juju Shortbow §6✪§6✪§6✪§6✪§6✪
§dRenowned Superior Dragon Helmet §6✪§6✪§6✪§6✪§6✪
§dRenowned Superior Dragon Chestplate §6✪§6✪§6✪§6✪§6✪
§dRenowned Superior Dragon Leggings §6✪§6✪§6✪§6✪§6✪
§5Necrotic Spirit Mask §6✪§6✪§6✪§6✪§6✪
§9Bonzo's Mask §6✪§6✪§6✪§6✪§6✪
§6⚚ Ancient Adaptive Helmet §6✪§6✪§6✪§6✪§6✪
§6⚚ Ancient Adaptive Chestplate §6✪§6✪§6✪§6✪§6✪
§6⚚ Ancient Adaptive Leggings §6✪§6✪§6✪§6✪§6✪
§6⚚ Ancient Adaptive Boots §6✪§6✪§6✪§6✪§6✪
§6Ancient Tarantula Helmet §6✪§6✪§6✪§6✪§6✪
§6Ancient Frozen Blaze Helmet §6✪§6✪§6✪§6✪§6✪
§6Light Maxor's Chestplate §6✪§6✪§6✪§6✪§6✪
§6Light Maxor's Leggings §6✪§6✪§6✪§6✪§6✪
§6Shiny Storm's Leggings §6✪§6✪
§6Shiny Storm's Boots §6✪
§9⚚ Giant Bonzo's Mask §6✪§6✪§6✪§6✪§6✪
§5Ancient Tarantula Boots §6✪§6✪§6✪§6✪§6✪
§dGiant Reaper Mask §6✪§6✪§6✪§6✪§6✪ ✦
§dGiant Necromancer Lord Chestplate §6✪§6✪§6✪§6✪§6✪
§dGiant Necromancer Lord Leggings §6✪§6✪§6✪§6✪§6✪
§dGiant Necromancer Lord Boots §6✪§6✪§6✪§6✪§6✪
§dGiant Goldor's Leggings §6✪§6✪§6✪§6✪§6✪
§5Warped Mineral Helmet
§5Warped Mineral Chestplate
§5Warped Mineral Leggings
§5Warped Mineral Boots
§5Necrotic Spirit Boots §6✪§6✪§6✪§6✪§6✪
§cAncient Diamond Professor Head §6✪§6✪§6✪§6✪§6✪
§cAncient Diamond Thorn Head §6✪§6✪§6✪§6✪§6✪§c➍
§cAncient Diamond Livid Head §6✪§6✪§6✪§6✪§6✪
§cAncient Diamond Sadan Head §6✪§6✪§6✪§6✪§6✪§c➍
§fRogue Sword §6✪§6✪§6✪§6✪§6✪
§dNecrotic Wise Dragon Helmet §6✪§6✪§6✪§6✪§6✪
§dHeroic Spirit Sceptre §6✪§6✪§6✪§6✪§6✪
§dHeroic Reaper Scythe §6✪
§dAncient Storm's Helmet §6✪§6✪§6✪§6✪§6✪
§6Precise Spirit Bow §6✪§6✪
§dRenowned Superior Dragon Helmet §6✪§6✪§6✪§6✪§6✪ ✦
§6Midas' Sword §6✪§6✪§6✪
§6Spiritual Bonemerang §6✪§6✪§6✪§6✪§6✪
§6Spiritual Bonemerang §6✪§6✪§6✪§6✪§6✪
§6Spiritual Bonemerang §6✪§6✪§6✪§6✪§6✪
§6Spiritual Bonemerang §6✪§6✪§6✪§6✪§6✪
§6Spiritual Bonemerang §6✪§6✪§6✪§6✪§6✪
§6⚚ Last Breath §6✪§6✪§6✪§6✪§6✪
§9⚚ Heroic Bonzo's Staff §6✪§6✪§6✪§6✪§6✪
§6Heroic Jerry-chine Gun §6✪§6✪§6✪§6✪§6✪
§dWithered Dark Claymore
§dHasty Terminator §6✪§6✪§6✪§6✪§6✪§c➎
§dBulky Florid Zombie Sword §6✪§6✪§6✪§6✪§6✪
§6Heroic Ice Spray Wand §6✪§6✪§6✪§6✪§6✪§c➍

The default character can't be seen on skycrypt:

the 4 is invisible... so we need custom icomoon icon for these 😑

https://codepoints.net/dingbats

Update:
Normal stars are represented as always, dungeon stars instead are now a number (in game it's red), representing the number of dungeon stars applied to the item:
➊ U+278A
➋ U+278B
➌ U+278C
➍ U+278D
➎ U+278E

Example, an Hyperion with 5 stars and 3 dungeon stars will be:
Hyperion ✪✪✪✪✪➌


Address command line injection in Git::Lib#fetch

Signed-off-by: James Couball<EMAIL_ADDRESS>Your checklist for this pull request
🚨Please review the guidelines for contributing to this repository.

[X] Ensure all commits include DCO sign-off.
[X] Ensure that your contributions pass unit testing.
[X] Ensure that your contributions contain documentation if applicable.

Description
This fixes issue #568

The "Ruby head on ubuntu-latest" build fails because of a problem with YARD.  This issue has been fixed in YARD's head so that a new version of YARD will fix this problem.

Great!
Is it pull request related to CWE-88, right?


Postgis in qfieldcloud

How to use postgis in qfieldclod. Is there a tutorial?
It is announced but is it functional?

@jitendrashahiitb please follow the documentation here how to add an existing PG database to your account:
https://docs.qfield.org/reference/qfieldcloud/secrets/#pg_service-configuration
QFieldCloud hosted PG database can be enable on request via email, but is not planned to be released to the general public soon.

Dear Mr Ivan Ivanov
Thanks for clarification that an email request is necessary as of now.
I hereby request you to enable postgis facility.
If this is not enough, can you please tell us how to specify request.
NB : The link you sent (
https://docs.qfield.org/reference/qfieldcloud/secrets/#pg_service-configuration
)
does not open. says 'We're having a really bad day.**'
regards
On Thu, Aug 18, 2022 at 12:38 PM Ivan Ivanov @.***>
wrote:

@jitendrashahiitb https://github.com/jitendrashahiitb please follow the
documentation here how to add an existing PG database to your account:
https://docs.qfield.org/reference/qfieldcloud/secrets/#pg_service-configuration
QFieldCloud hosted PG database can be enable on request via email, but is
not planned to be released to the general public soon.
—
Reply to this email directly, view it on GitHub
https://github.com/opengisch/qfieldcloud/issues/402#issuecomment-1219115400,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABFXS6LMA5ZN5TACBXYSMKLVZXONHANCNFSM5632T3NQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
jitendra

By the way, where did you see that QFieldCloud provides PostGIS hosting? Can you give us a link?

Dear Ivan,
Looks like we misunderstood the facility as mentioned in attached page
about Pro version with payment.
It is not that postgis is hosted by opengisch but we have to host it.
We stand corrected.
Thanks for clarification
Cheers
On Thu, 18 Aug, 2022, 22:12 Ivan Ivanov, @.***> wrote:

By the way, where did you see that QFieldCloud provides PostGIS hosting?
Can you give us a link?
—
Reply to this email directly, view it on GitHub
https://github.com/opengisch/qfieldcloud/issues/402#issuecomment-1219708518,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABFXS6LI47VY5LJZHP2ZCX3VZZRWZANCNFSM5632T3NQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

I am sorry to say that we are required to discontinue use of qfieldcloud .
We were very happy for a foss project . We are ready to pay a reasonable
price for a public project.
But technically we found it too cumbersome if not problematic.
Will look forward to a better scenario.
Cheers
On Thu, 18 Aug, 2022, 12:38 Ivan Ivanov, @.***> wrote:

@jitendrashahiitb https://github.com/jitendrashahiitb please follow the
documentation here how to add an existing PG database to your account:
https://docs.qfield.org/reference/qfieldcloud/secrets/#pg_service-configuration
QFieldCloud hosted PG database can be enable on request via email, but is
not planned to be released to the general public soon.
—
Reply to this email directly, view it on GitHub
https://github.com/opengisch/qfieldcloud/issues/402#issuecomment-1219115400,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABFXS6LMA5ZN5TACBXYSMKLVZXONHANCNFSM5632T3NQ
.
You are receiving this because you were mentioned.Message ID:
@.***>


tests: Remove ntop regression test

Been very flakely in CI recently. Type::ctx has also been removed a
while ago so what the test was originally testing for is probably not
going to happen again.

Checklist

[ ] Language changes are updated in man/adoc/bpftrace.adoc and if needed in docs/reference_guide.md
[ ] User-visible and non-trivial changes updated in CHANGELOG.md
[ ] The new behaviour is covered by tests

I'm tired of the CI being flakey so I'm merging this. If anyone has concerns we can revisit after


Allow TextStyle.lerp to lerp TextStyles with different inherit settings

Description
Currently the TextStyle.lerp function asserts if the two TextStyles do not have the same inherit field. It is a not exactly clear what the outcome of a lerp between two styles that where one is inheriting part of its properties from its parents and the other isn't should be. However, this does lead to the case were certain TextStyle combinations can't be lerp'd which can cause failures when trying to lerp between our default Themes. (See #43358).
We already allow for some weirdness in this when lerping between two styles that have different sets of fields set. It would be better to allow this with some possibly weird intermittent results rather than failing altogether. This PR just allows the function to lerp the 'inherit' field like we do other boolean fields (i.e. swap them at 0.5).
Related Issues
Fixes: #43358
Tests
I added a test to the existing TextStyle unit tests to verify that lerp works with different inherit settings.
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I signed the CLA.
[x] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[x] I updated/added relevant documentation (doc comments with ///).
[x] All existing and new tests are passing.
[x] The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require Flutter developers to manually update their apps to accommodate your change?

[x] No, this is not a breaking change.

This will lead to really ugly transitions. I think we should instead fix our defaults so this doesn't become a problem.

Closing until I can come up with a better solution.


Update url references for storage class name used at pmem-pvc.yaml

Signed-off-by: Morales Quispe, Marcela<EMAIL_ADDRESS>

Please don't merge. These files are going to be generated differently in PR #234


convolution bwd data is very slow in jit_gemm_convolution path

Hi, our workload is unbalance on jit_gemm_convolution_bwd_data path, we have a<PHONE_NUMBER> input need to be run on googlenet, batch size =2 , input channel =3. We find that the first conv layer's backward data goes into jit_gemm_convolution path, and it will fork two thread to do the sgemm and col2img(workamount = ngroupnbatchsize), per thread has to deal with 322402240 input. How ever, our cpu is Xeon phi knl, it has 68 cores, other 66 cores are idel during this time.So it is very efficient. How can we solve it? or is it possible to make it run through jit_avx512/jit_avx2 path? format has been set any now, Thanks

is there some workaround that we can make it run on avx2 / avx512 path. the only insufficient condition is our stride is 2., pad is 3, channel is 3

Current implementation does not have jit codepath for backward convolution by data in case of 3 channels.  So your case runs GEMM-based convolution which is threaded by minibatch currently and with minibatch 2 only 2 threads are used. To get better balancing you can increase minibatch and make it a multiple of number of cores.
In general convolution with 3 channels is used as the first layer in the topology and there's no need to compute gradient. In case of GoogleNet in particular backward by data for the first convolution is not computed. Is there a particular reason why you want to compute backward by data in this scenario?

Thanks for the quick response! Our framwork's graph is dynamic so its hard to know who is first layer. We will try to optimize it to make the first layer be recognized. Thanks


Cross-compile wheels for linux/aarch64 using qemu

Too slow; takes more than 20 mins to build in qemu. Closing.


Add more hooks

I was trying to use git hooks to take some actions on a database during changing a branch (checkout), creating a branch (branch), deleting branch (branch -d) etc. Is there a specific reason why such hooks are not available? It would be very handy to have them!

Because they simply don't exist. Here's a complete list of hooks: https://githooks.com

@ciembor Yes, I think you should ask the git team for implementing those hooks.


add remember me credentials

add remember me functionality

when remember me switch on then after logout schoolid & password auto save
when remember me switch off then schoolid & password field should be empty
need to check by switch school

when we login with first school and try to on the remember button in screen it is saving first school credentials. when i do logout from school and again go back to login page i am able to see first school data already filled that is the expected working of remember  me credentials . When I try to login with second school my remember button is already on . So we need to overwrite first school credentials with second school . This is currently missing in this pr .

i have changed in functionality like if we login with first school and do remember me on then after logout it will auto save ,
if we login with second school then after logout fileld will empty because credential save for first school , now fill fields for school 2 and do remember me on after logout you will see for school 2 credential @apurvabisht97 @dileep-gadiraju

after logout of second school it should not be empty it should save second school data . @navin9584

hi @dileep-gadiraju  i have done testing of above pr . it is working as expected .


⚠ ClamAV has degraded performance

In 5f420cd, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 202 ms

Resolved: ClamAV performance has improved in 46de3d7.


Parse route query parameters for use in component props

Routes that use query parameters i.e. /search?text=bob can now be passed down as component props if configured in routes.json
This does not interfere with any data or fetch configurations i.e. /products/123?price=usd can still make fetch requests using the url parameter and pass the query parameters as props.
In the future it may make sense to add the ability to use query parameters as fetch parameters.

Thanks for the PR! Unfortunately, it's no longer compatible with the latest version.


[staging & production] Education and employment drag to reorder does not work for position 1

Steps

Log in and go to the user settings page
Go to the employment or education tab
If you don't already have more than one position, create a new position
Drag position 2 above position 1
Attempt to drag position 1 below position 2
If you have more than two positions, try to drag position 2 below position 3

Expected
Since all positions say [drag to reorder], I assume that all positions are draggable and can thus be dragged above or below any other position to reorder. That is, I expect to be able to drag position 1 below position 2 to reorder.
Actual
Dragging one position below another does not permit reordering. The position kind of just sits on top of the other (text overlaps) and then goes back to the original positions.

I'll take this issue, since I already have a fix for it....

@huangginny This is a separate issue. The problem is that dragging the first item below subsequent items does not work (however, dragging other items works fine).

I'm looking into this.

Thanks!

Closing this issue since the fix has been merged in. :octocat:


feat: Upgrade all Dependencies especially storybook

Now that storybook 7 is finally released, running npm-check-updates to update everything

Switched rollup plugin to non deprecated one
Switched stylelint plugin to non depreciated one

enabled new rules from the new plugin

Upgraded dependencies
Upgraded eslint lit plugin

Warned me that nothing is better than html()

I've been waiting for storybook 7 before enabling any dependency management tooling but so excited