OptimizerArgs

This PR unbundles the OptimizerArgs approach from GaLore in #1192.

Your jsonargparse example has been super helpful for understanding things a bit more @carmocca . Many thanks for this!
But maybe it's because it's Fri evening but my brain is just not working today. I've just been banging my head against how I would get this into the finetuning method's setup.
Adding the optimizer subclass to the parser and then calling the finetune command yields a
  File "/home/zeus/miniconda3/envs/cloudspace/bin/litgpt", line 8, in <module>
    sys.exit(main())
  File "/teamspace/studios/this_studio/litgpt/litgpt/__main__.py", line 145, in main
    fn(**kwargs)
TypeError: setup() got an unexpected keyword argument 'optimizer.class_path'

But we can't add optimizer argument to the  finetuning setup signature because it's a duplicate command then. Conceptually, I am kind of stuck here.
Also, how would we get the args in
optimizer = instantiate_class(model.parameters(), init=args["optimizer"])

if we don't pass them on from the main() function in __main.py__. I suppose if I would do the
parser = jsonargparse.ArgumentParser()
parser.add_subclass_arguments(torch.optim.Optimizer, "optimizer", instantiate=False, fail_untyped=False, skip={"params"})
args = parser.parse_args()

in the finetuning script, but then it would erase all the previous arguments.

As far as integrating into  the scripts, I would:
Create an optimizer argument in https://github.com/Lightning-AI/litgpt/blob/36c6a77435d75872f525848ee1570467d120ae80/litgpt/finetune/lora.py#L40
To avoid the duplicate registration, you need to skip it when the function arguments are added https://github.com/omni-us/jsonargparse/blob/2de15ddfb1c02c2f7b3fe913ad11f13c5cb65dff/jsonargparse/_signatures.py#L166 https://github.com/Lightning-AI/litgpt/blob/main/litgpt/main.py#L121
And call instantiate_class here https://github.com/Lightning-AI/litgpt/blob/36c6a77435d75872f525848ee1570467d120ae80/litgpt/finetune/lora.py#L185-L187
This should be enough to unblock you. The not-so-nice thing is that the CLI args structure leaks into the actual script, meaning that users who don't go through the CLI will have to create this dictionary manually.

Awesome, thanks so much, this was great help! Figured it out now and got it to work. Many thanks, again learned something new!

I know got it to work as follows:
litgpt finetune full \
  --checkpoint_dir checkpoints/EleutherAI/pythia-160m

# Specify optimizer and optimizer args:
litgpt finetune full \
  --checkpoint_dir checkpoints/EleutherAI/pythia-160m \
  --optimizer  torch.optim.SGD \
  --optimizer.init_args.lr 1000

But I feel like the way I am passing the optimizer kwargs seems a bit hacky. Is this there a built-in/better way to handle it @carmocca ? The thing is that when I pass an --optimizer argument it also passes additional kwargs to the setup:
kwargs = {
    'optimizer.class_path': 'torch.optim.SGD',
    'optimizer.init_args.dampening': 0.0,
    'optimizer.init_args.differentiable': False,
    'optimizer.init_args.foreach': None,
    'optimizer.init_args.lr': 0.001,
    'optimizer.init_args.maximize': False,
    'optimizer.init_args.momentum': 0.0,
    'optimizer.init_args.nesterov': False,
    'optimizer.init_args.weight_decay': 0.0
}

That's why I added the parsing into class_path and init_args:
    optimizer_class_path = None
    optimizer_init_args = {}
    for key, value in list(kwargs.items()):
        if key.startswith("optimizer"):
            if "class_path" in key:
                optimizer_class_path = value
            elif "init_args" in key:
                init_arg_key = key.split(".")[-1]
                optimizer_init_args[init_arg_key] = value
            del kwargs[key]

Everything seems to work, but I wonder if there isn't a better way to do it?

@rasbt I pushed a commit with what I would suggest. The str code path could be improved if we want to expose arguments like the learning rate outside of the CLI, but that should be straightforward to implement.
Also fyi, you don't need to specify the .init_args substring through command line

The only caveat now is that the class path still needs to be specified. I.e., only specifying the learning rate doesn't work
litgpt finetune full  --optimizer.lr 200  --checkpoint_dir checkpoints/EleutherAI/pythia-160m

error: Parser key "optimizer":
  Not a valid subclass of Optimizer. Got value: NestedArg(key='lr', val='200')
  Subclass types expect one of:
  - a class path (str)
  - a dict with class_path entry
  - a dict without class_path but with init_args entry (class path given previously)

And the optimizer always needs to be specified explicitely
litgpt finetune full  --optimizer AdamW --optimizer.lr 200  --checkpoint_dir checkpoints/EleutherAI/pythia-160m

Do you know if that's a jsonargparse thing, @carmocca ? Because we already set a default value in the setup method I was thinking that this is a bit weird.

I hope this is ready now @carmocca

The azure failure does look real:
>       fit(fabric, devices, state, train_dataloader, val_dataloader, out_dir, tokenizer_dir, train, eval, optimizer)
E       TypeError: fit() takes 9 positional arguments but 10 were given

/__w/6/s/extensions/thunder/pretrain.py:229: TypeError
----------------------------- Captured stderr call -----------------------------
Missing logger folder: /tmp/pytest-of-root/pytest-0/test_pretrain0/out/logs/tensorboard
Seed set to 42
=========================== short test summary info ============================
FAILED tests/test_thunder_pretrain.py::test_pretrain - TypeError: fit() takes 9 positional arguments but 10 were given

It does. Let me investigate ...

Should be fixed for good now @carmocca . I can switch the link to the original tinystories now that you have seen the green checks haha 😆


[Important] Textdomain issue when updating theme in WordPress repository

Issue description:
Hey guys!
I've just noticed when updating Kirki to v5 in Page Builder Framework, an additional textdomain ('easy-digital-downloads') is being flagged when pushing to the WordPress repository.
It seems to be coming from the commit that merges the pro extensions into the free version - https://github.com/themeum/kirki/commit/fac009be1542324abff3e19b1c1c7257934d1698
For some reason, the file EDD_SL_Plugin_Updater.php was added which I think doesn't belong into the plugin at all.

We used to have the updater in the Kirki Pro Tabs (and others) in earlier commits when we were in charge of Kirki but removed those later on.
Looks to me that an older version of the Kirki Pro Tabs was merged into the free version and released with v5 (instead of the very latest master).
Can you please check?
Thank you! :)
Version used:
5.0

Did a quick comparison between our latest master & the version of Kirki Pro Tabs included in Kirki. From what I can tell, you only need to delete the edd folder and the EDD_SL_Plugin_Updater.php file included in it and we should be fine :)


added missing IS4 Client model fields to internal API Client model

This change is

@michaelvidal24, that's a good question. If we do that, will we still store the IS4 Client?

We would still want to store the IS4 Client model, as that is the 'domain' model. We would perform a conversion between the API model and the domain model in the API layer. This will allow us to iterate on the domain model and api model independently, providing us the ability to keep changes in the domain model from potentially breaking our API contract.


Update readme to explain why all repos should be updated

@chiragsalian @tgolen will you please review this?
This PR updates the readme to highlight why all internal repos that use expensify-common should be updated after a new PR is merged. More context in https://expensify.slack.com/archives/C03TQ48KC/p1636044257067200.
Fixed Issues
Discussion in slack
Tests/QA
N/A

@stitesExpensify updated

Updated the PR to include Comp and clarify not all repos need to be updated

Glad I don't have to update the hash for all the repos now 😂


Better hint for default resource

Now we got:
step "import-disk" validation error: step "import-virtual-disk" validation error: ResourceDoesNotExist: projects/xxxx/global/networks/default does not exist
But actually, there is no "network" input by user. The resource name is a default resource. To make it clear, we added one more sentence for "default" situation:
step "import-disk" validation error: step "import-virtual-disk" validation error: ResourceDoesNotExist: projects/xxxx/global/networks/default does not exist. please provide a specific resource name.

This change should be implemented in export/OVF import wrappers as well.

/gobuild


OLS-906: Benchmarks: real example

Description
Benchmarks: real example
Any benchmark can be write in the same style - as unit test, but with benchmark parameter + the benchmarked
function or method needs to be called by benchmark(function_method, parameters)
Type of change

[ ] Refactor
[x] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library
[ ] Bump-up library or tool used for development (does not change the final image)

Related Tickets & Documents

Related Issue #OLS-906

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.69%. Comparing base (3c71e79) to head (b76ec6f).
Report is 6 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1381   +/-   ##
=======================================
  Coverage   94.69%   94.69%
=======================================
  Files          58       58
  Lines        2505     2505
=======================================
  Hits         2372     2372
  Misses        133      133

/override "Red Hat Konflux / lightspeed-service-on-pull-request"

/lgtm
/approve

/approve


[FEATURE] OnScroll Zoom in/out on Map

Description
Add OnScroll Zoom In/Out event on the Map just like Google Maps does it.

OnScroll Wheel Forward : Map zoom IN
OnScroll Wheel Backward : Map zoom OUT

Screenshots

Additional information
This will be good with respect to User Experience.

@amandamartin-dev what are your thoughts on this?

@amandamartin-dev what are your thoughts on this?

I think this is fine, truly I don't remember disabling this ability but I just checked the code and it's set to false.
You will just want to change this to true in the map component and then test to make sure it still adheres to the map bounds which it should
 scrollWheelZoom={false}

@Rohanfulzele I am going to change to ready for dev, do you want to be assigned this issue?


update bot token

See issue #177 or discussion of these changes.

may be able to replace
#182
however, this can only really be tested by merging it, as it will only run and use the token on master
if people are uncomfortable with this, I can make a branch on the main repository to prove the implementation, and make it push to another branch, but this is more work

to keep the implementation clean, it would be good to remove the CI Environment variable changes made to facilitate
#182 as these are re-using environment variables, which could then conflict
I'm happy to do this before merge, if someone aims to merge this
consideration of this approach compared to #182 is useful, I feel
I hope this is helping

I'll take care of it and go ahead and merge here. It's not going to get any more broken.

 	cf-conventions.html 	Auto-generated from a884f3e 	14 seconds ago
	cf-conventions.pdf 	Auto-generated from a884f3e 	14 seconds ago

I believe we are cooking
:)

thank you @dblodgett-usgs


[BUG] Flakey test failure in SortedSetDVStringFieldDataTests testSortMissingLast

Describe the bug
First observed in https://github.com/opensearch-project/OpenSearch/pull/4183#issuecomment-1215586520, but I hit the same failure several times in #4222 and was able to reproduce it locally:
REPRODUCE WITH: ./gradlew ':server:test' --tests "org.opensearch.index.fielddata.SortedSetDVStringFieldDataTests.testSortMissingLast" -Dtests.seed=BC7C1431A4F70C5D -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ga-IE -Dtests.timezone=Europe/Belgrade -Druntime.java=17

org.opensearch.index.fielddata.SortedSetDVStringFieldDataTests > testSortMissingLast FAILED
    java.lang.AssertionError: expected:<1105> but was:<1023>

To Reproduce
See above
Expected behavior
Test consistently passes
Plugins
None
Screenshots
N/A
Host/Environment (please complete the following information):

OS: [e.g. iOS]
Version [e.g. 22]

Additional context

@nknize are you poking at this?

Yes. It's new as of the Lucene 9.4 snapshot PR. Only reproducible if you run the entire Test class, not the individual case.

https://github.com/opensearch-project/OpenSearch/pull/4066

Suspect there was a regression in #4183 in changes to BytesRefFieldComparatorSource.java

Some more investigation.  The number of documents in the search impacts the failure.  The random flakiness is introduced by this line, where it succeeds half the time:
https://github.com/opensearch-project/OpenSearch/blob/05a58192433f6e007ade61b4b4f21d0f068d50e0/server/src/test/java/org/opensearch/index/fielddata/AbstractStringFieldDataTestCase.java#L282
The failures occur in an edge case where the last element of the document list is null.  In the repro case listed above, there are 1105 documents, but the last one is null, and any number of documents 1103 or below fails (both 1104 and 1105 succeed).

From Lucene's IndexSearcher javaDoc:

NOTE: The search(org.apache.lucene.search.Query, int) and searchAfter(org.apache.lucene.search.ScoreDoc, org.apache.lucene.search.Query, int) methods are configured to only count top hits accurately up to 1,000 and may return a lower bound of the hit count if the hit count is greater than or equal to 1,000. On queries that match lots of documents, counting the number of hits may take much longer than computing the top hits so this trade-off allows to get some minimal information about the hit count without slowing down search too much. The TopDocs.scoreDocs array is always accurate however.

This conflicts with the assertion here:
https://github.com/opensearch-project/OpenSearch/blob/05a58192433f6e007ade61b4b4f21d0f068d50e0/server/src/test/java/org/opensearch/index/fielddata/AbstractStringFieldDataTestCase.java#L285
The value of topDocs.totalHits.value is not reliable if it is above 1000.  (If you output topDocs.totalHits in this case you will get the String 1001+.)
This test needs to be adjusted to account for this possible inaccuracy.


New rates for Greece (EL/GR)

As per page 25 of http://ec.europa.eu/taxation_customs/resources/documents/taxation/vat/how_vat_works/rates/vat_rates_en.pdf

Thanks @jokkedk for the correction!


save graph to geopackage

fixes #411

Coverage decreased (-0.4%) to 92.744% when pulling 32572f81c98f18f929b40a8668de1f9d003a1687 on gpkg into eeb1e76d49a14a4ed9ce481514e1e0f394e9fae0 on master.

@swiss-knight this has been merged into the master branch now if you wouldn't mind testing it out

Small review
I have quickly tested to write my downloaded graph which is of type:
<class 'networkx.classes.multidigraph.MultiDiGraph'>
G_raw is_directed ?: True

to a GeoPackage as well as Shapefiles.
Both operations succeeded.
Execution time
Execution times (for the saving operation only) are roughly the same (~40s on my old laptop).
Size
Sizes ; ~76 MB for both Shapefiles' folders and 8.9 MB for the GeoPackage single file.
I'm always astonished (!) by the difference in sizes, that's cool.
Number of features
In both the nodes and the edges tables, number of features written are the same in case of the Shapefile and the GeoPackage.
Fields of the attribute table and geographical position of the features
For the nodes, once opened in QGIS they show the same attributes and every feature is at the same geographical position.
There are extra x, y and fid columns in the attribute table in case of the GeoPackage, which may be normal I think (?).
For the edges, once opened in QGIS they show the same attributes (with full non 10-char truncated names in case of the GeoPackage \o/ !) and every feature is at the same geographical position.
There are extra u, v and fid columns in the attribute table in case of the GeoPackage, which again may be normal I think (?).
NULL fields
For the edges, some fields are written "NULL" in the attribute table of the Shapefile, but not in the GeoPackage attribute table, they are just left empty.
While for the nodes, features of the only column 'highway' in both the Shapefile and the GeoPackage are written "NULL".  It may be a normal behavior again, I don't know, I just noticed that tiny difference.
Overwriting existing file
I have tested both writing the GeoPackage from scratch, and overwriting the existing file.
No differences to notice, everything went fine.
During the write process, I sometime figured out extra files laying around with fancy names (one ending by journal), but they are temporary and go away at the end.
Writing methods used
I've tested both;
ox.save_graph_geopackage(G, filename=os.path.join(gpkg_dir,'raw_graph.gpkg'), encoding='utf-8')
and
ox.save_graph_geopackage(G, folder=gpkg_dir, filename='raw_graph.gpkg'), encoding='utf-8')
with success.

Thanks!

There are extra x, y and fid columns in the attribute table in case of the GeoPackage, which may be normal I think (?).

Yes, fid is the default name of the primary key in the SQLite database: https://gdal.org/drivers/vector/gpkg.html#layer-creation-options


Make javascript injection easier for things like cookies consent, or others

Is your feature request related to a problem? Please describe.
Sometimes it's required to add additional javascript scripts to the Educates Platform to allow for things like cookie consent due to GDPR laws. Currently you can add these to a theme, but it's maybe overly complicated.
Describe the solution you'd like
Investigate what kind of javascripts would need to be injected (also if CSPs would need to be modified) and provide an easy way to set these up. An example of scripts could be https://cookieconsent.orestbida.com/
Describe alternatives you've considered
No response
Additional information
Current Kubernetes slack thread with some information about this: https://kubernetes.slack.com/archives/C05UWT4SKRV/p1715589756629829

First possible change suggest is that currently you can specify Secret resource with theme files.
apiVersion: v1
kind: Secret
metadata:
  name: workshops.example.com-theme
  namespace: default
stringData:
  workshop-dashboard.html: ""
  workshop-dashboard.css: ""
  workshop-dashboard.js: ""
  workshop-instructions.html: ""
  workshop-instructions.js: ""
  workshop-instructions.css: ""
  workshop-started.html: ""
  workshop-finished.html: ""
  training-portal.html: ""
  training-portal.js: ""
  training-portal.css: ""

The .html file is embedded directly in head section of pages.
The .css file is included into the head section using a link element.
The .js file is included at the bottom of the HTML body element using a script element.
Right now everything has to be in the one file which is not convenient when needing to include multiple files from third party packages along with your own customisations.
To enable having multiple files, have convention whereby for example in addition to training-portal.html, you can have files training-portal-nnn.html where nnn is integer number. Where such files exist they will be embedded/included at appropriate point in sorted order.
Thus Secret might be defined as:

apiVersion: v1
kind: Secret
metadata:
name: workshops.example.com-theme
namespace: default
stringData:
training-portal-1.html: "..."
training-portal-2.html: "..."
training-portal-1.js: ""
training-portal-2.js: ""
training-portal-1.css: ""
training-portal-2.css: ""

If variant with no number is still provided, it would be embedded/included before the numbered ones.

Steps for how to use current theme support to add CookieConsent package.

https://github.com/educates/labs-educates-showcase/blob/develop/workshops/lab-cookie-consent/workshop/content/00-workshop-overview.md

So the idea of numbered files for stuff needed to be included in head of page is unnecessary since you can use the .html snippet file with link and script elements to include the arbitrary files attached to the secret.
<link rel="stylesheet" href="/static/workshops/theme/cookieconsent.css">
<script type="text/javascript" src="/static/workshops/theme/cookieconsent.umd.js"></script>

The .js file though is still a single entry only which is included using a script tag with src attribute.
The needed for numbered files would not arise for inclusion at the end of the body if option existed for supply arbitrary HTML at the end of the body. That way could use multiple script tags to include separate Javascript files, or just embed Javascript, or even plain HTML if that made sense.

Note that need for including multiple Javascript files at end of body element of page only becomes a requirement where front loading them in the head element is not possible for performance or runtime reasons.

Concern over CSP is that there are different constraint types and need to know which to add them to.
CSP_CONNECT_SRC = (
    "'self'",
    f"*.{INGRESS_DOMAIN}",
    "www.google-analytics.com",
    "*.clarity.ms",
    "c.bing.com",
    "*.amplitude.com",
)

CSP_DEFAULT_SRC = ("'none'",)
CSP_STYLE_SRC = ("'self'",)
CSP_SCRIPT_SRC = ("'self'",)
CSP_SCRIPT_SRC = ("'self'", "www.clarity.ms", "cdn.amplitude.com")
CSP_IMG_SRC = (
    "'self'",
    "data:",
    "www.google-analytics.com",
    "www.googletagmanager.com",
)
CSP_FONT_SRC = ("'self'",)
CSP_FRAME_SRC = ("'self'",)
CSP_INCLUDE_NONCE_IN = ("script-src",)
CSP_FRAME_ANCESTORS = ("'self'",)

Yeah I noticed this when going through the Django code - would it be possible/feasible to expose all CSP_* variables used in Django for configuration in e.g. the TrainingPortal definition?

I see. This definitely makes sense. I then share your concerns that hostnames passed to the portal's config might end up in the wrong/too many CSPs out of engineering necessity.
I'd say then, an easier way to inject Javascript would be a good start.
In addition, I'll create a PR to document the use-case of 'including multiple additional .js/.css files and referencing them in the .html snippet file of a theme' a bit better, as this looks like the best solution to me right now.

@mocdaniel Even with CookieConsent injected, how were you then enabling/disabling analytics as necessary?
In https://cookieconsent.orestbida.com/advanced/manage-scripts.html it talks about using data attributes to mark script sections which should be enabled/disabled based on consent, but that relies on setting script type to text/plain which means wouldn't run if CookieConsent not also be used.
Trying to work out whether anything else needs to be done to make existing analytics code injection work better with any cookie consent libraries.

I inserted the same snippets you insert upon setting up Google Analytics via G-Tag by adding new entries to a custom theme Secret and referencing them from the training-portal.html key within the same secret.
This way I was able to adjust the scripts according to the CookieConsent docs you linked above.


Ship a change so that Office 365 Groups can accept email.

@nenonix @JohanFreelancer9 - kindly do the copy editor review for this PR. Thanks.

@IrenePasternack - PR has been copy edited and is ready for final review.
Thank you.


Add compatibility for keywords

I don't have a good idea what we could do here. Renaming keywords makes testing with the dask test suite a PITA, but I also don't want to introduce too much debt.
We might want to consider a deprecation period in dask/dask instead so that we can switch over fresh to dask-expr when we enable dask-expr by default?

Are there keywords aside from shuffle= that this protects against?  Why is there a mismatch between dask/dask and dask-expr in this case?

Only shuffle for now,
Some keywords were renamed when re-implementing stuff in dask-expr, most of them happened before my involvement, so not sure about the reasoning (that said I generally in favour of cleaning up names), cc @rjzamora

My recommendation would be to just align dask-expr to dask/dask for these cases for now and use the names that we don't like, making a note of what we want to change.  Then when we flip the switch we make a bunch of changes back to the names that we like.  (better to do one big change than two medium changes)


setDisplayMatrix() does not match previous state after  zoom scale and translate

I zoom in and translate the photoView，then I saved current DisplayMatrix by
Matrix m = photoView.getDisplayMatrix()
After system configuration changed, the PhotoView  re-created, I wish to restore the scale and translate state.
I called photoView.setDisplayMatrix(m), but it does not match before.

Have you got the solution ?

I zoom in and translate the photoView，then I saved current DisplayMatrix by
Matrix m = photoView.getDisplayMatrix()
After system configuration changed, the PhotoView re-created, I wish to restore the scale and translate state.
I called photoView.setDisplayMatrix(m), but it does not match before.

How did you zoomin and translate photoview ?


Locally connected layer (convolution layer without weights shared)

I want to build a locally connected layer(just like a convolution layer without weights shared) with mxnet, any ideas?

this is locally connected layer with SliceChannel:
flatten = mx.symbol.Flatten(data=data, name="flatten") slice = mx.symbol.SliceChannel(flatten,num_outputs = 100, name="slicechannel") groups = [] for i in range(100):         fc = mx.symbol.FullyConnected(data=slice[i],num_hidden=1, name = "fc%d" % i)         groups.append(fc) concat = mx.symbol.Concat(*groups) relu = mx.symbol.Activation(data=concat, act_type='relu' ,name = 'relu') softmax = mx.symbol.SoftmaxOutput(data=relu,label = "softmax_label",name='softmax')
and when I trained with this code , it was really very very slow , slower than FullyConnected.why...

this is locally connected layer with SliceChannel:
flatten = mx.symbol.Flatten(data=data, name="flatten")  slice = mx.symbol.SliceChannel(flatten,num_outputs = 100, name="slicechannel")  groups = []  for i in range(100):       fc = mx.symbol.FullyConnected(data=slice[i],num_hidden=1, name = "fc%d" % i)      groups.append(fc)  concat = mx.symbol.Concat(*groups)  relu = mx.symbol.Activation(data=concat, act_type='relu' ,name = 'relu')  softmax = mx.symbol.SoftmaxOutput(data=relu,label = "softmax_label",name='softmax')
and when I trained with this code , it was really very very slow , slower than FullyConnected.why...

this is locally connected layer with SliceChannel:
flatten = mx.symbol.Flatten(data=data, name="flatten")
slice = mx.symbol.SliceChannel(flatten,num_outputs = 100, name="slicechannel")
groups = []
for i in range(100):
     fc = mx.symbol.FullyConnected(data=slice[i],num_hidden=1, name = "fc%d" % i)
     groups.append(fc)
concat = mx.symbol.Concat(*groups)
relu = mx.symbol.Activation(data=concat, act_type='relu' ,name = 'relu')
softmax = mx.symbol.SoftmaxOutput(data=relu,label = "softmax_label",name='softmax')
and when I trained with this code , it was really very very slow , slower than FullyConnected.why...

I am not sure, but I think you can try something like this.
flatten = mx.symbol.Flatten(data=data, name="flatten")
multiplier = mx.sym.Variable(name='multiplier', shape=(100,))
relu = mx.sym.Activation(flatten*multiplier, act_type='relu', name='relu')
softmax = mx.sym.SoftmaxOutput(data=relu, label='softmax_label', name='softmax')

@eldercrow I'm sorry,but It didn't work. And what I wish is a locally connected layer. something like below:
a---
e---
b---/
output
c---\       /
f---/
d---/

@eldercrow I'm sorry,but It didn't work. And what I wish is a locally connected layer,something like below:

This issue is closed due to lack of activity in the last 90 days. Feel free to ping me to reopen if this is still an active issue. Thanks!


Vertical port

Hello,
I see in the different issues that it is possible to change the horizontal type of the ports into vertical via CSS.
I would like to know if it is possible to do this but with conditions.
I would like to have 4 connectors (2 vertical (top - bottom) and 2 horizontal (left - right). Is there a possibility to configure this via properties?

It's possible change position with className:
Example shape:

https://github.com/jerosoler/Drawflow/issues/454

View 2 Inputs (Top. left) 2 Outputs (Right, bottom):

https://github.com/jerosoler/Drawflow/issues/20#issuecomment-781132941


为什么api一直连接失败，

我用的api-key，然后我网络应该没问题，访问网页版一直正常的，但是用这个项目一直失败

直接post访问接口也没问题

我也是，怎么处理的呀？跟我开着vpn有关吗？

你们使用api有用吗，我这一直Error code: 429 - {'error': {'message': 'You exceeded your current quota, please check your plan and billing details.

why
{
"error": {
"message": "You didn't provide an API key. You need to provide your API key in an Authorization header using Bearer auth (i.e. Authorization: Bearer YOUR_KEY), or as the password field (with blank username) if you're accessing the API from your browser and are prompted for a username and password. You can obtain an API key from https://platform.openai.com/account/api-keys.",
"type": "invalid_request_error",
"param": null,
"code": null
}
}

同求


can you offer the training code of lt ?

can you offer the training code of lt ?

Sorry but I am having troubles to understand your question.
You may use Chinese or other languages if you find it's easier.

Since no further question is asked, I'm closing it for now.
Please reopen it or create a new issue if you have further question.


[Suggestion] Larger (or possibly more realistic) trees?

I was wonder if possibly there could be a config option or something like that to make it so that when forest and trees generate, they will naturally generate being around twice there current height.
I'd like that because in real life, many of these large old growth forests (or virgin forests) would be 30+ meters in height (depending on tree species and local climate of course)
I mean, the Conifer Forest in BoP already has some pretty realistically sized old growth firs, so it's not a question of height. The Roofed forest already has some pretty wide branches so not that either.
But I could just be underestimating the complexity of having to generate such large branches in three dimensions which is a good possibility
I just think walking under the canopy of such a large forest with there branches and imposing sizes would be such a cool and immersive feature
Sincerely, Corki

i am so for this

Thank you for the suggestion.  I intend to add more mega varieties of some existing vanilla species soon.


Add more information to exceptions occurred while writing temporary data

This typical error is seen when ingestion is writing intermediate data to a temporary file
java.io.IOException: No space left on device
	at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:?]
	at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:62) ~[?:?]
	at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113) ~[?:?]
	at sun.nio.ch.IOUtil.write(IOUtil.java:79) ~[?:?]
	at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280) ~[?:?]
	at org.apache.druid.io.Channels.writeFully(Channels.java:32) ~[?:?]
	at org.apache.druid.segment.writeout.FileWriteOutBytes.flush(FileWriteOutBytes.java:62) ~[?:?]
	at org.apache.druid.segment.writeout.FileWriteOutBytes.flushIfNeeded(FileWriteOutBytes.java:54) ~[?:?]
	at org.apache.druid.segment.writeout.FileWriteOutBytes.write(FileWriteOutBytes.java:86) ~[?:?]
	at org.apache.druid.io.Channels.writeFully(Channels.java:32) ~[?:?]

From this exception, it's hard to know how much data we were trying to write or what location we were exactly trying to write to. I added a bit of code to intercept this exception and add some more info to the outbound exception.
This PR has:

[ ] been self-reviewed.
[ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.

I see your point. Maybe “space needed for this file” or “current size of
the file”?
On Wed, 12 Oct 2022 at 4:47 PM, Laksh Singla @.***>
wrote:

@.**** commented on this pull request.
In
processing/src/main/java/org/apache/druid/segment/writeout/FileWriteOutBytes.java
https://github.com/apache/druid/pull/13217#discussion_r993325018:

@@ -59,7 +60,12 @@ private void flushIfNeeded(int bytesNeeded) throws IOException
public void flush() throws IOException
{
buffer.flip();

Channels.writeFully(ch, buffer);

try {

 Channels.writeFully(ch, buffer);

}
catch (IOException e) {

 throw new IOE(e, "Failed to write to file: %s. Space needed: %d", file.getAbsolutePath(), writeOutBytes);

From the exception, it seems like that the writeOutBytes is the space
deficit due to which the ingestion is failing, whereas the actual deficit
for the job to complete can be much more than that right?
Should the error message omit the information Space needed: %d or be
reworded such that the user knows that writeOutBytes more are required to
complete the current operation (and not the ingestion)?
—
Reply to this email directly, view it on GitHub
https://github.com/apache/druid/pull/13217#pullrequestreview-1138901445,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AALIWUOVZA64VZPCG2UZSHDWC2M6NANCNFSM6AAAAAARDFGFPA
.
You are receiving this because you authored the thread.Message ID:
@.***>

Current size of the file: %d seems like a better alternative to me.

@LakshSingla @cryptoe - thanks for reviewing. I have made the changes.


Downloading Multiple Files

Currently file downloading is limited to one file at a time. It would be nice to be able to download multiple files at a time or even have a download all button.

@GaryKriebel this issue will be fixed with download as zip right?

I belive download as a zip will be for the entirety of OSF Storage, not a select number of the files.

@GaryKriebel is correct - Download as Zip covers top level OSF folder, so let's keep this open for issue prioritization by Community and @jlcohoon

OSF Storage and folders can now be downloaded as a zip, but you still can't select multiple files to be downloaded. Any chance this can be put on the list for next epic?

I'd suggest you put it on the Epic planing Trello -- ping @erinmayhood . Thanks.


nan to support node-webkit?

How to use in node-webkit environment?

Wait for an updated version of node-webkit. #238


map custom model state_dict back to huggingface format

This is a standalone PR that depends on #461.
There are commits here that's also part of #461. The reason I'm not creating a PR from main is that I want to use #461 to run some experiments, but the content here and that in #461 should be independent of each other.
The workflow for integrating these two PRs are

review and merge #461
rebase from main for this PR
review and merge this PR

I just merged #461, can you rebase?

I just merged #461, can you rebase?

absolutely!

given our offline discussions, let me close this one and start a new PR.


Tell rust-analyzer to only look at **.rs files

This will fix an issue when trying to open the log file with the rust plugin installed.  (I'm not too sure, but I'd suggest testing whether saving a Cargo.toml file - with a new dependency or whatever - causes RA to reanalyze things with your change)

You are right. RA should refresh its state whenever Cargo.toml is edited. I checked the official RA plugin for VSCode and it seems to not use any patterns at all?
https://github.com/rust-lang/rust-analyzer/blob/master/editors/code/src/client.ts#L75


[Fern's Documentation] Highlight the Postman Generator earlier on in the README and/or Docs

What can be improved
We should make it clearer that Fern can publish to Postman.
Example that we can link to: https://github.com/revertinc/revert/blob/dd0c6a5157b1ec536ed99ad2e3e27caae9b94066/fern/generators.yml#L12-L23

Postman Collections syncing seems like a big enough deal that it deserves its own highlighted section in the docs, how about changes similar to this? I'll mock down what I'm thinking into a PR and tag it here

Bump up higher into docs, as you pitched, while adding more emphasis on what the result looks like

Add Postman logo into diagram as per screenshot

Add video of Postman Collection sync for a visual example, similar to Fern Express Video, similar to this in README:

I like the idea of adding it to the graphic. Let me take an action here and chat with our designer. May take days to get back here


Use venv for tests, pipx for install

This approach to fixing the issues in Ubuntu 24 is to use pipx for a normal installation of rosdoc2, but tests using venv.
Using pipx required some changes to sphinx_builder. The venv that is setup behind the scenes for pipx did not translate into a python subprocess, so runs of sphinx-build failed because of a missing exhale dependency. The solution to that was to follow the same process I do in tests, namely to run sphinx directly as a python module and not as a subprocess. That did not seem to be needed for sphinx-apidoc.
pipx has enough issues that it was worth adding directly to the test suite, simulating what we expect for  user install.

Just a note. This patch is more of a demonstration of an alternate approach than a serious request to land code. The larger ROS2 project will need to decide if PIP_BREAK_SYSTEM_PACKAGES: 1 is the recommendation to users, or if the use of venv and pipx, as recommended in pip's "Externally Managed Environments" is to be followed.
What this patch demonstrates is that rosdoc2 can, with minor changes, be used with pipx. We should probably just do those changes anyway regardless of the overall project decision.

We should probably converge our systems to using pipx in the future. For the moment I'd suggest we hold off until we're ready to do it more generally.

I'm going to close this, but rework it to only include the changes (and tests) for pipx compatibility. I'll explain why in a new PR.


Revisit feasibility of server pages as POST API endpoints

You can wire up any page server component today and field JSON requests, etc
You should be able to POST to any of these endpoints from the client as well to e.g. submit a form
However, in order to run an async mutation operation to e.g. persist a form's data, you have to do it in a Suspense-friendly way. Our answer to this is useQuery right now.
However, Suspense components run multiple times. This is really bad for mutating things.

We need to either propose a better pattern for mutations on the server, or introduce a way to handle API requests in a Hydrogen app without being part of the React rendering lifecycle.

You should be able to POST to any of these endpoints from the client as well to e.g. submit a form

Not sure if this talking future or current state, but FYI after our conversation yesterday, POSTing to a 0.7.0 page returns a 404.

Update: aligned with Bret and Tobi that the following API inspired by Remix makes the most sense:
// pages/some-endpoint.server.jsx

export async function api(request: Request) {
  // do some asynchronous stuff, like write to a database or 3p service!

  // `Response` can be returned
  // TODO: If a string is returned, auto-wrap in a Response.
  // TODO: If a JS object is returned, auto-wrap in a Response with the correct headers.
  return new Response('ok');
}

The tricky part will be loading up the pages graph in handle-event manually, since React Router has been responsible for matching the correct page up to this point.

Perhaps RR exposes a handy API to matchRoute based on a given input?
What does Remix do?
Or does a simple regex suffice?

Probably worth considering as we approach this ticket as well: #267

@jplhomer Yes, RR exposes a matchRoutes utility. However, we are removing RR from the server so I think we probably need to do this manually instead.
One thing about that interface you mention is that, if the same file exports a page component as the default export, then a GET request should do rendering instead of calling the API function. To work around that we could:

Require a /api/ prefix in the URL when this is an actual API call.
Detect if the request comes from our fetch utility (via headers?) and skip rendering conditionally (too much magic and limited?).

Otherwise, as Bret mentioned, having a separate folder where all the API routes are located could also work. For example, there could be functions/api/hello/world.ts or functions/graphql.ts (this is how it works in Vitedge right now).


Update subprojects

Signed-off-by: Carlos San Vicente<EMAIL_ADDRESS>

@LanderU Could you review this one?