_pickle.UnpicklingError: invalid load key, '{'.

Hi, Maybe the pickle file is not saved properly, could you please have a look at the file?
% python test.py
Traceback (most recent call last):
File "test.py", line 8, in
entitylinker = core.MLLinker(path_to_model="trainedmodels/FeatureModel_SimplifiedVCG.param")
File "/john/starsem2018-entity-linking/entitylinking/core/ml_entity_linker.py", line 20, in init
self._elmodel.load_from_file(path_to_model)
File "/john/starsem2018-entity-linking/entitylinking/mlearning/models/pytorchmodel.py", line 403, in load_from_file
super(ELModel, self).load_from_file(path_to_model)
File "/john/starsem2018-entity-linking/entitylinking/mlearning/models/pytorchmodel.py", line 162, in load_from_file
self._model.load_state_dict(torch.load(path_to_model))
File "/home/zchen/anaconda3/envs/starsem2018/lib/python3.6/site-packages/torch/serialization.py", line 267, in load
return _load(f, map_location, pickle_module)
File "/home/zchen/anaconda3/envs/starsem2018/lib/python3.6/site-packages/torch/serialization.py", line 410, in _load
magic_number = pickle_module.load(f)
_pickle.UnpicklingError: invalid load key, '{'.
test.py:
#coding=utf-8
from entitylinking import core
#entitylinker = core.MLLinker(path_to_model="trainedmodels/FeatureModel_Baseline.torchweights")
#FeatureModel_SimplifiedVCG.param
entitylinker = core.MLLinker(path_to_model="trainedmodels/FeatureModel_SimplifiedVCG.param")
#entitylinker = core.MLLinker(path_to_model="trainedmodels/FeatureModel_Baseline.torchweights")
output = entitylinker.link_entities_in_raw_input("Barack Obama is a president.")
print(output.entities)

Hi,
sometimes git messes up pickle files, try downloading the models from our server: https://public.ukp.informatik.tu-darmstadt.de/starsem18-entity-linking/FeatureModel_SimplifiedVCG.zip
Daniil


add endpoint_attrs param to simplify_graph to flexibly relax strictness

Resolves #625
This PR:

adds endpoint_attrs param to simplification.simplify_graph function to flexibly relax strictness
deprecates strict param in simplification.simplify_graph function in favor of new endpoint_attrs param
improves docstrings and comments in the simplification module

@csebastiao would you like to test this out with a specific case?

@gboeing I have tested it on my side on the two examples of the Toy graph and Copenhagen custom bicycle attribute and both are returning the expected behaviors.
It's all good for me.

Thanks. Can you provide a code snippet for testing your use case?

Oh sure sorry about that, here is the one for the Toy Graph made by hand:
import networkx as nx
import shapely

import osmnx as ox

G = nx.Graph()
G.add_node(1, x=1, y=1)
G.add_node(2, x=2, y=1)
G.add_node(3, x=2.5, y=1.5)
G.add_node(4, x=3, y=2.5)
G.add_node(5, x=3.5, y=3.5)
G.add_node(6, x=3, y=4)
G.add_node(7, x=3, y=5)
G.add_node(8, x=3.5, y=5.5)
G.add_node(9, x=3.5, y=6.5)
G.add_node(10, x=4, y=7)
G.add_node(11, x=5, y=8)
G.add_node(12, x=6, y=8)
G.add_node(13, x=6.5, y=8.5)
G.add_node(14, x=7, y=9)
G.add_node(15, x=7.5, y=8.5)
G.add_node(16, x=8, y=8)
G.add_node(17, x=7.5, y=7.5)
G.add_node(18, x=7, y=7)
G.add_node(19, x=6.5, y=7.5)
# add length and osmid just for the osmnx function to work
for i in range(1, 19):
    G.add_edge(i, i + 1, length=1, osmid=i)
G.add_node(20, x=4, y=4)
G.add_node(21, x=4, y=5)
G.add_edge(5, 20, length=1, osmid=20)
G.add_edge(20, 21, length=1, osmid=21)
G.add_edge(21, 8, length=1, osmid=22)
G.add_edge(19, 12, length=1, osmid=23)
# give three value of color to see the discrimination for an attribute
for i in range(2, 8):
    G.edges[i, i + 1]["color"] = 1
G.edges[1, 2]["color"] = 2
G.edges[5, 20]["color"] = 1
G.edges[20, 21]["color"] = 1
G.edges[21, 8]["color"] = 1
for i in range(8, 11):
    G.edges[i, i + 1]["color"] = 3
for i in range(11, 19):
    G.edges[i, i + 1]["color"] = 2
G.edges[19, 12]["color"] = 2
G = nx.MultiDiGraph(G)
# add crs for the ox_plot_graph to work
G.graph["crs"] = "epsg:4326"
ec = ox.plot.get_edge_colors_by_attr(G, "color", cmap="Set1")
ox.plot_graph(
    G,
    figsize=(12, 8),
    bgcolor="w",
    node_color="black",
    node_size=30,
    edge_color=ec,
    edge_linewidth=3,
)
G_simple = ox.simplify_graph(G, endpoint_attrs=["color"])
ec_s = ox.plot.get_edge_colors_by_attr(G_simple, "color", cmap="Set1")
ox.plot_graph(
    G_simple,
    figsize=(12, 8),
    bgcolor="w",
    node_color="black",
    node_size=30,
    edge_color=ec_s,
    edge_linewidth=3,
)

Which can be asserted with
assert len(G_simple) == len(ox.simplify_graph(G)) + 2

One way to see it on an actual OSMnx graph is with this basic example:
import osmnx as ox

protected_dict = {}
protected_dict["sidewalk:left:bicycle"] = "yes"
protected_dict["sidewalk:left:right"] = "yes"
protected_dict["cycleway:left"] = ["shared_lane", "shared_busway", "track"]
protected_dict["cycleway:right"] = ["shared_lane", "shared_busway", "track"]
protected_dict["cycleway:both"] = "lane"
protected_dict["cycleway"] = ["shared_lane", "shared_busway", "opposite_lane", "opposite"]
protected_dict["bicycle"] = ["designated", "yes", "official", "use_sidepath"]
protected_dict["highway"] = ["cycleway", "bridleway"]
protected_dict["cyclestreet"] = "yes"
protected_dict["bicycle_road"] = "yes"
for val in protected_dict:
    if val not in ox.settings.useful_tags_way:
        ox.settings.useful_tags_way += [val]
G = ox.graph_from_place("Frederiksberg Municipality, Denmark", simplify=False)
for edge in G.edges:
    for key in list(protected_dict.keys()):
        if key in list(G.edges[edge].keys()):
            if G.edges[edge][key] in protected_dict[key]:
                G.edges[edge]["cycling"] = 1
                break
            else:
                G.edges[edge]["cycling"] = 0
        else:
            G.edges[edge]["cycling"] = 0
G_s = ox.simplify_graph(G, endpoint_attrs=None)
G_ns = ox.simplify_graph(G, endpoint_attrs=["osmid"])
G_attr = ox.simplify_graph(G, endpoint_attrs=["cycling"])
ec = ox.plot.get_edge_colors_by_attr(G_attr, "cycling", cmap="RdYlGn")
ox.plot_graph(
    G_attr,
    figsize=(12, 8),
    bgcolor="w",
    node_color="black",
    node_size=10,
    edge_color=ec,
    edge_linewidth=1,
)

To take a specific node (even thought with the ever-updating OSM I don't know if the exact values will stay) one can look at the node<PHONE_NUMBER>:
G_attr.edges(1262273553, data=True)

It only exists when discriminating with the cycling (or osmid) attribute:
assert<PHONE_NUMBER> not in G_s
assert<PHONE_NUMBER> in G_attr

This should be the case as there are only two edges<PHONE_NUMBER>,<PHONE_NUMBER>, 0) and<PHONE_NUMBER>,<PHONE_NUMBER>, 0) with a different value for the cycling attribute, because for the latter we have 'highway': 'cycleway' and 'bicycle': 'designated' and for the former 'highway': 'footway'.

endpoint_attrs looks very useful, thanks for working on this!

I believe this is ready to merge.

Note that #1145 additionally adds a node_attrs_include param to the simplify_graph function to further flexibly relax graph simplification strictness. It also renames the endpoint_attrs param to edge_attrs_differ for consistent and clear naming, given the new param.


Activating AnimationTree under certain conditions causes editor to freeze forever

Godot version
v4.1.stable.official [970459615]
System information
Godot v4.1.stable - Windows 10.0.19045 - Vulkan (Compatibility) - AMD Radeon(TM) Graphics (Advanced Micro Devices, Inc.; 31.0.14057.5006) - AMD Ryzen 5 5600G with Radeon Graphics (12 Threads)
Issue description
A specific AnimationNodeStateMachine layout freezes the editor under certain conditions (Seems fine when parented to a Node2D). I've tried waiting for it to unfreeze for a few minutes with no improvement.
Steps to reproduce
Activate the scene tree in the minimal reproduction project to freeze the editor. The contents of each animation does not matter.
Minimal reproduction project
Minimal reproduction project

I am encountering this issue as well while using AnimationNodeStateMachine. It completely crashes Godot.
Specs: Godot v4.1.stable from steam with Pop_os! 22.04 LTS and 5900x CPU and 6800xt GPU)

Duplicate of #79012.


fix(core): resolve race in IAsyncResult.wait()

closes #485

I thought about adding a test, but I didn't do it because a failure would cause the test code to hang forever. However, it would be very simple to do. Should I add one?

I didn't mean to close it.

Updated jgh-ds:master with a new test case for the issue: test_wait_race()

It should be ready to merge now, including a test case.
Thanks.

Added the doc string and removed the unnecessary try/except from the test case. Also verified that failing the test case will not hang the testing process.

I'm going to go ahead and merge. The test is the only part I'm slightly unsure of, but I am confident the race condition exists and your fix looks good.

I've determined that the part of the test which is:
        if not cv.is_set():
            async_result.set("kick")

is not needed (but is harmless). I was thinking that it would be necessary in the failure case, but the assertion failure from eq_(_) keeps it from being run in that case anyway. Since the thread has a timeout anyway, it cannot hang the testing process.
Would it be better to remove it?

Yes, please remove it. No one later will realize it can be removed.
Re: the merge, I accidentally clicked the wrong button, then had to wait til tests passed...

The dead code in the test is removed now.

Apparently the removed code is needed in Python 3.5 and 3.6 to avoid hanging.

The test failures are not on the new test case. The actual IAsyncResult.wait() code is the same as the first commit, where all the tests pass.
Could other test cases depend on the old (broken) behavior? The latest errors are on test_race_condition_new_partitioner_steals_the_lock

I've verified that the failing test case in test_race_condition_new_partitioner_steals_the_lock does not use the modified IAsyncResult.wait() code.
It appears that test_race_condition_new_partitioner_steals_the_lock is sensitive to execution performance (using 0.1 second and 0.2 second time constants). Perhaps the Travis environment is slower now.

It should be good to go now.


QC: Github actions to deploy app to shinyapps.io

Once we make the package public, let's create 2 pipelines that deploys the app using {clindata} whenever there is a:

release
PR merged to dev

Talk with Devin about internal deployment - wait until open-source for shinyapps.io deployment.

@jwildfire Do we have a gilead-biostats (or similar) shinyapps.io account? Free tier should be fine for what we want (although the sample data MIGHT end up pushing things a bit; they don't technically have a size limit that I can see, but I suspect there's SOME sort of limit).


Update Install-AdcsNetworkDeviceEnrollmentService.md

Made changes to Example1 as using this will provide information on properties that will be configured when it is installed.

@JohanFreelancer9 please copy review

Issue#https://github.com/MicrosoftDocs/windows-powershell-docs/issues/2621

Docs Build status updates of commit 817d4db:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/winserver2019-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2019-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 9eca1b2:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/winserver2012-ps/adcsdeployment/install-adcsnetworkdeviceenrollmentservice.md
:white_check_mark:Succeeded
View (winserver2012-ps)

docset/winserver2019-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2019-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 2c6247c:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/winserver2012-ps/adcsdeployment/install-adcsnetworkdeviceenrollmentservice.md
:white_check_mark:Succeeded
View (winserver2012-ps)

docset/winserver2012r2-ps/adcsdeployment/install-adcsnetworkdeviceenrollmentservice.md
:white_check_mark:Succeeded
View (winserver2012r2-ps)

docset/winserver2016-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2016-ps)

docset/winserver2019-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2019-ps)

docset/winserver2022-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2022-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

@JohanFreelancer9 made changes to the example format

Docs Build status updates of commit 6977988:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/winserver2012-ps/adcsdeployment/install-adcsnetworkdeviceenrollmentservice.md
:white_check_mark:Succeeded
View (winserver2012-ps)

docset/winserver2012r2-ps/adcsdeployment/install-adcsnetworkdeviceenrollmentservice.md
:white_check_mark:Succeeded
View (winserver2012r2-ps)

docset/winserver2016-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2016-ps)

docset/winserver2019-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2019-ps)

docset/winserver2022-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2022-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 19dc0a3:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/winserver2012-ps/adcsdeployment/install-adcsnetworkdeviceenrollmentservice.md
:white_check_mark:Succeeded
View (winserver2012-ps)

docset/winserver2012r2-ps/adcsdeployment/install-adcsnetworkdeviceenrollmentservice.md
:white_check_mark:Succeeded
View (winserver2012r2-ps)

docset/winserver2016-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2016-ps)

docset/winserver2019-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2019-ps)

docset/winserver2022-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2022-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

@tseward  Made changes per your suggestion.

Docs Build status updates of commit d188ebe:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/winserver2012-ps/adcsdeployment/install-adcsnetworkdeviceenrollmentservice.md
:white_check_mark:Succeeded
View (winserver2012-ps)

docset/winserver2012r2-ps/adcsdeployment/install-adcsnetworkdeviceenrollmentservice.md
:white_check_mark:Succeeded
View (winserver2012r2-ps)

docset/winserver2016-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2016-ps)

docset/winserver2019-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2019-ps)

docset/winserver2022-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2022-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit ee13bd1:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/winserver2012-ps/adcsdeployment/install-adcsnetworkdeviceenrollmentservice.md
:white_check_mark:Succeeded
View (winserver2012-ps)

docset/winserver2012r2-ps/adcsdeployment/install-adcsnetworkdeviceenrollmentservice.md
:white_check_mark:Succeeded
View (winserver2012r2-ps)

docset/winserver2016-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2016-ps)

docset/winserver2019-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2019-ps)

docset/winserver2022-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2022-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit eed0186:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/winserver2012-ps/adcsdeployment/install-adcsnetworkdeviceenrollmentservice.md
:white_check_mark:Succeeded
View (winserver2012-ps)

docset/winserver2012r2-ps/adcsdeployment/install-adcsnetworkdeviceenrollmentservice.md
:white_check_mark:Succeeded
View (winserver2012r2-ps)

docset/winserver2016-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2016-ps)

docset/winserver2019-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2019-ps)

docset/winserver2022-ps/adcsdeployment/Install-AdcsNetworkDeviceEnrollmentService.md
:white_check_mark:Succeeded
View (WindowsServer2022-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Support seting topic schema when creating producers/consumers

Added class to allow setting topic schema when creating
producers/consumers

Even if we can set the schema information, the current Node.js client can handle message payloads only with Buffer type.
https://github.com/apache/pulsar-client-node/blob/cc79c67c3c330be0667203bf369b5de625eb0044/src/Message.cc#L136-L143
https://github.com/apache/pulsar-client-node/blob/cc79c67c3c330be0667203bf369b5de625eb0044/src/Message.cc#L85-L93

This simply allows the NodeJS client to specify the schema when creating the producer or consumer - it doesn't go to the same extent that, say, the Python client does where passing an object (to the producer's send) causes that object to get encoded using the schema info or where the subscriber decodes the message payload to a value using the schema info.
I'm planning on adding the automatic encode/decode parts in a similar fashion to how the Python client does it i.e. by wrapping the Producer/Consumer/Message bindings in language-native classes that add information regarding schema and support for automatic encode/decode, but I'm not quite sure yet of the exact implementation.

@massakam I think this is fine - any user of a schema'd topic is (with this implementation) still responsible for serializing to Buffer when sending messages or deserializing from Buffer into whatever schema when consuming messages.

I think this is fine - any user of a schema'd topic is (with this implementation) still responsible for serializing to Buffer when sending messages or deserializing from Buffer into whatever schema when consuming messages.

Once users implement such an application, isn't it a breaking change for them to support automatic encoding/decoding?
const buffer = /* object serialization> */;
producer.send({
  data: buffer,
});

↓
const obj = {
  key1: "val1",
  key2: "val2",
};
producer.send({
  data: obj,
});

If so, I think parts for automatic encoding / decoding should be added in this PR or as soon as possible.

isn't it a breaking change for them to support automatic encoding/decoding?

This is a great point - I hadn't thought about it yet. I could work around it in producer.send by skipping the automatic encode if data is of a buffer type, or make automatic encode opt-in either during producer creation or on each send call.
It's definitely a bit more tricky for the message handling function in the consumer. Would it be reasonable to do something along the lines of what the Python lib does? It assigns the decoded message.data to a message.value attribute. Alternatively, could make the automatic decode opt-in during consumer creation.

In the case of Python and Go libraries, it seems that binary data and deserialized data can be obtained from different fields of the message object. If the Node.js library follows this, it won't be a breaking change.
https://github.com/apache/pulsar/blob/d5036eaefcba42e6c9d159ffb14bee597d13380f/pulsar-client-go/pulsar/message.go#L47-L75

Any news about this? It would be great to have schemas in nodejs! Can I help to complete the PR?

Any news about this? It would be great to have schemas in nodejs! Can I help to complete the PR?

I've not taken a look at this in quite some time unfortunately - I was working on this as part of related work at a prior employer. I'll take a look at this again this weekend, but cannot make any guarantees as to completing the requested changes.
Contributions that complete the requests are absolutely welcome, I believe all that's left is:

provide example documentation of how a NodeJS client would use this feature.

There has been some chatter in this PR re adding automatic encode/decode support, similar to how Python and Go client libs do it. Contributions that add that would be above and beyond but are absolutely welcome as well.
I'll update this weekend if I'm able to accomplish the requested changes and/or automatic encode/decode support feature request.

@lcartercondon hi, thanks for the reply.
If it's ok for you, I can continuo working on the two request:
documentation and auto encode/decode, that wold be super helpful for my use case!
Alternatively, you can close this PR with docs and I can submit an new PR with audo encode/decode!

Any updates on this? Seems like a really useful feature. As I understand it, without schema support, Pulsar SQL is basically non-functional without this feature. Am I correct in that understanding?

@sijie Anu update on this request and it is really important to the world because numbers of nodejs developmer become more amd more

@lcartercondon Could you please rebase to the master branch?

I've resolved the conflict, but the current code fails to compile.

@lcartercondon Could you fix the compilation error?

This change is merged as a new PR with fixing some compliation errors:
https://github.com/apache/pulsar-client-node/pull/223
@lcartercondon Thank you for your contribution!


Florence2 Inference

I'm not able to run inference for my fine tuned model - tried running the command in the documentation.
 I used 2 3090 gpu
From the doc, I tried running inference with:
swift infer \
    --ckpt_dir /root/models/florence-2-large/v7-20241011-025045/checkpoint-1251 \
    --stream false \
    --max_new_tokens 1024

This is the output:
swift infer     --ckpt_dir /root/models/florence-2-large/v7-20241011-025045/checkpoint-1251     --stream false     --max_new_tokens 1024
2024-10-12 01:07:51.174459: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-10-12 01:07:51.188696: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-10-12 01:07:51.193105: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-10-12 01:07:51.204467: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-12 01:07:52.009240: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
run sh: `/usr/local/bin/python /usr/local/lib/python3.10/site-packages/swift/cli/infer.py --ckpt_dir /root/models/florence-2-large/v7-20241011-025045/checkpoint-1251 --stream false --max_new_tokens 1024`
2024-10-12 01:07:56.737825: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-10-12 01:07:56.751559: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-10-12 01:07:56.755794: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-10-12 01:07:56.766359: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-12 01:07:57.563820: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[INFO:swift] Successfully registered `/usr/local/lib/python3.10/site-packages/swift/llm/data/dataset_info.json`
[INFO:swift] Start time of running main: 2024-10-12 01:08:00.573659
[WARNING:swift] The checkpoint dir /root/models/florence-2-large/v7-20241011-025045/checkpoint-1251 passed in is invalid, please make surethe dir contains a `configuration.json` file.
[INFO:swift] ckpt_dir: /root/models/florence-2-large/v7-20241011-025045/checkpoint-1251
[INFO:swift] Successfully registered `/root/app/florence2/dataset_info.json`
[INFO:swift] Setting model_info['revision']: main
[INFO:swift] Setting self.eval_human: True
[INFO:swift] Setting overwrite_generation_config: True
[INFO:swift] args: InferArguments(model_type='florence-2-large', model_id_or_path='microsoft/Florence-2-large', model_revision='main', sft_type='full', template_type='florence', infer_backend='pt', ckpt_dir='/root/models/florence-2-large/v7-20241011-025045/checkpoint-1251', result_dir=None, load_args_from_ckpt_dir=True, load_dataset_config=False, eval_human=True, seed=42, dtype='bf16', model_kwargs=None, dataset=[], val_dataset=[], dataset_seed=42, dataset_test_ratio=0.01, show_dataset_sample=-1, save_result=True, system=None, tools_prompt='react_en', max_length=None, truncation_strategy='delete', check_dataset_strategy='none', model_name=[None, None], model_author=[None, None], quant_method=None, quantization_bit=0, hqq_axis=0, hqq_dynamic_config_path=None, bnb_4bit_comp_dtype='bf16', bnb_4bit_quant_type='nf4', bnb_4bit_use_double_quant=True, bnb_4bit_quant_storage=None, max_new_tokens=1024, do_sample=None, temperature=None, top_k=None, top_p=None, repetition_penalty=None, num_beams=1, stop_words=[], rope_scaling=None, use_flash_attn=None, ignore_args_error=False, stream=False, merge_lora=False, merge_device_map='cpu', save_safetensors=True, overwrite_generation_config=True, verbose=None, local_repo_path=None, custom_register_path=None, custom_dataset_info='/root/app/florence2/dataset_info.json', device_map_config=None, device_max_memory=[], hub_token=None, gpu_memory_utilization=0.9, tensor_parallel_size=1, max_num_seqs=256, max_model_len=None, disable_custom_all_reduce=True, enforce_eager=False, vllm_enable_lora=False, vllm_max_lora_rank=16, lora_modules=[], tp=1, cache_max_entry_count=0.8, quant_policy=0, vision_batch_size=1, self_cognition_sample=0, train_dataset_sample=-1, val_dataset_sample=None, safe_serialization=None, model_cache_dir=None, merge_lora_and_save=None, custom_train_dataset_path=[], custom_val_dataset_path=[], vllm_lora_modules=None, device_map_config_path=None)
[INFO:swift] Global seed set to 42
[INFO:swift] device_count: 2
[INFO:swift] Loading the model using model_dir: /root/models/florence-2-large/v7-20241011-025045/checkpoint-1251
[INFO:swift] model_kwargs: {'low_cpu_mem_usage': True, 'device_map': 'auto'}
Could not locate the modeling_florence2.py inside /root/models/florence-2-large/v7-20241011-025045/checkpoint-1251.
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/swift/cli/infer.py", line 5, in <module>
    infer_main()
  File "/usr/local/lib/python3.10/site-packages/swift/utils/run_utils.py", line 32, in x_main
    result = llm_x(args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/swift/llm/infer.py", line 304, in llm_infer
    model, template = prepare_model_template(args, device_map=args.device_map_config)
  File "/usr/local/lib/python3.10/site-packages/swift/llm/infer.py", line 185, in prepare_model_template
    model, tokenizer = get_model_tokenizer(
  File "/usr/local/lib/python3.10/site-packages/swift/llm/utils/model.py", line 6561, in get_model_tokenizer
    model, tokenizer = get_function(model_dir, torch_dtype, model_kwargs, load_model, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/swift/llm/utils/model.py", line 2870, in get_model_tokenizer_florence
    model, tokenizer = get_model_tokenizer_from_repo(
  File "/usr/local/lib/python3.10/site-packages/swift/llm/utils/model.py", line 928, in get_model_tokenizer_from_repo
    model = automodel_class.from_pretrained(
  File "/usr/local/lib/python3.10/site-packages/modelscope/utils/hf_util.py", line 65, in from_pretrained
    module_obj = module_class.from_pretrained(model_dir, *model_args,
  File "/usr/local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 547, in from_pretrained
    model_class = get_class_from_dynamic_module(
  File "/usr/local/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 506, in get_class_from_dynamic_module
    final_module = get_cached_module_file(
  File "/usr/local/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 310, in get_cached_module_file
    resolved_module_file = cached_file(
  File "/usr/local/lib/python3.10/site-packages/transformers/utils/hub.py", line 374, in cached_file
    raise EnvironmentError(
OSError: /root/models/florence-2-large/v7-20241011-025045/checkpoint-1251 does not appear to have a file named modeling_florence2.py. Checkout 'https://huggingface.co//root/models/florence-2-large/v7-20241011-025045/checkpoint-1251/tree/None' for available files.

Seems to be complaining about missing files but I am not sure if I am missing a step here. Any help is appreciated, thank you!

What version of Transformers?

I tried both 4.45.0.dev and 4.46.0.dev. Copying the modeling_florence2.py file directly from the model to the checkpoint folder seemed to solve itOn Oct 12, 2024, at 1:31 AM, Jintao @.***> wrote:
What version of Transformers?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

please use transformers==4.44.*

hi, ive tried installing 4.44.2 with the same error. copying the file directly into the checkpoint folder seems to fix it although im not sure what exactly is intended.
am i supposed to train with 4.44.2 for florence2?

fixed in https://github.com/modelscope/ms-swift/pull/2341
https://github.com/huggingface/transformers/issues/24737

Please retrain or copy the file to the checkpoint.


Complete copy edit pass

@rotycenh

OPS Build status updates of commit 08342bd:
:clock10: Preparing: average preparing time is 45 sec(s)

OPS Build status updates of commit 08342bd:
:clock10: Incremental building: average incremental building time is 23 sec(s)

OPS Build status updates of commit 08342bd:
:warning: Validation status: warnings

File
Status
Preview URL
Details

toc.json
:warning:Warning

Details

building-blocks/extending-templates/toc.json
:warning:Warning

Details

bread/toc.json
:warning:Warning

Details

docs/cloud-adoption/infrastructure/mainframe-migration/overview.md
:white_check_mark:Succeeded
View

toc.json

[Warning] Error happen when converting toc.json to Pdf. Details: Could not find file 'T:\i5gy\toc.json'.

building-blocks/extending-templates/toc.json

[Warning] Error happen when converting building-blocks/extending-templates/toc.json to Pdf. Details: Could not find a part of the path 'T:\i5gy\building-blocks\extending-templates\toc.json'.

bread/toc.json

[Warning] Error happen when converting bread/toc.json to Pdf. Details: Could not find a part of the path 'T:\i5gy\bread\toc.json'.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Overwritten by #1127


How to use the base_class?

I have a question about how to use the base_class. I tried looking into multiple places for an answer, but I cannot find any example. I believe that I need to override the class method, but I am not sure which one.
This is what I am doing:
class Author(models.Model):

    name = models.CharField(max_length=250)

class Article(models.Model):

    authors = SortedManyToManyField(Author,  base_class="ArticleAuthorsSort")

class ArticleAuthorsSort(models.Model):

    author = models.ForeignKey(Author, on_delete=models.CASCADE)

    article = models.ForeignKey(Article, on_delete=models.CASCADE)

    sort_value = models.PositiveSmallIntegerField()

    class Meta:

        ordering = ('sort_value',)

    def __str__(self):

        return self.sort_value

And this is the error I am getting:
File sortedm2m/fields.py", line 330, in create_sortable_many_to_many_intermediary_model

    '_sort_field_name': sort_field_name,

TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

There's an example of base_class here
https://github.com/jazzband/django-sortedm2m/blob/5f357c20d60e0fe8bb9735ae5e0f30bab50982dc/example/testapp/models.py#L14-L21
And documented
https://github.com/jazzband/django-sortedm2m/blob/5f357c20d60e0fe8bb9735ae5e0f30bab50982dc/README.rst#L97-L114
I only see testing of the __str__ use described in the docs. I see no examples of using a base_class that defines new fields, and it is unclear if such a class should subclass Model or be a dumb mixin. The advice "You also could use it to add additional fields to the through model" is trouble without a corresponding test.
However, the advice here:

consider not using sortedm2m but use a ordinary Django ManyToManyField and specify your own through model

Sounds like a failed middle-ground. Why not a sorted m2m field and specify your own through model? An example exists in the tests right here:
https://github.com/jazzband/django-sortedm2m/blob/5f357c20d60e0fe8bb9735ae5e0f30bab50982dc/sortedm2m_tests/models.py#L32-L51
Thus, my high-level suggestion for your use class would be to replace authors = SortedManyToManyField(Author,  base_class="ArticleAuthorsSort") with authors = SortedManyToManyField(Author,  through="ArticleAuthorsSort"). This is a non-trivial change and requires a number of balancing changes to align the through fields and the "sort_value_field_name". However, in this case, you have a template to start with, the advice on the basic usage of through models in m2m fields is widely available on the internet.


Custom number of runs per property

I'm using scalacheck with sbt. When I run the test command in sbt, Scalacheck, by default, generates 100 test cases for each Prop in all Properties classes in my test path.
Can we add a feature so that I can specify minSuccessfulTests for each Prop in my source code? This would be useful so that I can run fewer tests for tests that take longer to run.
What I have in mind is an annotation that I can apply to either my Properties class or each property block in my Properties class, specifying minSuccessfulTests for the appropriate tests.
I think this would be easy to implement, so I wouldn't mind implementing this if you guys like the idea.

This should be possible to add in ScalaCheck 2. Adding it to the current implementation would probably be too involved.

I'm not sure that I want to wait until ScalaCheck 2.
I think I'll take a stab at this and see if we can get this working without changing the current implementation too much?

PR #188 has been merged.


Anchor tag helper not working when angular binding

what is the proplem

{{ movie.Id }}  it works expected

Is not worked ..

Generate Html element

228  //id is binded

//but not binded anchor tag helper
my entire page data binding is works expected ,but when i use anchor tag helper its not working.
any suggestion for this

stackoverflow thread http://stackoverflow.com/questions/34353425/anchor-tag-helper-not-working-in-mvc-6-and-angular-js

angular is marked issue on MVC Side https://github.com/angular/angular.js/issues/13582#event-495744974

Here's what you are doing.
You are sending [tag:angularjs] code to ASP.NET. Do you see the issue? You're telling ASP.NET:

My movie ID is {{ movie.Id }}

So ASP.NET takes {{ movie.Id }} and encodes it into your URL.
You're using client-side code in a server side concept.

So what is the solution.

Which way to overcome to this issue. I was wondering to say this is a bug.

This is not a bug. It's a misunderstanding in usage. I'll now answer on the SO question only since this can be closed.

I moved to client side URL generation its OK for now ..


[RFC] api: drivers: serial: uart_async: Call UART_RX_RDY event after rx_disable()

As proposed in #25317 modified API description and updated the test and nrf implementations.
Short summary:
Before: calling rx_disable() stops receiver and discards any pending, received data.
After: after callingrx_disable() UART_RX_RDY event is generated if there is any pending data.
PR contains changes in all 3 drivers which implemented asynchronous API.

There is one more implementation of this API: uart_sam0.c added by @Sizurka. Any chance to update that implementation? As you can see these are usually small tweaks.

I believe that's how I already made the SAM0 implementation work: uart_sam0_notify_rx_processed.  Except that I released the inactive (if any) buffer before the ready event (i.e. UART_RX_BUF_RELEASED (inactive), UART_RX_RDY (active), UART_RX_BUF_RELEASED (active), UART_RX_DISABLED).  So if the new test case enforces that, then the release would need to be re-ordered.
That said, that particular path was untested and I no longer have a functional Zephyr environment, so I can't verify it against the new test case.

@Sizurka can you take a look at https://github.com/zephyrproject-rtos/zephyr/pull/25967/commits/3d71e790b9acb6d2f3ea0b5694211f232444221a where I've modified the order. I will try to find someone with this board to run the test.

I think that there is no need to change the order of events generated by the uart_sam0 driver. As long as UART_RX_RDY is generated before UART_RX_BUF_RELEASED for the same buffer, it should be fine. It should not matter if the second buffer is released before or after that.

I think that there is no need to change the order of events generated by the uart_sam0 driver. As long as UART_RX_RDY is generated before UART_RX_BUF_RELEASED for the same buffer, it should be fine. It should not matter if the second buffer is released before or after that.

I think that it would be good to specify that and have it consistent. We have only 3 implementation so it is still quite early. I was thinking that user may expect that buffer will be released in the same order as the were provided.

@nzmichaelh and @pabigot could you test the changes on sam0?

@anangl can you revisit?


Problematic raw pointer conversion in buf.rs

Hello all,
I'm not sure if I made a conceptual mistake here and of not if this is actually a problem, but I would like to ask you to have a look at this.
Here are the tests that I ran on buf.rs
#[cfg(test)]
mod buf_test {
    use crate::{util::Binding, Buf};
    use libgit2_sys::git_buf;

    #[test]
    #[allow(unused)]
    fn pointer_tests() {
        let buf: Buf = Buf::new();
        let buf_addr: u64 = (&buf) as *const Buf as u64;
        let buf_raw_addr: u64 = (&buf.raw) as *const git_buf as u64;
        /* Since Buf is just a wrapper around raw::git_buf which actually holds the raw::git_buf struct,
        they should refer to the same memory address */
        assert_eq!(buf_addr, buf_raw_addr);

        let git_buf: *mut git_buf = Binding::raw(&buf);
        let git_buf_addr: u64 = git_buf as u64;
        /* It should not make a difference if we retrieve the raw::git_buf from Binding::raw or from struct Buf.raw */
        assert_eq!(buf_raw_addr, git_buf_addr);

        let from_raw_buf: Buf = unsafe { Binding::from_raw(git_buf) };
        let from_raw_buf_addr: u64 = (&from_raw_buf) as *const Buf as u64;
        /* If we recreate Buf from raw pointer to git_buf and Buf actually holds the struct raw::git_buf we should end up with
        the same memory address as to begin with (which is not possible) or we have moved raw::git_buf which could cause trouble
        on the C side since C does not know about our move.   */
        assert_eq!(buf_addr, from_raw_buf_addr);

        let from_raw_buf_raw_addr: u64 = (&from_raw_buf.raw) as *const git_buf as u64;
        /* This actually shows that we moved raw::git_buf here
            impl Binding for Buf {
                type Raw = *mut raw::git_buf;
                unsafe fn from_raw(raw: *mut raw::git_buf) -> Buf {
                    Buf { raw: *raw } // move of raw pointer
                }

        This behavior may be problematic when Rust receives a buffer from C to write into e.g. here
            @param out git_buf to write data into
            @param repo Repository to read prepared message from
            @return 0, GIT_ENOTFOUND if no message exists or an error code
            GIT_EXTERN(int) git_repository_message(git_buf *out, git_repository *repo);
        */
        assert_eq!(buf_raw_addr, from_raw_buf_raw_addr);
    }
}

So my assumption is, that the struct
pub struct Buf {
    raw: raw::git_buf,
}

should have been
pub struct Buf {
    raw: *mut raw::git_buf,
}

similar to repo.rs
pub struct Repository {
    raw: *mut raw::git_repository,
}

That looks like it would probably work to me.
It is kinda strange that git_filter_buffered_stream_new is undocumented, whereas the rest of the filter api is documented.


RFCs 0036 and 0037 (and others?): need formal state names

I was in these RFCs, and noticed that although they have nice choreography diagrams, they don't provide formal names for their states. This is probably because they didn't build state machine matrices, which is fine (the matrices are just an optional diagram; the choreography diagrams are an experiment to see if the same content can be better represented the other way). Whatever diagram we do or do not use, we need formal names for the states, because problem reports need to report state in an interoperable way. This is the basis of DIDComm features like "I experienced an error while in <state 7>, and I'm now dropping back to <state 6>", and they are also implied by RFC 0317 please_ack. Today this is mostly tribal knowledge about future direction, not a feature we depend on. But I think this will become important as protocols become more complex, and as our standards for robustness and error handling go up.
So I recommend that we agree to assign formal state names in each place that they are lacking. If we do, I will then take an action item to submit PR(s) to do so.
@rajeshkalaria80, @dkulic

I agree that formalized names make sense. @dhh1128 I think an example PR would make this pretty clear and promote any discussion we need to have.

Agreed. I will prepare an example that we can discuss on Wednesday's community call.

I have raised PR #412 as an example of what I'm thinking. I have also prepared a slide deck to explore the issue in a community call.

Closing this.  There is a new PR that is in the same vein for Present Proof (0037), but we can deal with this without having this issue open.


Feature Request: new ChipField component

I'd like to create a new component for Material UI. The component would be called something like ChipField and the purpose of it would be to an editable form field for the current Chip component.

[x] I have searched the issues of this repository and believe that this is not a duplicate.

Expected Behavior
It would work similar to StackOverflow's tags input, where it looks like a normal text field but when you type a certain delimiter, it would create a chip. If you click the chip, the chip will turn into an editable text field.

Context
The exact behavior of the field can be determined later. I just wanted to see if you would be willing to accept such a pull request before I start working on it. My thought is that many people want an editable ChipField and providing it in this library would benefit it.
Thanks!

Thanks for quick response! And yes, it would be similar to the react-select demo but it would allow free-form text input with creating and replacing the chips with text inputs.
If you feel like that functionality should exist externally then so be it. Thanks for the support!

The Downshift version is close too:


REALMC-7146: Initialize empty next realm-cli repo

Just clearing out the new branch for a clean start.  Hoping to get this PR running something in EVG?  Not sure if I might need to actually write some code/tests first...

Should be merged before #18


[0.6.1] Error parsing data.xml based on generated template

I'm using version 0.6.1 on Windows 10 64-bit, installed from Node.js.
My data.xml is based on the new company command's generated template. And, according to the recommended validator, it is valid. Yet, I'm seeing the following error on build:
Finding data…
There was an error during the parsing of `C:\presskitsource\data.xml`. Check that your XML document is valid.

Here's a zip file containing two data.xml that I tried (one full, and one trimmed).

Hello,
Indeed, it looks like there's an error with a recent version of node and one of the package we use.
I will publish a fix soon.
Thanks. :)

While trying to find a workaround from my end, I tried these Node.js versions:
6.11.3
7.10.1
8.1.4
8.4.0
I also tried these presskit versions:
0.6.1
0.6.0
0.5.0
Same error in all cases.

0.6.2 has been published. Re-open this issue if the problem is still here. ;)


Fixed styling issues and created seperate CSS/JS files for the simulator

Describe the changes you have made in this PR:
Fixed issues with styling; created seperate files for the simulator CSS and JS so not relying on the application files.
Screenshots of the changes (If any):
Fixed properties box:
Was previously

Now

Fixed layout of menus with multiple lines:
Was previously

Now

Coverage remained the same at 83.278% when pulling 7d7740f638f796a6e4fbec5de3c6ba2beaec24ee on nrs1729:fix-overridden-styles into 5053bffa360af3f895cefef50ea0dc3c3da6d6eb on CircuitVerse:master.

Green font color is not right, that styles is from theme.scss, circuitverse.org/simulator refer this for the font settings we need


indentation from and multi-line strings/comments are incorrectly used to determine tab length

Does this issue occur when all extensions are disabled?: Yes

Version: 1.65.2 (user setup)
Commit: c722ca6c7eed3d7987c0d5c3df5c45f6b15e77d1
Date: 2022-03-10T14:33:55.248Z
Electron: 13.5.2
Chromium: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Windows_NT x64 10.0.19042
Steps to Reproduce:

create a new file, eg. test.py
paste in the following code:"""
  hi
"""

if True:
    if True:
        print(1)

save and close the file
re-open the file

the indentation margins are now 2 spaces apart instead of 4 even though the code uses 4 space indentation

It looks like that the indentation rules for Python is not covering this case. Currently the indentation rules for  auto indent feature in VS Code is regex based, which doesn't really understand the scope (function, statement) etc. Formatting works more consistently in this scenario, so I would recommend to enable formatting on type or save. Closing this as we don't expect to get around it in the near future.


feat: Add new ptn modules azd/monitoring and azd/insights-dashboard.

Description

Fixes https://github.com/Azure/Azure-Verified-Modules/issues/1206 and https://github.com/Azure/Azure-Verified-Modules/issues/1226.
Pipeline Reference

Pipeline

Type of Change

[ ] Update to CI Environment or utilities (Non-module affecting changes)
[x] Azure Verified Module updates:

[ ] Bugfix containing backwards-compatible bug fixes, and I have NOT bumped the MAJOR or MINOR version in version.json:

[ ] Someone has opened a bug report issue, and I have included "Closes #{bug_report_issue_number}" in the PR description.
[ ] The bug was found by the module author, and no one has opened an issue to report it yet.

[ ] Feature update backwards compatible feature updates, and I have bumped the MINOR version in version.json.
[ ] Breaking changes and I have bumped the MAJOR version in version.json.
[ ] Update to documentation

Checklist

[x] I'm sure there are no other open Pull Requests for the same update/change
[x] I have run Set-AVMModule locally to generate the supporting module files.
[ ] My corresponding pipelines / checks run clean and green without any errors or warnings

@jongio for notification.

This PR contains the development of two ptn modules: azd/monitoring and azd/insights-dashboard, and since these two modules have dependencies, I had to put them in one PR for review.
Currently, module ptn/azd/monitoring will reference ptn/azd/insights-dashboard, and if a user reference ptn/azd/insights-dashboard individually, they also need to pass it the relevant parameters of the logAnalytics existing resource. We are not sure whether such a scenario exists at present. In short, the only difference between the two modules is:

azd/monitoring: Create logAnalytics resource in the module and pass its id to azd/insights-dashboard.
azd/insights-dashboard: Pass the id of an existing logAnalytics resource directly.

So, should we consider merging the two into one module? Do you have any ideas? @jongio

There are currently three errors in static validation in Pipeline:
Error: [-] [azd/insights-dashboard] Owning team should be specified correctly in CODEWONERS file. 11ms (10ms|2ms)
Error: [-] [azd/insights-dashboard] Telemetry deployment should have expected telemetry identifier. 114ms (113ms|1ms)
Error: [-] [azd/insights-dashboard] Module identifier should be listed in issue template in the correct alphabetical position. 32ms (31ms|1ms)

@jongio - Can you help me with the first error?
Error: [-] [azd/insights-dashboard] Owning team should be specified correctly in CODEWONERS file. 11ms (10ms|2ms)

I'm not sure what to fill in the Owning team here, you can refer to this file: https://github.com/Azure/bicep-registry-modules/blob/main/.github/CODEOWNERS
@AlexanderSehr - Can you help take a look at the remaining two errors?  What do I need to do to fix them. Thank you very much!
Error: [-] [azd/insights-dashboard] Telemetry deployment should have expected telemetry identifier. 114ms (113ms|1ms)
Error: [-] [azd/insights-dashboard] Module identifier should be listed in issue template in the correct alphabetical position. 32ms (31ms|1ms)

liste

Hey @zedy-wj,
regarding the telemetry etc. - can you confirm you created an AVM issue for both modules and that they got approved? If the case, @matebarabas should've updated this file, which is ultimately the one that contains the telemetry value the test expects.
Regarding the order, please make sure the module is listed in the module issues file here as defined in the specs.

liste

Hey @zedy-wj, regarding the telemetry etc. - can you confirm you created an AVM issue for both modules and that they got approved? If the case, @matebarabas should've updated this file, which is ultimately the one that contains the telemetry value the test expects.
Regarding the order, please make sure the module is listed in the module issues file here as defined in the specs.

It worked for me, thanks a lot! 😃

@jongio - Could you help me with the first error?
Error: [-] [azd/insights-dashboard] Owning team should be specified correctly in CODEWONERS file. 11ms (10ms|2ms)

@jongio Any ideas about this error?

@jongio - Could you help me with the first error?
Error: [-] [azd/insights-dashboard] Owning team should be specified correctly in CODEWONERS file. 11ms (10ms|2ms)

@jongio Any ideas about this error?

This is just about this file. An entry for each respective application must be added there, and the teams created. For context: Based on this file, GitHub will request the module owner's team to review PRs for their modules. And by being part of that team (and it being created in the correct way), owners can approve set PRs.
If this news to you, I'd strongly encourage you to read through the AVM specs starting from here (all the way through until you hit the CODEOWNERS file header.

@jongio - Corresponding pipelines / checks run clean and green. Please review this PR, thanks.

@AlexanderSehr - We created a new PR for azd/insights-dashboard: https://github.com/Azure/bicep-registry-modules/pull/3175. After it is merged, we will create a new PR for azd/monitoring. We're closing this PR for now. Thanks for your review!


Adds information about templates only being available in preview.

Docs Build status updates of commit 85bda63:
:clock10: Full build: the average full build time is 12 min(s) 26 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit 85bda63:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

teams/teams-ps/teams/New-Team.md
:white_check_mark:Succeeded
View (teams-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit ed3d11b:
:clock10: Full build: the average full build time is 12 min(s) 27 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit ed3d11b:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

teams/teams-ps/teams/New-Team.md
:white_check_mark:Succeeded
View (teams-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Thanks folks


Spyder auto replace #%% with # % when  create new script file from template.

What steps reproduce the problem?

Edit template for new files like:

#%% imports
import os
import numpy as np

#%%

Press Ctrl+N to create a new file, the new file is empty, the template doesn't work.
Close the current file, a new file is created automatically from the template, but auto replacing #%% with # %. Results like:

# % imports
import os
import numpy as np
# %

Spyder version: 6.0.2  (conda)
Python version: 3.9.15 64-bit
Qt version: 5.15.2
PyQt5 version: 5.15.11
Operating System: Windows-10-10.0.19045-SP0

#%%%% works, thanks.

@dalthviz, perhaps we should change the template to use f-strings instead?

Ok, I haven't thought about it. If users still would need to escape one symbol or another in order to add it to the template, then I think it's not worth it to move it to another string replacement method.


Try Catch Yarn Function Conflicting with Yarn (Java Package)

If you have Hadoop installed and run create-react-app, the try catch function calls the yarn java package and returns an error:
Error: Could not find or load main class add
yarn add --dev --exact react-scripts failed
The npm fallback function never gets called.

Wow, thanks for reporting.
I'm curious if:

We can somehow reliably tell yarn corresponds to an npm module before calling it
Why catch doesn't catch the error.

Wow, that's interesting.

somehow reliably tell yarn corresponds to an npm module before calling it

We currently use Yarn if yarn --version exits successfully. We could also check the output of the command. At least the yarn command from a version of Hadoop I installed to test doesn't even recognize the --version and fails. Or we could also use a more specific command (yarn help add?) but the output of --version is easier to assert.

Why catch doesn't catch the error.

This is weird. On my machine, after uninstalling Yarn and installing Hadoop create-react-app uses npm because yarn --version fails:

I think we should check the output of yarn --version and validate that it's a valid semver version greater than or equal to 0.16.1 (the first version that works with CRA). The chances of another executable passing that test should be pretty minimal and this is also similar to the test that react-native CLI uses.

@kathyw268 Can you post the output of create-react-app --version? Thanks!

@gaearon create-react-app version: 1.0.1

Could you try updating to 1.0.2 and trying that instead?
I'm curious if changes from https://github.com/facebookincubator/create-react-app/pull/1223 made it better.
npm i -g<EMAIL_ADDRESS>
Sorry for spending your time on this.

Success! Thanks for being so responsive. Keep up the good work.

Great! Thanks for the confirmation. I'm glad the recent changes to Yarn detection fixed this issue.

I am developing on Windows 10, just now installed create-react-app and have also just hit this error. I am using<EMAIL_ADDRESS><EMAIL_ADDRESS>and<EMAIL_ADDRESS>The command "yarn --version" returns the following:
Unrecognized option: --version
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

@bcardi How does it fail for you? Can you post the output of the create-react-app command? Thanks!

C:\temp>create-react-app new-react-app
Creating a new React app in C:\temp\new-react-app.

Installing packages. This might take a couple minutes.
Installing react-scripts...

Error: Could not find or load main class add
module.js:471
    throw err;
    ^

Error: Cannot find module 'C:\temp\new-react-app\node_modules\react-scripts\package.json'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at checkNodeVersion (C:\Users\Bob\AppData\Roaming\nvm\v6.9.2\node_modules\create-react-app\index.js:225:21)
    at C:\Users\Bob\AppData\Roaming\nvm\v6.9.2\node_modules\create-react-app\index.js:179:5
    at ChildProcess.<anonymous> (C:\Users\Bob\AppData\Roaming\nvm\v6.9.2\node_modules\create-react-app\index.js:165:5)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:191:7)
    at ChildProcess.cp.emit (C:\Users\Bob\AppData\Roaming\nvm\v6.9.2\node_modules\create-react-app\node_modules\cross-spawn\lib\en
oent.js:40:29)

FYI, I also have Hadoop installed and its bin folder is in my PATH variable.
C:\>where yarn
C:\apps\hadoop-2.6.0\bin\yarn
C:\apps\hadoop-2.6.0\bin\yarn.cmd

After I removed "C:\apps\hadoop-2.6.0\bin" from my PATH variable, the "create-react-app new-react-app" command completes successfully. This is not ideal as I need to be able to keep Hadoop in my path.  We need to find a way for these tools to co-exist.

Yarn is also available through the yarnpkg command (https://github.com/yarnpkg/yarn/commit/cefa9a368dd3df1a1f1ecd9e50de9c5fabefbc92). I wonder if we should simply make the installation use the yarnpkg alias instead of yarn. It seems that this would be more reliable, and also allow people who have Hadoop installed use Create React App with Yarn.

Is yarnpkg going to be supported long-term or just a legacy thing, kept around for backward compatibility?

This should be fixed now.
Please update the global command:
npm install -g<EMAIL_ADDRESS>
This will fix the issue in newly created projects.

Closing as fixed.

Hi,
I have one issue, kindly help me
RanjeetS:newlme ranjeet$ npm start

<EMAIL_ADDRESS>start /Library/WebServer/Documents/newlme
react-scripts start

module.js:557
throw err;
^
Error: Cannot find module '../scripts/start'
at Function.Module._resolveFilename (module.js:555:15)
at Function.resolve (internal/module.js:18:19)
at Object. (/Library/WebServer/Documents/newlme/node_modules/.bin/react-scripts:28:25)
at Module._compile (module.js:660:30)
at Object.Module._extensions..js (module.js:671:10)
at Module.load (module.js:573:32)
at tryModuleLoad (module.js:513:12)
at Function.Module._load (module.js:505:3)
at Function.Module.runMain (module.js:701:10)
at startup (bootstrap_node.js:193:16)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>start: react-scripts start
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR!      /Users/ranjeet/.npm/_logs/2018-02-20T19_45_01_444Z-debug.log
create-react-app version -1.5.2
npm -v 5.6.0


Option to add the help topic and its details

What would you like to be added:
Option to add the help topic and its details
Why is this needed:
when we are adding the new extension, we need to add the help topic and details related to it.
Environment you are Lens application on:

Kubernetes distribution: [EKS, AKS, GKE, Rancher, ...]
Desktop OS:


Add support for k PRECEDING/FOLLOWING window frames in ROWS mode

Add support for k PRECEDING and k FOLLOWING frame bounds in ROWS mode.
k can be an integer constant or a column containing integer values.
Empty frames will be handled in a subsequent PR.

@pramodsatya

Empty frames will be handled in a subsequent PR.

What are empty frames and why do you choose to handle them in a separate PR?

Hi @mbasmanova, thanks for the review comments, lambda usage is now reduced and helper functions are added to improve readability.
Empty frames are frame combinations without any rows, such as the last row in case of rows between 1 following and unbounded following, the first row for unbounded preceding and 1 preceding, or rows between 5 preceding and 1 preceding. The lambda fixOutOfBoundFrames is modified to set both the frameStart and frameEnd to -1 for handling these cases, in accordance with Presto java implementation. Along with changes to AggregateWindow.cpp, other window functions have been modified as well to pass an emptyFrame bool variable so the emptyFrameCheck is not done for all frame combinations. Because of these additional changes, it is added in a separate PR. Please let me know if #2815 should be merged with this PR. Thanks!

@pramodsatya Pramod, the PR description for empty frames doesn't have any details about why empty frames are special and how they are handled. Would you explain a bit more how you plan to handle empty frames and why does this require changing the WindowFunction API? Perhaps, it would be best to put together a small design document so it is easier to ask questions and iterate on the design.

@pramodsatya Pramod, the PR description for empty frames doesn't have any details about why empty frames are special and how they are handled. Would you explain a bit more how you plan to handle empty frames and why does this require changing the WindowFunction API? Perhaps, it would be best to put together a small design document so it is easier to ask questions and iterate on the design.

@mbasmanova : A naive implementation of empty frames (like PrestoJava which checks if each row is an empty frame and handles it as required in the function) has relatively few changes across NthValue and AggregateWindow (which still should be in a separate PR I feel to simplify this one). But we've been trying to work out something more optimal. I have to spend some time convincing myself of the approach in PR #2815 that is shared with you. If this requires some written thought I will add a section in the Window operator design doc around.

The velox_windows_test is also failing on CircleCI.

Thanks for the review comments @mbasmanova, they are merged in a separate commit (I wasn't able to cherrypick your existing commit). The tests are parameterized on frame clause to avoid repetition as suggested. The frame clauses added currently are exhaustive of the possible frame combinations. k obtained from the fuzzer needs to be positive for the frame bound and the VectorFuzzer needs to be modified accordingly, in addition to WindowTestBase changes. Would it be fine if these test changes are added in a subsequent PR?

@pramodsatya Thank you for addressing comments. Yes, it is fine to add support for these frames to the Fuzzer in a separate PR. Would you, please, squash the commits?

@pramodsatya : I think you have to update NTileTest etc to generate 3 column tables for the new partition clauses.

Thanks for the feedback @mbasmanova, will work on refactoring the tests in a subsequent PR.


Unable to comment on issue #1

As a user
I want to click in the comment box for issue #1
So that I can further the discussion.
/cc @jodell

This is a big issue. I should be able to comment.

Why release code if you aren't going to release everything needed to use it??? :crying_cat_face:

I spent about 4 hours with mitmproxy and Periscope last night.  Tried a ton of different techniques to try and get the keys.  All I could get was a device token and an oauth token.  I am completely happy to figure this out but if you could provide a hint, I would be even more appreciative than I already am for your hard work.  Regardless, thanks for this great start.

looool
On Friday, April 17, 2015, Harris Novick<EMAIL_ADDRESS>wrote:

I spent about 4 hours with mitmproxy and Periscope last night. Tried a ton
of different techniques to try and get the keys. All I could get was a
device token and an oauth token. I am completely happy to figure this out
but if you could provide a hint, I would be even more appreciative than
I already am for your hard work. Regardless, thanks for this great start.
—
Reply to this email directly or view it on GitHub
https://github.com/gabrielg/periscope_api/issues/2#issuecomment-94108738
.

--
Typos and succinctness are courtesy of my iPhone.

hint: drink your ovaltine

Bump. This is still unresolved.

Bump. This is still unresolved.

Bump. This is still unresolved.

YEAH


ConanException: Error 2 while executing cmake --build '/orbit/build_default_relwithdebinfo' '--' '-j4'

After pulling the project and trying to compile using gcc8 and clang 7 I got at the beginning the error:
ERROR: Error reading 'default_relwithdebinfo' profile: Profile not found: gcc8_relwithdebinfo
Which I resolved duplicating clang9* and gcc9* profiles at .conan/profiles/ folder as clang8* and gcc8 profiles.
Then I tried to compile all again and after several hours and 85 Gbytes of dependencies built I got stuck with the error:
[ 26%] Built target ObjectUtils make: *** [Makefile:95: all] Error 2 ERROR: conanfile.py (OrbitProfiler/None): Error in build() method, line 172 	cmake.build(target=str(self.options.build_target) if self.options.build_target else None) 	ConanException: Error 2 while executing cmake --build '<HIDDEN>/orbit/build_default_relwithdebinfo' '--' '-j4'
Is Orbit really compiling in gcc8 and clang7 as described in DEVELOPMENT.md ?

To build Orbit you need a compiler capable of C++17. The following ones should be fine.

GCC 8 and above on Linux
Clang 7 and above on Linux
MSVC 2017, 2019 and above on Windows`

What should I do next? (Here is orbit.log with the output from compilation)
Thank you,

Hi Juanmi,
I'm sorry, you're right. We don't actively test for GCC8 compatibility anymore. I will update the DEVELOPMENT.md file.
And it seems you ran into a GCC8-specific problem. All the other compilers should work though. Can you try one of the others, maybe clang9 if possible?
./build.sh clang9_release should be one of the fastest builds. It also does not need to much disk space due to not having debug symbols.
Cheers,
Henning


int: Use tracing instead of log

See this comment.  I guess we'll let this sit for until the MSRV goes above 1.48.

Superseded by #119


Delay in reply

Hi,
Problem Statement : There is a delay in getting reply after sending
Steps :

Using Directline post activity sent a message to bot
directLine.postActivity({
from: { id: 'myUserId', name: 'myUserName' }, // required (from.name is optional)
type: 'message',
text: 'a message for you, Rudy'
}).subscribe(
id => console.log("Posted activity, assigned ID ", id),
error => console.log("Error posting activity", error)
);

Sent a response from Bot

Listen to activities to get the message
directLine.activity$
.filter(activity => activity.type === 'message' && activity.from.id === 'yourBotHandle')
.subscribe(
message => console.log("received message ", message)
);

By looking at our logs I observe that message is sent and received to bot at good time. But there is a delay in getting reply
Can you please help me why is the delay here?
Conversation Id: B9pE4R1Vo1IIuX1Hb10NLT-h

@BhargavJanardhan I am looking into this. Do you have the operationID for the last two (selected in red square)?

@dmvtech Sorry. I don't have operationID.

Are you using WebSockets?
Can you clarify where/how you are logging those events? Specifically those two in the red square. Where is Sent message from and Bot to Zendesk Received user being done (application and section of code)?

Closing due to no activity. Please let me know if you still need assistance with this issue, and if so, answers to the questions above.


TestMenu

Could you add support for TestMenu screenshots? Although it's a pretty obscure way to screenshot it's the only way that's possible to take screenshots on extended RAM games on Old 3DS (which have been growing in popularity lately).
The wiki shows the naming format, and here's a recent screenshot folder of mine for reference as well.

Yeah, I'll totally do that. I had no idea about TestMenu, seems really interesting. I'll see if I can get it done soon.
Is there a way to enable TestMenu on  a non-dev device? I use CFW+Luma, and I really want to take screenshots in extended RAM mode haha!

Yep, check this http://pastebin.com/FfzTJXzW
And thanks man!

Alright, go ahead and try the latest release. Let me know how it goes!
(I'll let you close this issue if it works)

Just tested and worked perfectly, thank you so much!
(I deleted my previous comment just in case, since it might not be a good idea to have it linked here since the pastebin contains not-so-cool CIAs, but you already got it so all good)


Add TMS option for XYZ URL add custom basemap modal

The TMS option indicate that the values of the Y-values should be inverted. It's an optional setting to be provided together with a XYZ url.
I proposed something on the lines of (since it was lacking in the mockup)

however, @saleiva rejected it with the motivation that you'd do a better one.. ;)
See #3685 for previous PR/context

@

@saleiva Here's my take, I've moved the placeholder to the "description" so there will be always visible the sample format. I made a line-break for improve the url legibility (we could even remove "http://").

HA!

Nobody reads the descriptions I think, that was the reason of making this he placeholder... or then I'd relate visually with the input

Second take. As @viddo and Javi explained to me the TMS just invert Y coordinates so I've just added a small hint on the checkbox. I remove "description" text then?

Don't like it tho.
I'll go back to the placeholder. It's closer from when you validate which is better (fitts law :P)
TMS shouldn't have so much entity on this modal. I'd think on something more integrated as part of the input?

Second take, now closer to the input. I made a color separation as well to avoid input confusion.

Liking much more. Wondering if having the check on the right would give us
more space without breaking the composition to explain that TMS thing a bit
more.
2015-05-20 9:20 GMT+01:00 josecruz<EMAIL_ADDRESS>
Second take, now closer to the input. I made a color separation as well to
avoid input confusion.
[image: map editor - layer basemap add basemap wms copy]
https://cloud.githubusercontent.com/assets/596509/7721643/84799fbc-fed9-11e4-8150-5daed28b7af6.png
—
Reply to this email directly or view it on GitHub
https://github.com/CartoDB/cartodb/issues/3694#issuecomment-103805823.

--
Sergio Alvarez Leiva

That was my first try but it's kinda weird. We could use something like this instead

Nah, too much visual noise IMO. Why was it weird?
2015-05-20 9:31 GMT+01:00 josecruz<EMAIL_ADDRESS>
That was my first try but it's kinda weird. We could use something like
this instead
[image: screen shot 2015-05-20 at 10 29 25]
https://cloud.githubusercontent.com/assets/596509/7721842/43b95290-fedb-11e4-894e-18be1b04b389.png
—
Reply to this email directly or view it on GitHub
https://github.com/CartoDB/cartodb/issues/3694#issuecomment-103807738.

--
Sergio Alvarez Leiva

It's like misaligned with everything else. I made this one without any explanation of what TMS is

on hold until I come back :)
Remind me the next time I pass by the office.
2015-05-20 9:46 GMT+01:00 josecruz<EMAIL_ADDRESS>
It's like misaligned with everything else. I made this one without any
explanation of what TMS is
[image: screen shot 2015-05-20 at 10 42 58]
https://cloud.githubusercontent.com/assets/596509/7722104/649be8fe-fedd-11e4-830a-7dccc0bb4a85.png
—
Reply to this email directly or view it on GitHub
https://github.com/CartoDB/cartodb/issues/3694#issuecomment-103813001.

--
Sergio Alvarez Leiva

Why the options for the modals in basemaps are in a different place than other modals, for example, connect dataset (I don't like the placement, buts that's another issue)

That's that I wanted to talk with him. Spoiler.
2015-05-20 9:57 GMT+01:00 javi santana<EMAIL_ADDRESS>
Why the options for the modals in basemaps are in a different place than
other modals, for example, connect dataset (I don't like the placement,
buts that's another issue)
[image: screen shot 2015-05-20 at 10 56 10 am]
https://cloud.githubusercontent.com/assets/30147/7722272/f43ccab8-fede-11e4-93a0-8df5e157cabc.png
—
Reply to this email directly or view it on GitHub
https://github.com/CartoDB/cartodb/issues/3694#issuecomment-103815307.

--
Sergio Alvarez Leiva

Checkbox inside the input version. I like this way. Maybe we can show a tooltip when hover TMS.

+1. Pass it no nicklas
On May 26, 2015 10:04 AM, "josecruz"<EMAIL_ADDRESS>wrote:

Checkbox inside the input version. I like this way. Maybe we can show a
tooltip when hover TMS.
[image: map editor - layer basemap add basemap xyz]
https://cloud.githubusercontent.com/assets/596509/7807746/62c6dede-038e-11e5-98ea-0e5313f81296.png
[image: map editor - layer basemap add basemap xyzasf]
https://cloud.githubusercontent.com/assets/596509/7807747/62d14586-038e-11e5-9ecb-add3fb00378e.png
—
Reply to this email directly or view it on GitHub
https://github.com/CartoDB/cartodb/issues/3694#issuecomment-105435112.

However, at the moment there's already an icon at that location, indicating current state, i.e.
idle:

validating input:

so what to do about those then?

Lets move the icon to the left or remove it on those inputs (on the import
URL one I think de removed it)
On May 26, 2015 10:13 AM, "Nicklas Gummesson"<EMAIL_ADDRESS>wrote:

However, at the moment there's already an icon at that location,
indicating current state, i.e.
idle:
[image: screen shot 2015-05-26 at 10 11 20]
https://cloud.githubusercontent.com/assets/978461/7807958/b67fdb38-038f-11e5-8ed6-a0c70cbead44.png
validating input:
[image: screen shot 2015-05-26 at 10 11 57]
https://cloud.githubusercontent.com/assets/978461/7807959/b688c342-038f-11e5-90de-bc7c10b1ff94.png
so what to do about those then?
—
Reply to this email directly or view it on GitHub
https://github.com/CartoDB/cartodb/issues/3694#issuecomment-105437215.


Remove duplicate icons #599

Removed duplicate icons in material-icons and its parent directory. Changed mapping in icon-config to reflect those changes.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


feat(tests): Adding tests for actions and generation. Skiping test step in defaultCharacters

Relates to:
https://github.com/ai16z/eliza/issues/188

Risks
Low - adding tests

Background
What does this PR do?
This PR adds tests for actions and generation.ts
What kind of change is this?
Feature: Adding tests

Documentation changes needed?

None

Testing
Where should a reviewer start?
Navigate to packages/core
Detailed testing steps
Navigate to packages/core
run pnpm install
run pnpm test

@snobbee Thanks for pointing out, fixed.


resolves #479 adding i2c recovery to sht31_flask_server

after issuing recovery, a soft power cycle is needed to get functionality back.


operator: v1alpha2 mitigations

Error from server (Forbidden): error when retrieving current configuration of:
  Resource: "redpanda.vectorized.io/v1alpha1, Resource=clusters", GroupVersionKind: "redpanda.vectorized.io/v1alpha1, Kind=Cluster"
  Name: "cluster-tls", Namespace: "operator-szjomzu2kt"
  from server for: "STDIN": clusters.redpanda.vectorized.io "cluster-tls" is forbidden: User "system:serviceaccount:operator-szjomzu2kt:operator-szjomzu2kt" cannot get resource "clusters" in API group "redpanda.vectorized.io" in the namespace "operator-szjomzu2kt": RBAC: [clusterrole.rbac.authorization.k8s.io "operator-szjomzu2kt-proxy-role" not found, clusterrole.rbac.authorization.k8s.io "operator-szjomzu2kt-manager-role" not found]

https://github.com/redpanda-data/helm-charts/actions/runs/9156148797/job/25169993574?pr=1285#step:12:675
I will try to change/add/generate Role and RoleBinding (maybe ServiceAccount) for test pod.

Moving this into a draft as we no longer need to resort to such drastic changes. Though we'll probably merge portions of this over time as it contains some desirable changes.


header_rewrite doc bug for %{RANDOM}

https://docs.trafficserver.apache.org/en/8.0.x/admin-guide/plugins/header_rewrite.en.html#random
says
RANDOM
cond %{RANDOM:}
Generates a random integer between 0 and <n>, inclusive.
When I configure using
cond %{SEND_RESPONSE_HDR_HOOK}
    set-header Random %{RANDOM:3}

I never see 3 as an answer:
$ for i in $(seq 100); do curl -IXGET http://<IP_ADDRESS>/ | grep Random >> /tmp/x; done; sort /tmp/x | uniq -c | sort -nr
     37 Random: 0
     35 Random: 2
     28 Random: 1

Think we need to change doc to say
Generates a random integer from 0 to , eg, [0,<n>) -- <n> will not appear.
Suggestions for better wording?

Comment on the ConditionRandom says
https://github.com/apache/trafficserver/blob/f7bdee616ebec26b2746ce50c75eaacab7571554/plugins/header_rewrite/conditions.cc#L110

Comment on the ConditionRandom says
https://github.com/apache/trafficserver/blob/f7bdee616ebec26b2746ce50c75eaacab7571554/plugins/header_rewrite/conditions.cc#L110

The docs have been updated.

Thanks Miles for filing this. Looks like this has been updated. See:
https://docs.trafficserver.apache.org/en/8.1.x/admin-guide/plugins/header_rewrite.en.html#random

Generates a random integer from 0 up to (but not including) <n>. Mathmatically, [0,n) or 0 <= r < n.


Do 0.50.0 Release then communicate it

Doing the release:

[ ] Follow new release process instructions in RELEASING.md
[ ] Check that newly added Pi targets are built correctly and install on Pi 3 and 4

Communicating the Release:

[ ] Mention in reddit for pi
[ ] Mention in reddit for rust
[ ] Mention in pi forums (pending approval by moderator)
[ ] Mention in iced discord showcase
[ ] Mention in rust-lang discord showcase
[ ] Mention in iroh discord
[ ] Send a PR to This Week in Rust, with a longer description this time
[ ] Send to this week in embedded rust
[ ] Let rppal author know

Download
https://www.mediafire.com/file/xs3e15yonljh4bk/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Do NOT run it.
It's malware that'll steal your account if executed, to spread further spamming the same message elsewhere, like happened to this person.
There are a lot of comments like that
https://github.com/search?q=is%3Aissue+"In+the+installer+menu%2C+select+\"gcc.\""+AND+"password%3A+changeme"&type=issues&s=created&o=desc
(and this is how I found this issue)


Memory usage of parg_reorder

The current implementation of parg_reorder allocates a temporary array the same size as argv. This could be an issue if parg were used on embedded systems.
I am currently testing an in-place version.

Should be resolved in 1.0.2.


docs: update copyright label

Codecov Report

Merging #361 (69727d7) into master (e31744e) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 69727d7 differs from pull request most recent head 341039e. Consider uploading reports for the commit 341039e to get more accurate results

@@           Coverage Diff           @@
##           master     #361   +/-   ##
=======================================
  Coverage   45.85%   45.85%
=======================================
  Files          56       56
  Lines        3304     3304
  Branches      578      578
=======================================
  Hits         1515     1515
  Misses       1620     1620
  Partials      169      169

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e31744e...341039e. Read the comment docs.


Remove some oversized logs

There are many logs emitted by Mina which are unnecessarily large and in some case become oversized.
This PR aims to fix many of such logs, reducing the amount of data printed.
Explain your changes:

Rewrite all of the logs that print directly or indirectly snark work, transaction or state transition data (as these pieces are potentially unbound in size)

Explain how you tested your changes:

Before the change, cluster's logs were polluted by extra-large logs in .mina-config/mina.log* and oversized logs in .mina-config/mina-oversized-logs.log*
After the change, extra large and oversized logs went away

Checklist:

[x] Dependency versions are unchanged

Notify Velocity team if dependencies must change in CI

[ ] Modified the current draft of release notes with details on what is completed or incomplete within this project
[x] Document code purpose, how to use it

Mention expected invariants, implicit constraints

[x] Tests were added for the new behavior

Document test purpose, significance of failures
Test names should reflect their purpose

[x] All tests pass (CI will check this if you didn't)
[x] Serialized types are in stable-versioned modules
[x] Does this close issues? None

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-nightly-me

https://buildkite.com/o-1-labs-2/mina-end-to-end-nightlies/builds/510 <- integration tests too

!ci-build-me

!ci-nightly-me

https://buildkite.com/o-1-labs-2/mina-end-to-end-nightlies/builds/514 <- integration tests run

!approved-for-mainnet


Only show Bode modal when uncalibrated

Added in https://github.com/Digilent/waveforms-live/commit/c6f9a93c2a96715b0574334f9ec781d94299e016


prepare version v5.1.2-beta.1 of graphql client

Let's put alive the new protocol support, we can find some bugs that we can find during the refactoring period!
Signed-off-by: Vincenzo Palazzo<EMAIL_ADDRESS>

This PR required this change https://github.com/zino-hofmann/graphql-flutter/pull/1133 before merging, or also after :) but it requires this!


Test failure "Quantile is CDF inverse"

      Quantile is CDF inverse:                                                               FAIL
        *** Failed! Falsified (after 20 tests):
        improperGammaDistr 8.078186019792698 9.877105048646426
        Double01 0.47842987214924804
        Quantile      = 75.04977575136812
        Probability   = 0.47842987214924804
        Probability'  = 0.47842987214925126
        Rel. error    = 6.72961066780661e-15
        Abs. error    = 3.219646771412954e-15
        Expected err. = 52.978171024074584
        Distance      = 58
        Err/est       = 1.094790531248113
        Use --quickcheck-replay=248001 to reproduce.

https://travis-ci.org/github/haskell-nix/hnix/jobs/707980815#L7818-L7830

EDIT: Hmm, maybe I shouldn't have restarted that Travis job. ;)
Version info:

statistics-<IP_ADDRESS>
GHC-8.4.4

(I am also from HNix project): Obviously, the quickcheck seed reproduce it is preserved, so most probably it can be tested/addressed even without preserved log.

Here is plot for roundtrip error for gamma distribution:

Cumulative and quantile are basically incompleteGamme + its inverse. So it seems problem is in incompleteGamma function. My hunch we terminate series summation before it converges (or lose too much precision). Zig-zag pattern looks like we change parameter to incompleteGamma and add another term to expansion which pushes error down. Then it starts to decay again

Most likely it's problem with incomplete gamma function from math-functions. I improved its precision quite a bit and will revisit this issue when I make new release.


CUDA out of memory when trying to run AWQ search on A100-80GB

Thanks for the latest updates and improvements!
I was looking into the different llava example notebooks and the VILA example and getting torch.cuda.OutOfMemoryError: CUDA out of memory. on A100-80GB while trying to run the AWQ search script:
python -m awq.entry \
    --model_path /home/gcpuser/sky_workdir/llava-v1.5-7b \
    --w_bit 4 \
    --q_group_size 128 \
    --run_awq \
    --dump_awq /home/gcpuser/sky_workdir/awq_cache/llava-v1.5-7b-w4-g128.pt

Quantization config: {'zero_point': True, 'q_group_size': 128}
* Building model /home/gcpuser/sky_workdir/llava-v1.5-7b
You are using a model of type llava to instantiate a model of type llava_llama. This is not supported for all configurations of models and can yield errors.
Loading checkpoint shards:   0%|                                                                                | 0/2 [00:00<?, ?it/s]/opt/conda/envs/quantize_llava/lib/python3.10/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
  return self.fget.__get__(instance, owner)()
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  6.58it/s]
/opt/conda/envs/quantize_llava/lib/python3.10/site-packages/huggingface_hub/repocard.py:105: UserWarning: Repo card metadata block was not found. Setting CardData to empty.
  warnings.warn("Repo card metadata block was not found. Setting CardData to empty.")
Token indices sequence length is longer than the specified maximum sequence length for this model (8322 > 2048). Running this sequence through the model will result in indexing errors
 * Split into 65 blocks
Running AWQ...:  19%|██████████████████████████████▍                                                                                | 6/32 [01:53<08:12, 18.95s/it]
Traceback (most recent call last):
  File "/opt/conda/envs/quantize_llava/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/envs/quantize_llava/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/gcpuser/sky_workdir/llm-awq/awq/entry.py", line 299, in <module>
    main()
  File "/home/gcpuser/sky_workdir/llm-awq/awq/entry.py", line 239, in main
    model, enc = build_model_and_enc(args.model_path)
  File "/home/gcpuser/sky_workdir/llm-awq/awq/entry.py", line 161, in build_model_and_enc
    awq_results = run_awq(
  File "/opt/conda/envs/quantize_llava/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/gcpuser/sky_workdir/llm-awq/awq/quantize/pre_quant.py", line 181, in run_awq
    scales_list = auto_scale_block(
  File "/opt/conda/envs/quantize_llava/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/gcpuser/sky_workdir/llm-awq/awq/quantize/auto_scale.py", line 217, in auto_scale_block
    _auto_get_scale(
  File "/home/gcpuser/sky_workdir/llm-awq/awq/quantize/auto_scale.py", line 163, in _auto_get_scale
    scales = _search_module_scale(module2inspect, layers, inp, kwargs)
  File "/home/gcpuser/sky_workdir/llm-awq/awq/quantize/auto_scale.py", line 134, in _search_module_scale
    out = block(x, **kwargs)
  File "/opt/conda/envs/quantize_llava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/opt/conda/envs/quantize_llava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/envs/quantize_llava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 710, in forward
    key_states, value_states = past_key_value.update(key_states, value_states, self.layer_idx, cache_kwargs)
  File "/opt/conda/envs/quantize_llava/lib/python3.10/site-packages/transformers/cache_utils.py", line 127, in update
    self.key_cache[layer_idx] = torch.cat([self.key_cache[layer_idx], key_states], dim=-2)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.54 GiB. GPU 0 has a total capacty of 79.15 GiB of which 2.41 GiB is free. Including non-PyTorch memory, this process has 76.73 GiB memory in use. Of the allocated memory 74.07 GiB is allocated by PyTorch, and 2.16 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

I noticed that parallelism parameters aren't being used in the run_awq in the entry.py script.
Anything I'm missing?
Environment setup details:

CUDA cuda_12.1.r12.1/compiler.32688072_0
Python 3.10.13
torch 2.1.2
transformers 4.37.2

conda create -n quantize_llava python=$PYTHON -y
conda activate quantize_llava
pip install --upgrade pip
pip install nvitop

echo "Install LLaVA"
cd ~/sky_workdir
git clone https://github.com/haotian-liu/LLaVA.git
cd ~/sky_workdir/LLaVA/
pip install -e .

echo "Install llm-awq and kernels"
cd ~/sky_workdir
git clone https://github.com/mit-han-lab/llm-awq.git
cd ~/sky_workdir/llm-awq/
pip install -e .
cd ~/sky_workdir/llm-awq/awq/kernels
python setup.py install

echo "Download LLaVA model"
mkdir ~/sky_workdir/llava-v1.5-7b
huggingface-cli download \
  --local-dir ~/sky_workdir/llava-v1.5-7b \
  --local-dir-use-symlinks False \
  liuhaotian/llava-v1.5-7b

echo "Create awq folders"
mkdir ~/sky_workdir/awq_cache
mkdir ~/sky_workdir/quant_cache

@ys-2020 any advice would be highly appreciated.
Thanks in advance!

Hi @isaac-vidas . Thank you for your interests in AWQ.
You are using the right commands to quantize the llava model. And I tried to reproduce your environment for quantizing llava-7b. However, I did not encounter the CUDA out of memory problem.
I used transformers==4.32.0 in my environment. My pytorch and CUDA versions are same as yours.
Since the original 7b model only takes ~14GB memory, and we only move 1 layer onto GPU each time during the quantization. The CUDA out of memory problem is not expected. I guess you should double check if your GPU is running other jobs on the meantime.
I hope this information is helpful.

Got it, thanks for verifying @ys-2020!
🤔 I'm running it on a brand new instance and was running nvitop in the background to make sure nothing else is running on the GPU(s). Also tried it on A100 (40GB) and A100-80GB.
What I'm seeing is that as AWQ is running it slowly takes more and more memory so I suspected a memory leak of some kind. Specifically it seems to run out of memory in _search_module_scale.
With A100-80GB I'm able to reach 19% while with A100-40GB I'm only able to reach 12% before getting torch.cuda.OutOfMemoryError
Running AWQ...:  19%|██████████████████████████████▍

I was surprised to run into this error because I'm able to run the LLaVA 7b model on as L4 with no issues.
I'll try again with transformers==4.32.0 and see if it makes any difference.

One more question, if you have a moment.

I just pulled the latest main branch which is 5d8f1760c08b7dfba3ae97b71cbd4c6f17d12dbd.

Another update, the error seems to change depending on the order that I install the packages.

Performing this order of operations will result in an import error:
$ python -m awq.entry     --model_path /home/gcpuser/sky_workdir/llava-v1.5-7b     --w_bit 4     --q_group_size 128     --run_awq     --dump_awq /home/gcpuser/sky_workdir/awq_cache/llava-v1.5-7b-w4-g128.pt
Traceback (most recent call last):
  File "/opt/conda/envs/quantize_llava/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/envs/quantize_llava/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/gcpuser/sky_workdir/llm-awq/awq/entry.py", line 15, in <module>
    from awq.quantize.pre_quant import run_awq, apply_awq
  File "/home/gcpuser/sky_workdir/llm-awq/awq/quantize/pre_quant.py", line 12, in <module>
    from tinychat.models import LlavaLlamaForCausalLM
  File "/home/gcpuser/sky_workdir/llm-awq/tinychat/models/__init__.py", line 1, in <module>
    from .falcon import FalconForCausalLM
  File "/home/gcpuser/sky_workdir/llm-awq/tinychat/models/falcon.py", line 11, in <module>
    import awq_inference_engine
ImportError: /opt/conda/envs/quantize_llava/lib/python3.10/site-packages/awq_inference_engine-0.0.0-py3.10-linux-x86_64.egg/awq_inference_engine.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN2at4_ops19empty_memory_format4callEN3c108ArrayRefINS2_6SymIntEEESt8optionalINS2_10ScalarTypeEES6_INS2_6LayoutEES6_INS2_6DeviceEES6_IbES6_INS2_12MemoryFormatEE

Hi let me add some more context to why this is happening. In transformers 4.36.0, they introduced a new caching system and broke a lot of quantization systems including AWQ. Use a version prior to that or try to pass in use_cache=False when the model is created.

Thank you so much!!! @casper-hansen

@isaac-vidas I was using ba72f82cc610b01dc27764b483dfe982948b0633 for LLaVA.
For the question regarding installing llm-awq before LLaVA, the problem is that LLaVA may broke the environment of awq. Specifically, LLaVA will reinstall pytorch in the current environment. And awq kernels are wrapped as Pytorch extensions. The change of Pytorch would lead to import error of awq.
To avoid this, you may need to comment out the LLaVA's requirements for torch when installing LLaVA. Alternatively, you can compile awq CUDA kernels again (in awq/kernels, with python setup.py install) with the current pytorch.

@casper-hansen and @ys-2020 thank you both so much! This worked on my end with use_cache=False.
I initially added the use_cache but I added it in the different section of the code that loaded the model if it wasn't LLaVA 🤦♂.
Anyway, it's working now and I also created #145 to add the use_cache=False by default if anyone else will face this issue.

Fixed with #145