Change mxnet training notebooks to checkpoint the model while training

Altered the mxnet_onnx_export notebook to checkpoint the model for every epoch while training.
Sample training logs -
INFO:root:Epoch[0] Batch [0-100]#011Speed: 51372.08 samples/sec#011accuracy=0.114851
INFO:root:Epoch[0] Batch [100-200]#011Speed: 52639.03 samples/sec#011accuracy=0.112100
INFO:root:Epoch[0] Batch [200-300]#011Speed: 58106.78 samples/sec#011accuracy=0.111000
INFO:root:Epoch[0] Batch [300-400]#011Speed: 52952.43 samples/sec#011accuracy=0.109500
INFO:root:Epoch[0] Batch [400-500]#011Speed: 53285.57 samples/sec#011accuracy=0.113900
INFO:root:Epoch[0] Train-accuracy=0.132133
INFO:root:Epoch[0] Time cost=1.188
INFO:root:Saved checkpoint to "/opt/ml/checkpoints/mnist-0001.params"
INFO:root:Saved optimizer state to "/opt/ml/checkpoints/mnist-0001.states"
INFO:root:Epoch[0] Validation-accuracy=0.358000
INFO:root:Epoch[1] Batch [0-100]#011Speed: 47583.49 samples/sec#011accuracy=0.480495
INFO:root:Epoch[1] Batch [100-200]#011Speed: 51100.07 samples/sec#011accuracy=0.667900
INFO:root:Epoch[1] Batch [200-300]#011Speed: 59368.87 samples/sec#011accuracy=0.781200
INFO:root:Epoch[1] Batch [300-400]#011Speed: 55654.24 samples/sec#011accuracy=0.801700
INFO:root:Epoch[1] Batch [400-500]#011Speed: 53862.97 samples/sec#011accuracy=0.822100
INFO:root:Epoch[1] Train-accuracy=0.731533
INFO:root:Epoch[1] Time cost=1.122
INFO:root:Saved checkpoint to "/opt/ml/checkpoints/mnist-0002.params"
INFO:root:Saved optimizer state to "/opt/ml/checkpoints/mnist-0002.states"
INFO:root:Epoch[1] Validation-accuracy=0.860600
INFO:root:Epoch[2] Batch [0-100]#011Speed: 48318.69 samples/sec#011accuracy=0.862178
INFO:root:Epoch[2] Batch [100-200]#011Speed: 49978.06 samples/sec#011accuracy=0.875500
INFO:root:Epoch[2] Batch [200-300]#011Speed: 48435.09 samples/sec#011accuracy=0.882900
INFO:root:Epoch[2] Batch [300-400]#011Speed: 48435.70 samples/sec#011accuracy=0.897800
INFO:root:Epoch[2] Batch [400-500]#011Speed: 53920.58 samples/sec#011accuracy=0.905800
INFO:root:Epoch[2] Train-accuracy=0.889217
INFO:root:Epoch[2] Time cost=1.195
INFO:root:Saved checkpoint to "/opt/ml/checkpoints/mnist-0003.params"
INFO:root:Saved optimizer state to "/opt/ml/checkpoints/mnist-0003.states"
INFO:root:Epoch[2] Validation-accuracy=0.913900

@sandeep-krishnamurthy please review and also add relevant people for review.

Thanks @anirudhacharya
@ishaaq @mvsusp - Can you please help review.

Besides code changes, I believe we should mention about checkpointing in the notebooks as well, so customers can understand better how the feature works.

I do not understand what you mean by "mention about checkpointing in the notebooks ...". Could you please elaborate?
There is already a line of comment just before we checkpoint a model.

@anirudhacharya each example include a Jupyter notebook detailing the process to use the documented features in that example. See https://github.com/awslabs/amazon-sagemaker-examples/blob/944a8a79f0c6010a2e94cc87bcb0b4f534099dc2/sagemaker-python-sdk/mxnet_gluon_cifar10/mxnet_cifar10_local_mode.ipynb for example. You changed the file that the notebook example is using, but you don't mention anywhere what your feature does or how customers can find additional documentation for the feature.

@anirudhacharya each example include a Jupyter notebook detailing the process to use the documented features in that example. See https://github.com/awslabs/amazon-sagemaker-examples/blob/944a8a79f0c6010a2e94cc87bcb0b4f534099dc2/sagemaker-python-sdk/mxnet_gluon_cifar10/mxnet_cifar10_local_mode.ipynb for example. You changed the file that the notebook example is using, but you don't mention anywhere what your feature does or how customers can find additional documentation for the feature.

@mvsusp addressed this comment. Have added a line of description in each of the notebooks.


Spark 1.2

Makes cooccurrence work on Spark 1.2 (and 1.3) by removing the use of Javaserializer on Guava. Implements a new BiDictionary based on a BiMap, based on two Scala Maps. Only one map is serialized in broadcast.
moved from pushing to master to pushing to mahout-0.10.x branch

this has too much history, going to remerge


Correct reference to Wikipedia page

Line 28, fixed URL to correct subsection reference of Wikipedia page.

@Muarkosh : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Looks good; verified. Thanks for this update @Muarkosh.


how to add security for field and parts in orchard

I need Display Some of the Parts or fields to Anonymous user and prevent from see other

By implementing a ShapeTableProvider, you should be able to provide custom Placement info, which would check the current user and assign "-" to hide specific shapes.
You can also assign some specific content based on that.
Or override the shape templates to check for the current user, that might be the simplest solution.


Error while building $less

New steal while traspliing $less seems to import less module in a strange manner, so not less object with method render is imported but function that returns this object.
OPENING: web/web
error [TypeError: Error loading "styles/base.less!$less" at file:.../styles/base.less
Error loading "styles/base.less!$less" from "web/web" at file:.../web/web.js
Object function (environment, fileManagers) {
    var SourceMapOutput, SourceMapBuilder, ParseTree, ImportManager, Environment;

    var less = {
        version: [2, 4, 0],
        data: require('./data'),
        tree: require('./tree'),
        Environment: (Environment = require("./environment/environment")),
        AbstractFileManager: require("./environment/abstract-file-manager"),
        environment: (environment = new Environment(environment, fileManagers)),
        visitors: require('./visitors'),
        Parser: require('./parser/parser'),
        functions: require('./functions')(environment),
        contexts: require("./contexts"),
        SourceMapOutput: (SourceMapOutput = require('./source-map-output')(environment)),
        SourceMapBuilder: (SourceMapBuilder = require('./source-map-builder')(SourceMapOutput, environment)),
        ParseTree: (ParseTree = require('./parse-tree')(SourceMapBuilder)),
        ImportManager: (ImportManager = require('./import-manager')(environment)),
        render: require("./render")(environment, ParseTree, ImportManager),
        parse: require("./parse")(environment, ParseTree, ImportManager),
        LessError: require('./less-error'),
        transformTree: require('./transform-tree'),
        utils: require('./utils'),
        PluginManager: require('./plugin-manager'),
        logger: require('./logger')
    };

    return less;
} has no method 'render'] TypeError: Error loading "styles/base.less!$less" at file:.../styles/base.less
Error loading "styles/base.less!$less" from "web/web" at file:.../web/web.js
Object function (environment, fileManagers) {
    var SourceMapOutput, SourceMapBuilder, ParseTree, ImportManager, Environment;

Hm I had a strange version of less that imported from less folder not less-node in index.jt


Help with compiling the source code

Describe your problem here.
When I would try to compile this error would pop up annoyingly.
Error: Could not find haxelib "flxanimate", does it need to be installed?
Then I would do:
haxelib install flxanimate
I would try to compile again:
source/psychlua/FlxAnimateFunctions.hx:139: characters 7-18 : flxanimate.FlxAnimate has no field loadAtlasEx (Suggestion: loadAtlas)
How do I solve this? Do I downgrade my Haxe version?
Are you modding a build from source or with Lua?
Source
What is your build target?
Windows x64
Did you edit anything in this build? If so, mention or summarize your changes.
No response

pls can you tell me how do yo solve that?

try to use haxelib git flxanimate https://github.com/ShadowMario/flxanimate dev it should Work if not try to Recompile


Document API endpoints

Summary
Document the API endpoints described in #85, ideally with an automated tool such as OpenAPI/swagger
Requirements
TBD
*Related tickets

#85

It looks like this is relatively straightforward to make with SwaggerUI / SwaggerHub (https://swagger.io/tools/swaggerhub/).  I'll get working on it!

I would only call this a first draft, as there are likely some mistakes in here and probably some missing parameters, but I am hoping this will get us by for now for SIT2:
imap_sdc_api.pdf
I also have an HTML file of this, but GitHub does not allow HTML files to be uploaded here.

That looks cool!
On the upload API, we shouldn't need the parameter "remote_filename".  That's the purpose of the regular (required) "filename" parameter.  If you noticed that it was sending the remote_filename parameter, that's probably an error with the script, I believe it sends all the arguments to the function as query parameters.  It doesn't hurt anything, but it also doesn't do anything.

We now have a documentation page for this! https://imap-processing.readthedocs.io/en/latest/data-access-api/index.html


Mismatch error when load int4 model

When I load the int4 model, I get the following error;
The run command is:  bash scripts/generate.sh --input-source input.txt
I use two a6000 graphics cards (2*48G)
Traceback (most recent call last):
  File "/ssd1/xingyum/GLM-130B/generate.py", line 210, in <module>
    main(args)
  File "/ssd1/xingyum/GLM-130B/generate.py", line 156, in main
    model, tokenizer = initialize_model_and_tokenizer(args)
  File "/ssd1/xingyum/GLM-130B/initialize.py", line 72, in initialize_model_and_tokenizer
    load_checkpoint(model, args)
  File "/home/xingyum/anaconda3/envs/vis/lib/python3.10/site-packages/SwissArmyTransformer/training/model_io.py", line 181, in load_checkpoint
    missing_keys, unexpected_keys = module.load_state_dict(sd['module'], strict=False)
  File "/home/xingyum/anaconda3/envs/vis/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1497, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for GLM130B:
        size mismatch for transformer.word_embeddings.weight: copying a param with shape torch.Size([18816, 12288]) from checkpoint, the shape in current model is torch.Size([75264, 12288]).
        size mismatch for transformer.layers.0.attention.query_key_value.weight: copying a param with shape torch.Size([4608, 12288]) from checkpoint, the shape in current model is torch.Size([18432, 12288]).
        size mismatch for transformer.layers.0.attention.query_key_value.bias: copying a param with shape torch.Size([4608]) from checkpoint, the shape in current model is torch.Size([18432]).
        size mismatch for transformer.layers.0.attention.dense.weight: copying a param with shape torch.Size([12288, 1536]) from checkpoint, the shape in current model is torch.Size([12288, 6144]).

Add from-quantized-checkpoint to your scripts will solve the error.

Thank you for the reply.But I still have the same problem.Due to memory issues, The run command is:  bash scripts/generate.sh --input-source input.txt --from-quantized-checkpoint --sequential-initialization

WARNING:torch.distributed.run:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
*****************************************
WARNING: No training data specified
WARNING: No training data specified
using world size: 2 and model-parallel size: 2
> padded vocab (size: 150528) with 0 dummy tokens (new size: 150528)
> initializing model parallel with size 2
> Set tokenizer as a icetk-glm-130B tokenizer! Now you can get_tokenizer() everywhere.
Namespace(num_layers=70, hidden_size=12288, num_attention_heads=96, vocab_size=150528, max_sequence_length=2048, layernorm_order='post', inner_hidden_size=32768, hidden_size_per_attention_head=None, model_parallel_size=2, skip_init=True, use_gpu_initialization=False, layernorm_epsilon=1e-05, hidden_dropout=0.1, attention_dropout=0.1, make_vocab_size_divisible_by=128, sandwich_ln=False, experiment_name='MyModel', train_iters=1000000, batch_size=4, lr=0.0001, mode='inference', seed=1234, zero_stage=0, checkpoint_activations=False, checkpoint_num_layers=1, fp16=True, bf16=False, gradient_accumulation_steps=1, epochs=None, log_interval=50, summary_dir='', save_args=False, lr_decay_iters=None, lr_decay_style='linear', lr_decay_ratio=0.1, warmup=0.01, weight_decay=0.01, save=None, load='./glm-130b-sat', save_interval=5000, no_save_rng=False, no_load_rng=False, resume_dataloader=False, distributed_backend='nccl', local_rank=0, exit_interval=None, eval_batch_size=None, eval_iters=100, eval_interval=None, strict_eval=False, train_data=None, train_data_weights=None, iterable_dataset=False, valid_data=None, test_data=None, split='1000,1,1', num_workers=1, block_size=10000, tokenizer_type='icetk-glm-130B', temperature=0.9, top_p=0.0, top_k=1, num_beams=4, length_penalty=1.0, no_repeat_ngram_size=3, min_tgt_length=0, out_seq_length=256, input_source='input.txt', output_path='samples', with_id=False, max_inference_batch_size=12, device=0, deepspeed=False, deepspeed_config=None, deepscale=False, deepscale_config=None, deepspeed_mpi=False, cuda=True, rank=0, world_size=2, master_ip='<IP_ADDRESS>', master_port='29500', bminf=False, bminf_memory_limit=44, quantization_bit_width=4, from_quantized_checkpoint=True, sequential_initialization=True, sampling_strategy='BeamSearchStrategy', min_gen_length=0, print_all_beams=False, do_train=False)
> Quantizing model weight to 4 bits
global rank 0 is loading checkpoint ./glm-130b-sat/49300/mp_rank_00_model_states.pt
Traceback (most recent call last):
  File "/ssd1/xingyum/GLM-130B/generate.py", line 210, in <module>
    main(args)
  File "/ssd1/xingyum/GLM-130B/generate.py", line 156, in main
    model, tokenizer = initialize_model_and_tokenizer(args)
  File "/ssd1/xingyum/GLM-130B/initialize.py", line 72, in initialize_model_and_tokenizer
    load_checkpoint(model, args)
  File "/home/xingyum/anaconda3/envs/vis/lib/python3.10/site-packages/SwissArmyTransformer/training/model_io.py", line 181, in load_checkpoint
    missing_keys, unexpected_keys = module.load_state_dict(sd['module'], strict=False)
  File "/home/xingyum/anaconda3/envs/vis/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1497, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for GLM130B:
        size mismatch for transformer.word_embeddings.weight: copying a param with shape torch.Size([18816, 12288]) from checkpoint, the shape in current model is torch.Size([75264, 12288]).
        size mismatch for transformer.layers.0.attention.query_key_value.bias: copying a param with shape torch.Size([4608]) from checkpoint, the shape in current model is torch.Size([18432]).
        size mismatch for transformer.layers.0.attention.query_key_value.weight: copying a param with shape torch.Size([4608, 12288]) from checkpoint, the shape in current model is torch.Size([18432, 6144]).

You should first use the checkpoint conversion script to convert the checkpoint to a 2-way-tensor-parallel style.

Thank you.Is there a limit on memory usage and can complete the convert command? My machine only has more than 160 memory available.

There is an argument to allow sequential conversion with small memory budget in the conversion script. If you have more questions, please feel free to reopen the issue.

I have the same problem and this is the error i am getting
size mismatch for transformer.layers.69.mlp.dense_h_to_4h.weight: copying a param with shape torch.Size([16384, 12288]) from checkpoint, the shape in current model is torch.Size([32768, 6144]).

I am using MB_SIZE=2 because i have only 2 A6000 available and i added --from-quantized-checkpoint to the args on the generate command (I also transformed the model to 4 bit using the convert_tp.py file.


ToC clarification: https://github.com/metanorma/metanorma-jis/issues/13

Metanorma PR checklist

[ ] Breaking changes (list related PRs)
[ ] Documentation update required (create task for this)
[ ] External dependency introduced (documentation update need)
[ ] Gem with native library introduced

merged into doc20230606


Wrong instance of HttpClient get's injected using typed clients with the same name but different namespace

If you encounter a case where you're adding 2 typed HttpClients with the same class name using AddHttpClient<IMyClient, MyClient>(), but from different namespaces. The second instance of HttpClient gets injected into the first one.
To Reproduce
Steps to reproduce the behavior:

Using version the latest version of .NET Core 2
Having this code in Startup.cs

services.AddHttpClient<NameSpaceX.IMyClient, NameSpaceX.MyClient>();
services.AddHttpClient<NameSpaceY.IMyClient, NameSpaceY.MyClient>();

Instance of HttpClient from typed client in NameSpaceY get's injected into typed client of NameSpaceX

Expected behavior
Dependencies of HttpClient in typed clients in NameSpaceY and NameSpaceX are injected as you would expect.
Additional context
In our case this specifically happened with a typed http client from a nuget package which happened to have exactly the same name. I suspect this happens due to the fact fullName is set to false here: https://github.com/aspnet/Extensions/blob/e580a9e1e86b52d55c8c5a084ecb3bd6106156d3/src/HttpClientFactory/Http/src/DependencyInjection/HttpClientFactoryServiceCollectionExtensions.cs#L211
Workaround
Rename either one of the class names of the typed clients.

Looks this is a dupe of #519.
In version >= 3.0.0-preview6 an InvalidOperationException will be thrown:
https://github.com/aspnet/Extensions/blob/17b28e83aa85612289e00fa250ec4647cf26bba5/src/HttpClientFactory/Http/src/DependencyInjection/HttpClientBuilderExtensions.cs#L545-L552

This was fixed as part of the 3.0 release. The right solution if you have multiple types with the same type name is to give them logical names.
services.AddHttpClient(typeof(A.MyClient).FullName).AddTypedClient<A.MyClient>();
services.AddHttpClient(typeof(B.MyClient).FullName).AddTypedClient<B.MyClient>();


Class imbalance

The 5 classes are very imbalanced, occurring roughly with relative frequency (after throwing away the top and bottom 200 slices):

class
occurrence (%)

water
30

background
26

air
4

root
0.1

soil
38

This causes the model to basically ignore the root, and to a lesser extent the air.
I've solved this by computing the relative frequencies of the classes in the training set, and using the inverse of that as weights in the loss function.
This results in a more even error across classes, but we should discuss if that's what we want.

This is implemented in the loss function.


[ARCH][TVMScript] remove relax parser_v1

Per discussed in https://github.com/tlc-pack/relax/issues/312, let's remove the parser_v1

Please also remove the TIR parser accordingly

Please also remove the TIR parser accordingly

One concern is that removing the TIR parser would cause troubles when syncing upstream.

Let us do it one step at a time, starting from relax one. One TIR parser v1 is deprecated in upstream, we can bring it back


Add a tool for gathering data to debug networking problems

This is still slightly a work in progress, but it's basically working (unless people hate what it does and want a total rewrite...).
This adds a script which you can run on the OpenShift master, which will gather data there, on each node, and in each running pod, which can then be sent to a human for debugging purposes. (Automatically diagnosing problems comes next.) Currently this includes:

journalctl --unit openshift-master.service on the master and journalctl --unit openshift-node.service on each node
journalctl --boot on the master and each node. (FIXME: too invasive? Might make admins nervous...)
ip a and ip r on the master, each node, and inside each pod
iptables-save on the master and each node
/etc/hosts from the master and each node
oc get nodes -o json and oc get pods --all-namespaces -o json
brctl show on each node
node-config.yaml from each node
ovs-ofctl -O OpenFlow13 dump-flows br0 on each node
a set of ovs-appctl ofproto/trace outputs on each node, showing traces of up to four different pairs (send/receive) of pod traffic (as many of "packets between local pods in the same namespace", "packets between local pods in different namespaces", "packets between local and remote pod in the same namespace" and "packets between local and remote pod in different namespaces" as it's possible to show given the currently running pods).
the results of every pod attempting to ping every other pod. (FIXME: even with the ping timeout set to 2 seconds, this still really slows things down when using multitenant. It should just do a "representative" sample of pings like with the flow traces.)
the results of every pod attempting to ping www.redhat.com. (FIXME: should do an ofproto/trace test of external network traffic as well)

One catch is that it requires that root@master be able to ssh to root on each node without needing a password. Alternatively, maybe it would make more sense to have the script run from an outside machine that has the ability to ssh to root at the master and each node, rather than running it from the master?
(To test in the vagrant setup, as root:
# root ends up with an erroneous KUBECONFIG when you do "sudo bash"
KUBECONFIG=/vagrant/openshift.local.config/master/admin.kubeconfig
ssh-keygen   # and hit return a few times
scp -pr ~/.ssh openshift-minion-1:.ssh   # root pw is "vagrant"
scp -pr ~/.ssh openshift-minion-2:.ssh
debug.sh

)

Should probably have 'ovs-ofctl -O OpenFlow13 show br0' too so that we get the real port numberss that match up with the ones in the flow tables.

Also lets do 'systemctl status openshift-[master|node]' too; that should get us the command line openshift was launched with and any very recent log messages.

And might as well grab /etc/sysconfig/network-scripts/ifcfg-*, 'nmcli dev', 'nmcli con', and NM journal output too just to isolate any network setup problems that 'ip a' and 'ip r' don't show.

I think we must decide on the operational method of this script (ssh to master/nodes etc). All other improvements can come as further PRs.
I had one proposal:

Run the script in master mode and collect all useful data.
Then run the script in node modes on all nodes.
Run in assemble mode that chews all the data and produces obvious broken points.

The above can be done all at once when password-less ssh is allowed between master/nodes; otherwise it will need manual runs.

Pushed a new version:

Fixes the ping and flow-trace FIXMEs above: the two tests are now merged together into a single "connectivity" test that does both ofproto/trace and ping
Adds ovs-ofctl -O OpenFlow13 show br0 as suggested by dcbw
Adds systemctl show openshift-{master,node} as sort-of-suggested by dcbw; as opposed to systemctl status, this includes more data and is more machine-parseable. It doesn't include journal output but that's ok because we already have that separately.
Adds /etc/sysconfig/network-scripts/ifcfg-*, nmcli -f all dev, and nmcli -f all con as suggested by dcbw (but with the addition of -f all). Doesn't add NM journal output since we already have that in the main journal file.
Can now be run either from the master, ssh'ing to the nodes from there, or from any random machine (taking the name/IP of the master as an argument), ssh'ing to master and nodes from there. In theory you could also run it manually on each machine, passing either --master or --node to tell it what info to gather, and then merge the data yourself...
FIXME just realized that I need to test the case where the master is also a node

Sample output at http://people.redhat.com/dwinship/openshift-sdn-debug-2015-09-18.tgz

Looks excellent.


Added Cycle Sort to Java

this already exists :(

I can see that now, sorry for doing a PR on an already existing algorithm!


HDDS-8496. S3 to return not found for object head/set when keyinfo indicates a directory.

What changes were proposed in this pull request?
This PR fixes a regression introduced by HDDS-7223.
Before HDDS-7223, OBS and FS use-cases use different APIs to get key metadata.

lookupKey: used by OBS, only lookup for the exact key.
lookupFile: used by FS, has additional logic to automatically look for a directory key if the original key doesn't hit. For example, if the key "mydir" is not found, OM tries looking for "mydir/" and returns the result as a directory.

After HDDS-7223, we've unified the two APIs as getKeyInfo, and this API inherits the directory automatic lookup like lookupFile does.
https://github.com/apache/ozone/blob/1e3ddd53f2db7cbc6beb482457ba10aab95570a8/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java#L439-L455
This is incompatible for S3A use-cases (see HDDS-8496).
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-8496
How was this patch tested?
Added acceptance tests.

@duongkame for FSO we can determine if the entry is a file based on table type. We should not need to do any checks with / as the entry will be either a directory or a file based on which table is being accessed.

Waiting for CI runs to finish. The change looks good. Add a comment that this method works only for OBS and Legacy to help readability and add a comment about legacy buckets and directory object's expected behavior

Waiting for CI runs to finish. The change looks good. Add a comment that this method works only for OBS and Legacy to help readability and add a comment about legacy buckets and directory object's expected behavior

Agree. CI looks good now.

Waiting for CI runs to finish. The change looks good. Add a comment that this method works only for OBS and Legacy to help readability and add a comment about legacy buckets and directory object's expected behavior

Agree. CI looks good now.

Looks like CI is still running but I merged the change. Will keep an eye out on the final run to make sure the change is good with CI

Looks like CI is still running but I merged the change. Will keep an eye out on the final run to make sure the change is good with CI

Yeah, only one step is left, rerun it as timed out. https://github.com/duongkame/ozone/actions/runs/4878562216/jobs/8706134345
Nw, I'll make sure the commit on master is green.


Can't pass sensitive output from a module to another one as a (sensitive) variable

I have a module database which outputs the database password as a sensitive output
# terraform/database/outputs.tf
output "password" {
  value     = aws_db_instance.db.password
  sensitive = true
}

I have a module app that depends on the database module and wanted to take the database password as a sensitive variable
# terragrunt/_env/app.hcl
dependency "database" {
  config_path = find_in_parent_folders("database")
}

inputs = {
  db_ password = dependency.database.outputs.password
}

# terraform/app/variables.tf
variable "db_password" {
  type      = string
  sensitive = true
}

When I terragrunt apply the app module, I got the error complaining Unsupported attribute; This object does not have an attribute named "password".. I understand the sensitive=true in the database module's output makes the output not included in the output file. Does the app module take the dependent variable from the output file or the dependent module's state file?

Hi,
reading of sensitive data is supported, maybe that find_in_parent_folders("database") find a different directory with different files
I have a simplified example in https://github.com/denis256/terragrunt-tests/tree/master/sensitive-output-2552
├── README.md
├── app
│   ├── main.tf
│   └── terragrunt.hcl
└── dependency
    ├── main.tf
    └── terragrunt.hcl

# dependency/main.tf
output "password" {
  value = "potato"
  sensitive = true
}

# app/terragrunt.hcl
dependency "d1" {
  config_path = "../dependency"
}

inputs = {
  content = dependency.d1.outputs.password
}

# app/main.tf
variable "content" {}
resource "local_file" "file" {
  content     = var.content
  filename = "${path.module}/file.txt"
}

$ cd app
# since it is first run, I apply on dependency too
$ terragrunt run-all apply
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
$ cat file.txt
potato

@denis256 thank you so much for the example! You're right that reading sensitive output works. I look into this again and realize the problem in my case is that my AWS RDS database was not setup with explicit password but auto-generated with AWS secret manager, so aws_rds_instance.db.password is (effectively) null. So to solve my problem, I need to pass the secret manager id and retrieve the password in the app, instead of relying on the sensitive output/input of terraform.
What is still confusing me is that, when I do something like the following, I can see from the state file that terraform does output this variable password which has a null value.
output "password" {
  value     = null
  sensitive = true
}

But when my value is aws_db_instance.db.password which is a sensitive null (see below), terraform skips including it in the output. No sure if this is expected behavior of terraform but it is somehow misleading.
> aws_db_instance.db.password
(sensitive value)
> nonsensitive(aws_db_instance.db.password)
tostring(null)

Hm, can be used data / aws_secretsmanager_secret_version to read password?
data "aws_secretsmanager_secret_version" "password" {
  secret_id = data.aws_secretsmanager_secret.password
}

data.aws_secretsmanager_secret_version.password


The SQLite file created by the Navicat tool is not recognized

open the sqlite file not found the table however is has a table

@bouyeijiang can you provide a sample database that could help to reproduce an issue?

db.zip

@bouyeijiang unfortunately cannot reproduce it. Connection to database is successfull for me.

what dbeaver version do you use? could you provide, e.g. video of the issue?
is there something in the log?
sorry for the late response

@bouyeijiang

In your example screenshot the table in Navicat has at least two columns (dian, fx) and the provided example database (db.db) has only one column (gz). What's the relation between the two?
Could it be that the db.db file you try to open in dbeaver doesn't contain any table and is not the same as the example you attached to the ticket?
Could you dump the database with the sqlite3 executable?
e.g. on Linux with the attached db.db

$ sqlite3 db.db .dump
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE IF NOT EXISTS "gz" (
"id"  INTEGER NOT NULL,
PRIMARY KEY ("id")
);
COMMIT;


Update conmon to include attach socket unlink

Fixes: containers/libpod#3381

LGTM

LGTM, will merge once the tests are green

/approve
/lgtm


Arduino board correct selection

What board option should i use on the arduino platform.
The board LilyGO T-internet-com is not listed in the options .

any progress on this issue?


Allow amplify to access another bucket

Is this related to a new or existing Amplify category?
storage
Is this related to another service?
S3
Describe the feature you'd like to request
I would like there to be a way to access another bucket through amplify, this would work for my use case where I have a bucket created with amplify where I upload the files through the application but I have another bucket where the files are saved, specifically the photos with a variety of sizes to be used by the application. So when I use Storage.get to get the url I get an invalid url because amplify doesn't have the necessary permissions to sign it or so I imagine.
One proposal is that more than one bucket can be created with amplify, or only existing buckets can be imported, which would be responsible for creating the necessary policies to access that bucket, otherwise it wouldn't make sense to be able to specify the bucket with the Storage.get method
Describe the solution you'd like
Use a command like: amplify import storage
that provides the necessary configurations to be able to use other buckets with the Storage class
Describe alternatives you've considered
I tried duplicating the policies of the s3-cloudformation-template.json file specifying another bucket but it still does not generate a valid url
Additional context
No response
Is this something that you'd be interested in working on?

[ ] 👋 I may be able to implement this feature request
[ ] ⚠ This feature might incur a breaking change

Hey @DeniferSantiago, this is currently supported via the Amplify custom functionality. Please refer to https://docs.amplify.aws/cli/custom/cdk/ for additional information.


Typo in the doc

There is a typo in a comment
"Inititate" -> "Initiate"
https://docs.microsoft.com/en-us/dotnet/desktop/wpf/advanced/walkthrough-enabling-drag-and-drop-on-a-user-control?view=netframeworkdesktop-4.8

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 23f605d0-6326-17d5-3087-4f6ef96ab9bb
Version Independent ID: 10411c7d-3586-4904-6d4f-79e84f0c9b98
Content: Walkthrough: Enabling Drag and Drop on a User Control - WPF .NET Framework
Content Source: dotnet-desktop-guide/framework/wpf/advanced/walkthrough-enabling-drag-and-drop-on-a-user-control.md
Product: dotnet-framework
Technology: dotnet-wpf
GitHub Login: @adegeo
Microsoft Alias: adegeo

Thank you for reporting this. I'll get it fixed.


SUBMARINE-1077. Model descriptive file

What is this PR for?
Create a file under each artifact. This file contains model type, model input dimensions and model output dimensions.
Also, change the artifact save path in minio pod for serving.
What type of PR is it?
Feature
Todos
What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-1077
How should this be tested?
Screenshots (if appropriate)
Questions:

Do the license files need updating? No
Are there breaking changes for older versions? No
Does this need new documentation? Yes

@KUAN-HSUN-LI please help me review the code.


Webpack changes don't align to routes

@AdrianRossouw
Our routes are set up to read files from folders ala
/css
/js
/img

See https://github.com/nearform/vidi-dashboard/blob/master/server/routes/client.js
We need to either change to output of webpack or the routes. @geek set the routes up originally. I think they are set up this way because edge cases in the greedy index route can cause it to be matched wrongly.
The greedy match is to handle routes from the client-side which react router is handling.

can't you just replace that whole thing with a '/dist' and expose that?

Closed via #38


error when adding message to thread

got the following error when running command following API.md
print(requests.post(
'http://<IP_ADDRESS>:8100/threads/266f60d1-ac42-43cd-ab93-f54b7e714971/state',
cookies= {"opengpts_user_id": "foo"}, json={
"values": [{
"content": "hi! my name is bob",
"type": "human",
}]
}
).content)
...
opengpts-backend   |   File "/backend/app/storage.py", line 122, in update_thread_state
opengpts-backend   |     await agent.aupdate_state(
opengpts-backend   |   File "/usr/local/lib/python3.11/site-packages/langgraph/pregel/init.py", line 479, in aupdate_state
opengpts-backend   |     raise InvalidUpdateError("Ambiguous update, specify as_node")

Same issue
Additionally, the issue is not present when when checking the thread state.
Checking thread state...
Thread state: { values: [], next: [] }
Adding prompt to the thread...
Error: Request failed with status code 500
Response data: Internal Server Error
Response status: 500
Response headers: Object [AxiosHeaders] {
'content-length': '21',
'content-type': 'text/plain; charset=utf-8',
date: 'Tue, 07 May 2024 22:21:06 GMT',
server: 'uvicorn'
}
On the endpoint:
INFO:     <IP_ADDRESS>:52084 - "POST /threads/2949af24-9653-404e-8d2d-5210e8c0ffff/state HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
result = await app(  # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in call
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 292, in call
await super().call(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 122, in call
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in call
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in call
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in call
raise exc
File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in call
raise e
File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in call
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 718, in call
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 66, in app
response = await func(request)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 273, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 190, in run_endpoint_function
return await dependant.call(**values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/backend/app/api/threads.py", line 64, in add_thread_state
return await storage.update_thread_state(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/backend/app/storage.py", line 122, in update_thread_state
await agent.aupdate_state(
File "/usr/local/lib/python3.11/site-packages/langgraph/pregel/init.py", line 479, in aupdate_state
raise InvalidUpdateError("Ambiguous update, specify as_node")
langgraph.channels.base.InvalidUpdateError: Ambiguous update, specify as_node

Confirming that this is indeed a valid issue.
The OpenGPTs API has changed, and existing scripts/programs that use it might break.
The payloads the API expects are largely just untyped dict objects. That means any existing scripts or clients generated from the OpenAPI spec could break at any time. I think we desperately need to more strongly type the API.
In the short term, I'll work on updating API.md so it better reflects what the backend actually expects.
Longer term, I feel the OpenGPTs community needs to decide upon and implement a more strongly-typed API. This is important since the untyped nature of the current API makes it inherently unstable.

It looks like the recent changes that decouple agents from threads changed the API a bit. You now need to add messages to a run instead.
Here's a sample of how to use the API now:
# Create a RAG-enabled assistant
resp = requests.post('http://<IP_ADDRESS>:8100/assistants', json={
  "name": "My Asisstant",
  "config": {
      "configurable": {
        "type": "agent",
        "type==agent/agent_type": "GPT 3.5 Turbo",
        "type==agent/system_message": "You are a helpful assistant",
        "type==agent/tools": [
            # {"type": "wikipedia"},
            # enable RAG
            {"type": "retrieval"}
        ]
      }
  },
  "public": False
}, cookies= {"user_id": "foo"}).content

assistant = json.loads(resp)

# Create a thread
thread = json.loads(requests.post('http://<IP_ADDRESS>:8100/threads', cookies= {"user_id": user_id}, json={
    "name": "My Thread",
    "assistant_id": assistant["assistant_id"]

}).content)

# Upload files for RAG
files = {
    'files': ("opengpt_blog_1.txt", open("opengpt_blog_1.txt", 'rb'), 'text/plain'),
}

config = {
        'configurable': {
            # RAG files can be attached to thread or assistants, but not both
            'thread_id': thread['thread_id'],
            # 'assistant_id': assistant['assistant_id'],
        }
}

config = {"config": json.dumps(config)}
cookie = {"user_id": user_id}

response = requests.post('http://localhost:8100/ingest', files=files, cookies=cookie, data=config, headers={'accept': 'application/json'})

# Send a message to the thread
payload = {
  "input" : [
    {
      "content" : "Tell me about OpenGPTs.",
      "type" : "human",
    }
  ],
  "thread_id" : thread["thread_id"]
}

response = requests.post(
    "http://<IP_ADDRESS>:8100/runs/stream",
    json.dumps(payload),
)

print(response.content)

I'll try to get the API.md file updated ASAP. But for now this should give you some ammunition to fix your client apps.

Thanks, confirmed working example


Improper UI label colors in dark themes

Type: Bug Report
Describe the bug

OS and Version: Windows 10 22H2 19045.2604
VS Code Version: 1.75.1
Serial Monitor Extension Version: 0.8.0
Target embedded device: Silabs USB-UART adapter, n/a
Other extensions you installed (and if the issue persists after disabling them): n/a
A clear and concise description of what the bug is: When selecting one of dark themes, e.g. Tomorrow Night Blue, the labels near Mode/Port/Baud/Line ending options are using wrong font color and are almost unreadable.

To Reproduce
Steps to reproduce the behavior:

Select Tomorrow Night Blue theme
Open the Serial Monitor tab
Try to read labels near options.

Expected behavior
Labels should be easily readable and have their font color contrasting with the background.
Code sample and logs
n/a
Screenshots
Note the black-on-blue labels.

Additional context
n/a

谢谢你让我们注意到这一点， @tehKaiN !
我们很快就会对预发布频道进行修复，希望能在明天之前完成，谢谢！

Same problem for me, sir. I'd like to see this to be fixed....just a small bug anyway.

@Eplankton Thanks for confirming that there is an issue. We have released a fix in the most recent pre-release channel! Feel free to try it out there by navigating to the extension pane for the Serial Monitor and selecting, "Switch to pre-release".

@Eplankton Thanks for confirming that there is an issue. We have released a fix in the most recent pre-release channel! Feel free to try it out there by navigating to the extension pane for the Serial Monitor and selecting, "Switch to pre-release".

Thanks for your and the team's efforts, finally we have an official extension for Serial Monitor (which is especially helpful for embedded software developers) , I wonder whether an official extension for memory viewer exists, for now I just use https://github.com/mcu-debug/memview as an alternative.

This should be fixed in v0.9.0. Thanks!


Ga31

Two main improvements:

A new base algebra has been added, G(3,1).
Most issues with accidental conversion to Float64 have been resolved, and new tests have been added for this.

The tests aren't passing... :)
Running autoformat in VSCode should get rid of the linter errors. But I think the main problem is that your new file is called GA31.jl rather than ga31.jl...
I'm guessing you're using windows natively rather than WSL? Because windows uses a case-insensitive filesystem (groan groan groan), it means that the tests will pass for you but fail here, because the CI runners will be using a case SENSITIVE file system.
Anyway, I'll take a proper look at this tomorrow once there are more green ticks and fewer red crosses :)

Sorted the naming issue with AG21.jl and that fixed all the errors. Still do not understand why the formatter keeps suggesting changes that don't actually change anything.

Still do not understand why the formatter keeps suggesting changes that don't actually change anything.

It looks like it's suggesting a newline at the end of the file. This is useful because it stops multiple files running together in bigger diffs and giving odd results.


FR: Start supporting ios 15's interruption-level

Describe your environment

Operating System version: All
Firebase SDK version: v4.7.0
Firebase Product: FCM

Describe the problem
With the release of iOS 15, they've added a new value to the payload:

interruption-level

A string that indicates the importance and delivery timing of a notification. The string values “passive”, “active”, “time-sensitive”, or “critical” correspond to the UNNotificationInterruptionLevel enumeration cases.

The FCM library at the moment doesn't support this key.

You can always use the CustomData field to add parameters that are not explicitly defined on the type.
aps := &messaging.Aps{
  CustomData: map[string]interface{}{
    "interruption-level": "active",
  }
}

The SDK will serialize the json request correctly.

You can always use the CustomData field to add parameters that are not explicitly defined on the type.
aps := &messaging.Aps{
  CustomData: map[string]interface{}{
    "interruption-level": "active",
  }
}

The SDK will serialize the json request correctly.

@hiranya911 You're right. That works. Thanks!


CI: add github action to build & push docker images

For every PR, we build the docker image (to verify the build is not broken).
For every push to master, we build a tag and push it + :latest
For every release (v* tag), we build the tag & :latest + push

Also adds basic badges to the repository

Thanks!

Sorry, should have made it a draft request, there is still an issue with fetching the git version


refactor(iam): versioned v1 clients

Part of the work for #10170
This one was a mixed library, that also has a handwritten backwards compatibility retry_traits.h.
The IAM policy client is also used directly in pubsub samples, so we might as well update that to use the non-deprecated symbols.

This change is

Oops, I messed up some of the handwritten samples. This PR needs some more work before review.


Simplify Introduction to Localization Help Page

This article is an introduction to localisation. It appears to contain a lot of useful information, but I think that users would already need to understand a lot about localisation to understand the article. Can I make the following suggestions.
Anyone looking at this article wants to be able to (a) create a Net Core MVC project, then (b) define a resource in say two languages, then (c) see a view with that string translated, then (d) know how to change the URL, or the code, to change the language used.
The example you've supplied is far far too long. We don't need a triple or four column layout. We don't need 20 links. We need (a) no columns, and (b) one localisation example.
If someone wants to localise a site, and they are starting, and looking at this article, they may want to know (a) how to have a single global resource file, I think a lot of small sites would need no more than this, and (b) how to use resource files at the controller level.
So you need :-

A very very very simple example of a global resource file
A very very very simple example of two non-global resource files

Once you have these two very simple examples, it would be really helpful for users who know very little about i18n but want to used it if you could :-

Comment these files profusely IMPORTANT
Comment every line that you add to the base project produced by Visual Studio IMPORTANT

On a final note, when you click on the feedback link at the bottom of the page, it moves to this page. The problem is that anyone making comments on a page wants to refer to the page when drafting the feedback. So, when users click on the feedback link, it makes much more sense to open a new tab.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d1833676-ff1b-c587-cc9c-81afffa8ef68
Version Independent ID: ad989395-3f49-4d55-521e-cf6fe56281ee
Content: Globalization and localization in ASP.NET Core
Content Source: aspnetcore/fundamentals/localization.md
Product: aspnet-core
Technology: aspnetcore-fundamentals
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

Thanks for the feedback. This is a complicated topic and we don't have the resources to make a quick guide and maintain the complete guide.

Hi Rick
Just to let you know that I quite understand the position, but really
appreciate the feedback.
I'm in the middle of creating a guide for my own use, and if it turns out
to look helpful, I'll keep you posted.
Cheers
Terence
Terence Schoenherr
Head of Development
Travinet Ltd
@.***
www.travinet.com
+44 (0)1493 600954
On Tue, 9 Aug 2022 at 22:37, Rick Anderson @.***> wrote:

Thanks for the feedback. This is a complicated topic and we don't have the
resources to make a quick guide and maintain the complete guide.
—
Reply to this email directly, view it on GitHub
https://github.com/dotnet/AspNetCore.Docs/issues/26676#issuecomment-1209919737,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIQ62AKM6TAHOCZLW3SLKUTVYLFP3ANCNFSM56CFV6QA
.
You are receiving this because you authored the thread.Message ID:
@.***>


Don't have access to the creator wallet.

Hello,  We launched our NFT project 'MetaMercs - MetaCreed Collection' back in November. The Smart contract (Candy machine) we utilized for the project had been configured with our wallet as Royalty payee but a different wallet as Creator. We don't have access to that.
Could you please provide functionality to let us manually set the creator address? Thank you

I have pushed a solution for you in the creator branch which requires the creator to deploy the contract first and the deployer can only be the staking creator or staking admin


Reduce total number of CI builds.

I do not believe there is any reason that we need to test every supported Ember version on every supported Node version.
I think we could probably just default the node version testing to the most restrictive (Node 4) for the ember-try scenario tests (which is what the default ember-cli addon blueprint suggests), and then run the fastboot and node tests once for each Node version supported. This would take our total number of builds from 30 down to a much more manageable number...

Indeed @rwjblue I expect we might get better milage out of testing various versions of Ember-Data instead of so many Node versions.

I definitely defer to yall here.


VS code gets messy

Type: Bug
After i changed my theme, it started to bug out, getting lines all over my VS code and the letters get scrambled(like all the letters, icons get scrambled), I tried to reinstall VScode , removed the extension and even after all this it didnt fix it, evenin the basic dark theme or light theme, its happening now
Please help me fix this
VS Code version: Code 1.70.2 (e4503b30fc78200f846c62cf8091b76ff5547662, 2022-08-16T05:35:13.448Z)
OS version: Windows_NT x64 10.0.22000
Modes:

System Info

Item
Value

CPUs
AMD Ryzen 9 5900HS with Radeon Graphics         (16 x 3294)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_renderer: enabled_onvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.41GB (5.94GB free)

Process Argv
--crash-reporter-id 6b597e05-a9d8-4cce-be91-e3ee70dbcf0d

Screen Reader
no

VM
0%

Extensions (32)

Extension
Author (truncated)
Version

vscode-django
bat
1.10.0

doxdocgen
csc
1.4.0

c-cpp-compile-run
dan
1.0.15

competitive-programming-helper
Div
5.9.2

jupyter
don
1.1.9

python-environment-manager
don
1.0.4

python-extension-pack
don
1.7.0

vscode-html-css
ecm
1.13.1

code-runner
for
0.11.8

better-cpp-syntax
jef
1.15.19

vsc-python-indent
Kev
1.17.0

python
ms-
2022.14.0

vscode-pylance
ms-
2022.9.20

jupyter
ms-
2022.7.1102252217

jupyter-keymap
ms-
1.0.0

jupyter-renderers
ms-
1.0.9

remote-containers
ms-
0.245.2

remote-ssh
ms-
0.84.0

remote-ssh-edit
ms-
0.80.0

remote-wsl
ms-
0.66.3

cmake-tools
ms-
1.12.27

cpptools
ms-
1.12.4

cpptools-extension-pack
ms-
1.3.0

neuron-IPE
neu
1.0.4

autodocstring
njp
0.6.1

LiveServer
rit
5.7.9

ayu
tea
1.0.5

open-in-browser
tec
2.0.0

cmake
twx
0.0.17

vscodeintellicode
Vis
1.2.27

jinja
who
0.0.8

html-css-class-completion
Zig
1.20.0

(1 theme extensions excluded)

A/B Experiments
vsliv368:30146709
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscoreces:30445986
pythondataviewer:30285071
vscod805cf:30301675
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411:30557514
vsaa593:30376534
pythonvs932:30410667
cppdebug:30492333
vscaat:30438848
vsclangdf:30486550
c4g48928:30535728
hb751961:30553087
dsvsc012:30540252
azure-dev_surveyone:30548225
2144e591:30553903
vsccc:30566497
fc301958:30567733

Thanks for creating this issue! It looks like you may be using an old version of VS Code, the latest stable release is 1.71.2. Please try upgrading to the latest version and checking whether this issue remains.
Happy Coding!

Hey Microsoft/Vscode,
Even after updating the vscode, it didn't fix this issue. Here are the
photos of the issue.
Please help me fix it.
thank you
On Sun, Sep 18, 2022 at 3:58 AM VSCode Triage Bot @.***>
wrote:

Thanks for creating this issue! It looks like you may be using an old
version of VS Code, the latest stable release is 1.71.2. Please try
upgrading to the latest version and checking whether this issue remains.
Happy Coding!
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode/issues/161181#issuecomment-1250149713,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ASIK5PMZIABCJLIBEAHODGLV6ZAZBANCNFSM6AAAAAAQPFXX4A
.
You are receiving this because you authored the thread.Message ID:
@.***>

Images emailed to GitHub do not survive the journey. Please drag and drop them directly into the Comment box at the website.
Please also use the Start Extension Bisect command to investigate whether the problem is being caused by one of your extensions.

Its getting worse today, the more I run it the worse it gets

Please close VS Code and then launch it with this command from a Windows Command Prompt-
code --disable-gpu

When I type this command in command prompt it says not recognized as an internal command or external command

I opened it by using this command, let me check if the issue continues

Unfortunately It didn't fix

Can you please  look into it fast, my projects are getting delayed because of this vscode bug, I cant even see my code properly, i have to restart my vscode every time it does and it is very difficult to read the code
thank you

Does the advice at https://github.com/microsoft/vscode/issues/156405#issuecomment-1196503399 help you?

So i gotta close my vscode and use this command to start my vscode?

No. Use F1 in VS Code to open the Command Palette.

oh ok let me try it once

Yeah it was commented out and i changed it now let me see now,  Now the code is more clear the letters are more clear now,  ill Use it for a while and ill let you know if it starts acting weird
thank you

Thank you so much for helping me fix this issue @gjsjohnmurray .


Update en.ts

Create a paragraph break between the two lines so it looks nicer via email.
The idea is for it to look like this:

But we also have to consider if it will look the same in the Description area of the Workspace profile and when you go to invite an individual member to a workspace (the last step where we pre-fill this message).

Explanation of Change

Fixed Issues

$
PROPOSAL:
Tests

Create a workspace as member A
On the workspace profile page, make sure the text in the Description section has a paragraph break (if it doesn't, that's fine too, I mainly want it to come through in the email - just make sure this text still looks OK at least)
Go to the Members page of the workspace and invite member B to the workspace (in the last step of the invite flow, confirm that the pre-filled invite message has a paragraph break - if it doesn't, that's ok too - as long as the text still looks OK I'm mostly worried about the email)
Member B should get the email - confirm the paragraph break is there

[x] Verify that no errors appear in the JS console

Offline tests

QA Steps

// TODO: These must be filled out, or the issue title must include "[No QA]."

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I used JaimeGPT to get English > Spanish translation. I then posted it in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.ts or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] I added unit tests for any new feature or bug fix in this PR to help automatically prevent regressions in this user flow.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

I think this does what you want! Let me know if this looks right:
Workspace page:

Invite message form:

Email:

Workspace chat after being invited:

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.ts or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[ ] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[ ] I verified that all the inputs inside a form are aligned with each other.
[ ] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] For any bug fix or new feature in this PR, I verified that sufficient unit tests are included to prevent regressions in this flow.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

Actually, this really helps to see it laid out like this. I might opt to keep it as it is, after all. Apologies for the extra hustle. The workspace profile page looks a bit odd, in particular,

oh okay - should we close this then?

Yeah, sorry. Is there a way to delete my branch?

done!


Hello!Monitor the data for how long

Description

Environment

Pinpoint version:
OS, WAS, affected library, etc:
Additional details (e.g. how your collector/web is set up):

Additional Info

please find same issue before question.


Export wallets after creation

Is your feature request related to a problem? Please describe.
There is no way to export a wallet after it’s created. So if you forget to backup for some reason, you can’t do it later. Or if you first install in Chrome, but then want it in Firefox as well, you can’t easily get the LNDhub connection string.
Describe the solution you'd like
At least for LNDhub wallets, we could have the option to export the connection string at any time (password protected). This way, your computer and mobile version of the wallet act like backups of eachother.
I don't think we need this for other connectors, but we could consider it.
Describe alternatives you've considered

Inspect /auth network requests to extract the connection string.
Look in the storage to extract the connection string.

Are you working on this? (No)

Feedback from another user:

A botton to export this URL / download QR Code image
A warning on not sharing this

We now got an accounts list. We can have an account detail page that shows the credentials and related information

I think the easiest to do this is to add an "export" menu item and then show the lndhub connection details in a modal.
As we show the details after the account is added: https://github.com/getAlby/lightning-browser-extension/blob/master/src/app/screens/connectors/NewWallet/index.tsx#L127-L141

Any changes? @bumi

I don't think we have the data for the lightning address there. so that would be just the export QR code AND we should add it as a string in an input field. - that allows users to copy it.

At a later stage we could think about adding a "copy to clipboard" button to an input. Not sure if we already have this somewhere. I.e. https://bionicjulia.com/blog/creating-copy-to-clipboard-button-in-react
Should not be done in this issue.


Only build Functions/Sites in the stack passed into sst deploy

Currently if you have 2 stacks (ie. stack1 and stack2), and you run sst deploy stack1, Lambda functions and Static sites in stack2 are also built. This is unnecessary because stack2 won't be deployed.
Requests

Bshr https://serverless-stack.slack.com/archives/C01JG3B20RY/p1645578284748999

We are transition this repo to https://github.com/sst/v2 and are closing this issue for now. If it is still relevant please reopen it there - sorry for the trouble


After error is caught, more partitions are consumed concurrently than are configured

Describe the bug
If the consumer option partitionsConsumedConcurrently is set to 1, I expect there never to be batches that are processed concurrently with different partitions. However, once an error is thrown and caught, I start to see multiple batches with different partitions being processed concurrently. Specifically I have seen this behavior when the heartbeat fails because the session timeout has been reached at the end of a batch being processed. I have a network call that sometimes takes a particularly long time which causes the session to timeout, and I've mimicked that with the example below.
To Reproduce
Run the following snippet after changing the broker address.
const { Kafka } = require('kafkajs')
const group = 'some-group'
const kafka = new Kafka({
  clientId: group,
  brokers: ['<BROKER_ADDRESS>'],
  logLevel: logLevel.DEBUG
})

const subscribe = async (topic) => {
  const consumer = kafka.consumer({ sessionTimeout: 9000, groupId: group })
  await consumer.connect()

  consumer.subscribe({ topic })

  await consumer.run({
    partitionsConsumedConcurrently: 1,
    eachBatch: async ({ batch }) => {
      const batchId = Math.random()

      console.log(`${batchId}: Starting`)
      console.log(`${batchId}: Partition`, batch.partition)

      await new Promise(resolve => setTimeout(resolve, 10000))
      console.log(`${batchId}: Finishing`)
    }
  })
}

const publish = async (topic) => {
  const producer = kafka.producer()
  const messages = Array(100).fill({ value: 'hey hey!' })

  await producer.connect()

  setInterval(async () => {
    await producer.send({
      topic,
      messages
    }, 1000)
  })
}

publish('some-topic')
subscribe('some-topic')

Expected behavior
Because there is a setTimeout awaited that is longer than the configured sessionTimeout on the consumer, I would expect to see 3 log outputs for a single batch, and then an error that the consumer can't be reached. This happens right at the beginning when I run the snippet:
// output
{"level":"INFO","timestamp":"2021-03-24T14:59:09.711Z","logger":"kafkajs","message":"[Consumer] Starting","groupId":"some-group"}
{"level":"INFO","timestamp":"2021-03-24T14:59:12.479Z","logger":"kafkajs","message":"[ConsumerGroup] Consumer has joined the group","groupId":"some-group","memberId":"some-group-70e07c08-9e2a-493f-af41-1415681897bb","leaderId":"some-group-70e07c08-9e2a-493f-af41-1415681897bb","isLeader":true,"memberAssignment":{"some-topic":[0,1,10,11,2,3,4,5,6,7,8,9]},"groupProtocol":"RoundRobinAssigner","duration":1729}*************************************************** 0.7488578966743542: Starting ***************************************************
*************************************************** 0.7488578966743542: Partition 3  ***************************************************
*************************************************** 0.7488578966743542: Finishing ***************************************************
{"level":"ERROR","timestamp":"2021-03-24T14:59:23.829Z","logger":"kafkajs","message":"[Connection] Response Heartbeat(key: 12, version: 1)","broker":"<IP_ADDRESS>:9092","clientId":"some-group","error":"The coordinator is not aware of this member","correlationId":5,"size":10}

Observed behavior
After the first error, I start to see batches being run concurrently. Notice the batchIds are interspersed AND I stop seeing consumer errors from session timeout (presumably because batches start running concurrently and then finish within the session timeout). The snippet below is filtered by INFO and ERROR levels for clarity, but the full DEBUG logs are attached to this ticket here - kafkaJS_output_ticket1053.txt
{"level":"INFO","timestamp":"2021-03-24T14:59:09.711Z","logger":"kafkajs","message":"[Consumer] Starting","groupId":"some-group"}
{"level":"INFO","timestamp":"2021-03-24T14:59:12.479Z","logger":"kafkajs","message":"[ConsumerGroup] Consumer has joined the group","groupId":"some-group","memberId":"some-group-70e07c08-9e2a-493f-af41-1415681897bb","leaderId":"some-group-70e07c08-9e2a-493f-af41-1415681897bb","isLeader":true,"memberAssignment":{"some-topic":[0,1,10,11,2,3,4,5,6,7,8,9]},"groupProtocol":"RoundRobinAssigner","duration":1729}*************************************************** 0.7488578966743542: Starting ***************************************************
*************************************************** 0.7488578966743542: Partition 3  ***************************************************
*************************************************** 0.7488578966743542: Finishing ***************************************************
{"level":"ERROR","timestamp":"2021-03-24T14:59:23.829Z","logger":"kafkajs","message":"[Connection] Response Heartbeat(key: 12, version: 1)","broker":"<IP_ADDRESS>:9092","clientId":"some-group","error":"The coordinator is not aware of this member","correlationId":5,"size":10}
*************************************************** 0.2423671227195845: Starting ***************************************************
*************************************************** 0.2423671227195845: Partition 0  ***************************************************
{"level":"ERROR","timestamp":"2021-03-24T14:59:23.833Z","logger":"kafkajs","message":"[Runner] The coordinator is not aware of this member, re-joining the group","groupId":"some-group","memberId":"some-group-70e07c08-9e2a-493f-af41-1415681897bb","error":"The coordinator is not aware of this member","retryCount":0,"retryTime":344}
{"level":"INFO","timestamp":"2021-03-24T14:59:27.893Z","logger":"kafkajs","message":"[ConsumerGroup] Consumer has joined the group","groupId":"some-group","memberId":"some-group-8fa9463c-8b27-4346-926a-222f33e4e2bf","leaderId":"some-group-8fa9463c-8b27-4346-926a-222f33e4e2bf","isLeader":true,"memberAssignment":{"some-topic":[0,1,10,11,2,3,4,5,6,7,8,9]},"groupProtocol":"RoundRobinAssigner","duration":4060}
*************************************************** 0.5128538695902092: Starting ***************************************************
*************************************************** 0.5128538695902092: Partition 9  ***************************************************
*************************************************** 0.2423671227195845: Finishing ***************************************************
*************************************************** 0.037022592071392246: Starting ***************************************************
*************************************************** 0.037022592071392246: Partition 6  ***************************************************
*************************************************** 0.5128538695902092: Finishing ***************************************************
*************************************************** 0.721932320326417: Starting ***************************************************
*************************************************** 0.721932320326417: Partition 6  ***************************************************
*************************************************** 0.037022592071392246: Finishing ***************************************************
*************************************************** 0.11081440546628074: Starting ***************************************************
*************************************************** 0.11081440546628074: Partition 9  ***************************************************
*************************************************** 0.721932320326417: Finishing ***************************************************
*************************************************** 0.29585173943794363: Starting ***************************************************
*************************************************** 0.29585173943794363: Partition 3  ***************************************************

Environment:

OS: [e.g. Mac OS 10.15.7]
KafkaJS version [e.g. 1.15.0]
Kafka version [e.g. 2.7.0]
NodeJS version [e.g. 14.15.0]

Additional context
Ran snippet on a topic with 12 partitions

#1015 may resolve this.


Pattern-matching: ?? shouldn't get shifted to the end of the pattern

Something like ??*x should match everything in the ??, so we can consistently state that the matching system works from left to right. If you want to capture the x, write the rule as x*??.

Doesn't apply in the new pattern-matching.


request - support scrape_interval override in relabel_configs

In kubernetes/openshift, I would like to be able to use annotations on my pods that define the scrape_interval
i.e. I would like to do something like this in the relabel config :
 - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape_interval]
    action: replace
    regex: (.+)
    target_label: __scrape_interval__
    replacement: $1

Then I could use annotations on pods/deployment configs such as :
 annotations:
   prometheus.io/port: '8081'
   prometheus.io/path: '/prometheus'
   prometheus.io/scrape_interval: '5m'

This has been discussed before and will not be supported. In general you should almost always have the same scrape interval for an entire Prometheus.

We are tackling something in that direction with the prometheus-operator. Currently this only works on Service objects with the respective ServiceMonitor. We are planning to expand this to Pods though.
The idea is that teams or a company likely have some common practice on metrics and that can be expressed with one ServiceMonitor or in the future also a PodMonitor. Feedback is appreciated! 🙂

@brian-brazil I'm interested in knowing the reasoning behind this decision, and I'd like you to reconsider this it.
Some exporters use a lot of resources (examples: prometheus-kafka-consumer-group-exporter, jmx_exporter for Cassandra), and often you're willing to reduce the scrape interval for services to keep the resource utilization down.
In the cloudwatch_exporter the scrape interval is directly linked to the cost of running the exporter. In the recommended setup data newer than 10 minutes is ignored anyway, so scraping it more often than lets say every 5 minutes is a complete waste of money.
A workaround is obviously to setup a separate scraping job that scrapes all pods in that job less frequently, but it's annoying to do it if you have lots of applications with different scrape intervals.

@brian-brazil I have the same use case.  I want to decrease the scrape_interval for the cloudwatch_exporter to something like every 5m, but keep the rest of the on cluster intervals to 60s.
It looks like a custom scrape_interval is supported for scrape_config, just not exposed as a relabel_configs target.

For slow exporters like cloudwatch and certain JMX uses the recommendation is to have a separate scrape config (don't go over 2m). These cases are (thankfully) fairly rare though. This does not require relabelling.

so scraping it more often than lets say every 5 minutes is a complete waste of money.

Delay and precision are different things, so this is not true.


Gospel Checksums

https://johnandersen777.github.io/gospel_checksums/

TODOs

[x] Game Plan
[ ] Decode the Decoding Process

[ ] Chuck Missler - How We Got Our Bible

[x] Transcribe - Love is the essence of the Father
[ ] Notes
[ ] To specs
[ ] To workflows
[ ] Matrixies

[ ] Languages Required

[ ] Vulgate
[ ] Greek
[ ] Hebrew

[ ] Texts

[ ] Wisdom the Goddess
[ ] Essene Gospel Of Peace: Book Two
[ ] The Gospel of the Holy Twelve
[ ] Coptic Gospels
[ ] Gospels

Game Plan
For those gospels that checksum correctly weight at 100 for patterns to follow.
For those that do not checksum but align to the checksum training we weight positively.
For those that do not we know that's the work of deception. Weight those high for anti-pattern avoidance stuff at 100.
Other texts from non biblical sources we can assess and add to training data and weight sources on generations appropriately.
Use concept mapping stuff sent to analyze other texts.
Decode the Decoding Process
python -m pip install --upgrade --force-reinstall git+https://github.com/openai/whisper.git
python -m pip install -U --force-reinstall 'https://github.com/yt-dlp/yt-dlp/archive/refs/heads/master.zip#egg=yt-dlp'

python -m yt_dlp --no-call-home --no-cache-dir -x --audio-format mp3 --add-metadata --audio-quality 0 --restrict-filenames --ignore-errors 'https://www.youtube.com/watch?v=5ZsZLDWWZMs'

Notes
h adds the essence. enoch knew of the flood
Languages Required
Texts

Moved to https://github.com/johnandersen777/johnandersen777.github.io/discussions/3


Create script to modify the AGS service cluster property

Need to automate changing the service cluster. There are situations where a service needs to be moved to a different cluster due to limited server resources

Created script Utilities\ManageServiceProperties.py in Esri\ops-server-config v.next.


Cat - "Inactive" status;  No data in Local Storage; Error message

ENVIRONMENT: Production
URL: https://scrapers.peviitor.ro
Browser: Chrome
Device: Laptop
OS: Windows 10
STEPS TO REPRODUCE:

Open the URL in the browser
Run scraper for "Cat" Company
Check the "Status" section
Chek the "Jobs" section
Check Local Storage
Go to logo -> Click on Careers

EXPECTED RESULTS:
The scraper runs correctly and the message "Active" is displayed in the "Status" section
The scraper displayed correctly Jobs number
Local Storage display all data
Careers link opens correctly
ACTUAL RESULTS:
The "Inactive" message is displayed in the "Status" section
Jobs numer is uknown
The scraper does not return data in Local Storage; Error message is displayed
Careers link open career section, but the message "The requested page "/ro-ro/articole/resurse-umane" could not be found." is displayed
On https://peviitor.ro there are displayed 12 jobs

i don't see any problem, scraper work fine

Retesting. Scraper work fine


Fix minor warnings on windows

Summary
Fix various warnings on Windows due to unused variables
and type conversion from double to integer.

There are a few more warnings on windows that showed up ... I'm going to clean them up in this PR and let you know when this is ready to go


ERROR: attempt to clone a non-affine global reference

Reproducing the behavior
I'm getting the error ERROR: attempt to clone a non-affine global reference. when I call a function to create a Map or List and then pass it to another function. Example:
empty = 0
white = 1
black = 2

def get_initial_board():
  return { 0: black, 1: white, 2: empty }

def get_empty_positions(board):
  # We don't even need to use the board variable, just passing it to this function raises the error
  return [2]

def get_possible_moves(board, pieces):
  fold pieces with pieces_with_possible_moves = []:
    case List/Cons:
      return pieces.tail(List/Cons((pieces.head, get_empty_positions(board)), pieces_with_possible_moves))
    case List/Nil:
      return pieces_with_possible_moves

def main():
  board = get_initial_board() # If I call any function to create the board I get the error
  # board = { 0: black, 1: white, 2: empty } # If I create the board here it works
  pieces = [0]
  possible_moves = get_possible_moves(board, pieces)
  return possible_moves

bend run test.bend -s gives this error:
ERROR: attempt to clone a non-affine global reference.

Errors:
Failed to parse result from HVM.

bend run-c test.bend -s works fine:
Result: [(0, [2])]
- ITRS: 369
- TIME: 0.00s
- MIPS: 0.16

System Settings
Example:

HVM: 2.0.22
Bend: 0.2.36
OS: Apple M2 Max, Sonoma 14.5

Additional context
No response

get_initial_board gets passed as the argument board and then cloned inside get_possible_moves.
This is one of the cases where the heuristic for finding the problematic cases is too strict. This function has some duplications inside it, but not in its normalized form.
The function desugars to Map/set(Map/set(Map/set(Map/empty, 0, black), 1, white), 2, empty), and Map/set duplicates the key (0, 1 and 2 in this example). However, after normalizing, get_initial_board reduces to just an ADT value and not a function so the duplications are not a problem.
Unfortunately this check happens directly at runtime with no way of disabling it for your specific function. Ideally we'd have a type system that can assertain whether or not a value will have problematic duplications, but that's not planned for the immediate future.
For now, you can circumvent this issue by applying a dummy value to the function that forces its evaluation:
def get_initial_board(dummy):
  return { 0: black, 1: white, 2: empty }

...

def main():
  board = get_initial_board(*)


UTF16 encoded text data might get damaged when parsing / decoding.

It appears text data, when UTF16 LE encoded, won't get properly parsed for some frame types.
Consider the following user defined text frame:
00000000: 54 58 58 58 00 00 00 25 00 00 01 ff fe 44 00 49  TXXX...%.....D.I
00000010: 00 53 00 43 00 49 00 44 00 00 00 ff fe 36 00 33  .S.C.I.D.....6.3
00000020: 00 30 00 41 00 43 00 30 00 30 00 38 00 00 00     .0.A.C.0.0.8...
Encoding key is 0x01 which is UTF16 with BOM. The description of that TXXX is "DISCID" and its trailing 'D' is encoded by [0x44, 0x00].
The problem I noticed with UTF16 LE encoded, user defined text frame contents that for the description, the trailing character may get garbled, the following value then starts at the wrong offset, adding garbage at the beginning.
readTillDelims will seek through that frame's byte stream and will try to detect the expected two trailing zero bytes without adding them to the respective frame-data - in this case the description.
The intend is to stop at the delimiter [0x00, 0x00], we do however incorrectly consider that last byte of 'D' rune data which happens to be 0x00 as being the first byte of the delimiter. We end up with truncated UTF16 data.
The user defined text frame parser will then try to read the value starting one byte too early, adding a prefixing 0x00.
The following output is what I get  based on id3v2's functionality;
TXXX: DISCI�: ��630AC008 (encoding: 1)

This nasty workaround -- not a fix!
diff --git a/user_defined_text_frame.go b/user_defined_text_frame.go
index f6cd917..ed722ba 100644
--- a/user_defined_text_frame.go
+++ b/user_defined_text_frame.go
@@ -30,6 +30,7 @@ func (udtf UserDefinedTextFrame) WriteTo(w io.Writer) (n int64, err error) {
 func parseUserDefinedTextFrame(br *bufReader) (Framer, error) {
        encoding := getEncoding(br.ReadByte())
        description := br.ReadText(encoding)
+       description = append(description, br.ReadByte())

        if br.Err() != nil {
                return nil, br.Err()

Then produces the correct output:
TXXX: DISCID: 630AC008 (encoding: 1)

A proper fix needs to be located in readTillDelims, I guess.

Possibly fixed in https://github.com/bogem/id3v2/commit/88f2646e81c1ad6fc4de6006754314e5faaf5634. Can you please test it? 🙏

Thanks a bunch @bogem - I finally go around testing it against some of my previously failed MP3s and all seems fine now - tested  v1.1.3.


remove some axiom annotations

I spoked to @cmungall about this recently - he said that these axiom annotations can be removed:

MONDO:kboom-pr-[number]
MONDO:Lexical
MONDO:ontobio
MONDO:LexicalVariant - these can be removed, should replace all the annotations with an actual URL that describes the pattern (such as Alzheimer vs Alzheimer's)

Could someone do this programmatically?

Note - I added this table to the Editors guide, I'll update it when this is done:
https://mondo.readthedocs.io/en/latest/editors-guide/f-entities/#axiom-annotations-summary-table