export model to TF

hi i am trying to export the model into .pb using https://github.com/amir-abdi/keras_to_tensorflow but i am getting an error :  NameError: name 'tf' is not defined
Any ideas how to convert the model?
Traceback (most recent call last):   File "keras_to_tensorflow.py", line 166, in <module>     app.run(main)   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\absl\app.py", line 300, in run     _run_main(main, args)   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\absl\app.py", line 251, in _run_main     sys.exit(main(argv))   File "keras_to_tensorflow.py", line 112, in main     model = load_model(FLAGS.input_model, FLAGS.input_model_json)   File "keras_to_tensorflow.py", line 61, in load_model     model = keras.models.load_model(input_model_path)   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\keras\engine\saving.py", line 419, in load_model     model = _deserialize_model(f, custom_objects, compile)   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\keras\engine\saving.py", line 225, in _deserialize_model     model = model_from_config(model_config, custom_objects=custom_objects)   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\keras\engine\saving.py", line 458, in model_from_config     return deserialize(config, custom_objects=custom_objects)   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\keras\layers\__init__.py", line 55, in deserialize     printable_module_name='layer')   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\keras\utils\generic_utils.py", line 145, in deserialize_keras_object     list(custom_objects.items())))   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\keras\engine\network.py", line 1022, in from_config     process_layer(layer_data)   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\keras\engine\network.py", line 1008, in process_layer     custom_objects=custom_objects)   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\keras\layers\__init__.py", line 55, in deserialize     printable_module_name='layer')   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\keras\utils\generic_utils.py", line 145, in deserialize_keras_object     list(custom_objects.items())))   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\keras\engine\network.py", line 1032, in from_config     process_node(layer, node_data)   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\keras\engine\network.py", line 991, in process_node     layer(unpack_singleton(input_tensors), **kwargs)   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\keras\engine\base_layer.py", line 457, in __call__     output = self.call(inputs, **kwargs)   File "C:\Users\larro\Anaconda3\envs\tensorflow1\lib\site-packages\keras\layers\core.py", line 687, in call     return self.function(inputs, **arguments)   File "C:\Users\larro\Desktop\keras-yolo2-master\backend.py", line 44, in space_to_depth_x2     return tf.space_to_depth(x, block_size=2) NameError: name 'tf' is not defined

I think you are getting this error because when you use the Full yolo backend it uses tf._space_to_depth in order to create a residual layer, and when you try to load your model the keras function miss the tensorflow library.
you can fix it doing something like this:
import tensorflow as tf
from keras.models import load_model
...
model = load_model("yourModel.h5", custom_objects={"tf":tf})

probably after to fix this error, you will get more errors like this, because the keras will not find functions like the custom loss function for the yolo model.
maybe this function can help you to load the model without problems

I am trying to export the model into TensorFlow serving format and up to now i get all the time errors. I managed to fix the error as you said but afterwards more errors are coming as again you described. I changed the loss function to loss='mean_squared_error' but again.
BTW i am using your fork but still no it doesn't help the function you mentioned. It returns "y_pred"
as argument must be defined.
Is it possible to use YOLO trained with Keras from this script with TensorFlow serving?
Please, i really need help as its part of my thesis to train Yolo and test it using TensorFlow serving

Yes, you can run it directly in tensorflow.
I think that is interesting to you the use of the "get_inference_model", because this function cuts out the lambda layer that is not usefull for inference.
So follow this steps:
1 - Load your model using the YOLO class
yolo = YOLO(backend             = config['model']['backend'],
                input_size          = (config['model']['input_size_h'],config['model']['input_size_w']),
                labels              = config['model']['labels'],
                max_box_per_image   = config['model']['max_box_per_image'],
                anchors             = config['model']['anchors'],
                gray_mode           = config['model']['gray_mode'])

2 - save the inference model
model = yolo.get_inference_model()
model.save("inference.h5")

3 - load the inference model and convert to tensorflow using your script
model = load_model("inference.h5", custom_objects={"tf":tf})

And it worked! @rodrigo2019 you are a god! I inspected the signature of the model and i got this
  The given SavedModel SignatureDef contains the following input(s):
    inputs['input_image'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 416, 416, 3)
        name: input_1:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['YOLO_output/Reshape:0'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 13, 13, 5, 6)
        name: YOLO_output/Reshape:0
  Method name is: tensorflow/serving/predict

The problem is that TensorFlow Serving can't understand the signature as it gives an error
Error occurred: Error: 3 INVALID_ARGUMENT: input tensor alias not found in signature: examples. Inputs expected to be in the set {input_image}.

Thats the only thing that is left. To infer an image with prediction score and boundary boxes. When i tried to predict something using the script the boundary boxes was there and of course the prediction score. Any ideas how to solve this? Thanks again for your help, i really appreciate

I don't know nothing about tensorflow serving, but it looks like that you're not setting the input alias into "input_image" tensor.
Maybe if you share your code I can help you.

i already changed it but now i get
Error occurred: Error: 3 INVALID_ARGUMENT: input must be 4-dimensional[4] 	 [[Node: Full_YOLO_backend/conv_1/Conv2D = Conv2D[T=DT_FLOAT, _output_shapes=[[?,416,416,32]], data_format="NHWC", dilations=[1, 1, 1, 1], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_input_1_0_0, Full_YOLO_backend/conv_1/Conv2D/ReadVariableOp)]]
and the code i run is in Node.js
// Create a protobuf for Tensorflow Serving predict request
    // tslint:disable-next-line:no-any
    private buildPredictRequest(buffer: Array<any>): Object {
        console.log(buffer);
        const request = {
            model_spec: {
                name: this.modelName,
                signature_name: this.signatureName,
            },
            inputs: {
                input_image: {
                    dtype: 'DT_FLOAT',
                    tensor_shape: {
                        dim: {
                            size: buffer.length,
                        },
                    },
                    string_val: buffer,
                },
            },
        };
        return request;

Did you did the preprocessing normalization in the image?
like this:
image = cv2.imread("image.jpg")
image = cv2.resize(image,(416,416))
image = image/255.0
image = image[np.newaxis]

reading the error looks like you forgot to create the fourth dimensionality
doing this:
image = image[np.newaxis]
the image shape will change from [416,416,3] to [1,416,416,3]

Whatever i tried it was impossible for me to do what you said! I don't know why probably NodeJs can't do something like that. I tried also to run as python script and invoke inside NodeJs but still the same, 4 dimensions error.
A question about that is if i do in order to use it in Node in theory it should be with 4 dimensions. Right?
image = image[np.newaxis]
print(image)

Also, is it possible using the predict script to print in console the class, the annotations and the score?

Also, is it possible using the predict script to print in console the class, the annotations and the score?

yes, it is. you shoudl edit the script here

@rodrigo2019 thank you for all your help.

@rodrigo2019 hi again. I am training a custom dataset from scratch but i am having some problems. I am using as i told your repo and i am using CSV for training.
For some reason in the end of every epoch the mAP is zero, for each of the labels.
Also, when i am trying to predict a photo it gives me
Cannot feed value of shape (35, 1024, 1, 1) for Tensor 'Placeholder_110:0', which has shape '(1, 1, 1024, 25)'

i also try to calculate manually the anchors but it runs forever.
Any ideas?

how many classes and anchor box do you have?
some times I get a training that results zero for mAP for around 5hours, after that I start to get some values, usually it occurs in hard problems.
it is hard to tell, but the problems looks like that is your dataset,

Ok OOK!! so it's not from the CSV dataset. this is my config and probably i have to disable early stopping.
{
  "model": {
    "backend": "Full Yolo",
    "input_size_w": 416,
    "input_size_h": 416,
    "gray_mode": false,
    "anchors": [
      0.57273,
      0.677385,
      1.87446,
      2.06253,
      3.33843,
      5.47434,
      7.88282,
      3.52778,
      9.77052,
      9.16828
    ],
    "max_box_per_image": 10,
    "labels": ["eva", "lazaros"]
  },
  "parser_annotation_type": "csv",
  "train": {
    "train_csv_file": "/Users/larry/Documents/Projects/thesis/detector-api/projects/final/annotations.csv",
    "train_csv_base_path": "/Users/larry/Documents/Projects/thesis/detector-api/projects/final/images/",
    "train_image_folder": "",
    "train_annot_folder": "",
    "train_times": 1,
    "pretrained_weights": "/Users/larry/Documents/Projects/thesis/detector-api/projects/final/larrougos.h5",
    "batch_size": 8,
    "learning_rate": 0.0001,
    "nb_epochs": 20,
    "warmup_epochs": 3,
    "workers": 12,
    "max_queue_size": 10,
    "early_stop": true,
    "tensorboard_log_dir": "/Users/larry/Documents/Projects/thesis/detector-api/projects/final/logs/",
    "object_scale": 5,
    "no_object_scale": 1,
    "coord_scale": 1,
    "class_scale": 1,
    "saved_weights_name": "/Users/larry/Documents/Projects/thesis/detector-api/projects/final/larrougos.h5",
    "debug": false
  },
  "valid": {
    "valid_csv_file": "",
    "valid_csv_base_path": "",
    "valid_image_folder": "",
    "valid_annot_folder": "",
    "valid_times": 1
  },
  "backup": {
    "create_backup": false,
    "redirect_model": true,
    "backup_path": "../backup",
    "backup_prefix": "Tiny_yolo_VOC"
  }
}

You have 2 classes and 5 anchors, it results in a array with size 35, so it looks you are feeding a correct size, but the network are expecting a size 25, maybe you have found a bug in the CSV parser, could you try using labels: [] ?

I think, i think that it was nothing more that i stopped the training early and the second time i tried it didn't gave me the error. Of course the training is very poor as from 5 images i tried it only recognised one image and really bad.
The error was for sure when i had labels: [], after i prefilled the labels with the two classes. But i dont think thats the problem for sure, bz i can see that from the csv parser it find two classes.

What are the preferable epochs for small dataset?
accuracy, loss and what percentage would be acceptable?

I do not know, I think you need discover it by the empirical way

i think i've found a bug as you mentioned yesterday. If in my config i don't have any labels during training everything is fine, with one class. If i try to predict again sthg with the same config i get the
Cannot feed value of shape (30, 1024, 1, 1) for Tensor 'Placeholder_110:0', which has shape '(1, 1, 1024, 25)'

The error disappear if i add the class into the label array.
Could it be the case that because of that i always have 0 mAP and i need to have also for the training the classes?

I think the problem is because you have a small dataset, and when the dataset is split into train and validation, the training dataset don't have all classes for training.

Right now my dataset is 230 images, and for the training it uses 201 images. Do i need more?
Right now i have
acc : 0.4
loss : 0.0644
val_acc : 0.4073
val_loss : 0.0062
mAP : 404

I just started the training again but this time i filled inside the config the labels rather depending on the the parser. Let's see.

hope this helps you ! https://github.com/YunYang1994/tensorflow-yolov3

@ricardo-0x07  in my fork


Make a setup script

Added a script that does all the node/npm commands

Hey, it would be great if u add a full setup with like a web installer to this pr. I also can help you to complete this pr.

would love the help!!

By setting isAdmin in the db to 1

removed a security vulnerability

why did thank you close this pr


Configurable Products not accessible in CSR when children visibility is set to "not visible individually"

Current behavior
In 1.9 one cannot access Configurable Products in CSR (e.g. clicked from category listing), when the visibility of the children is set to "not visible individually" in Magento. Instead one is redirected to Homepage with error message.
For further details view this slack thread: https://vuestorefront.slack.com/archives/CA68P1JUF/p1557923071162700
Expected behavior
This was working in versions before 1.9
Can you handle fixing this bug by yourself?

[x] YES
[ ] NO

Which Release Cycle state this refers to? Info for developer.
Pick one option.

[ ] This is a bug report for test version on https://test.storefrontcloud.io - In this case Developer should create branch from develop branch and create Pull Request 2. Feature / Improvement back to develop.
[ ] This is a bug report for current Release Candidate version on https://next.storefrontcloud.io - In this case Developer should create branch from release branch and create Pull Request 3. Stabilisation fix back to release.
[x] This is a bug report for current Stable version on https://demo.storefrontcloud.io and should be placed in next stable version hotfix - In this case Developer should create branch from hotfix or master branch and create Pull Request 4. Hotfix back to hotfix.

Additional information
I will provide a PR with a fix.

Could You please paste the info form Slack into comments to avoid the risk of Slack removing the history due to messages limit (we're on free version)?

yogeshsuhagiya   [7 days ago]
I'm also facing the same issue after upgrading VS version
ZigaBBS   [7 days ago]
Me too, maybe it is problem in default.json config file, im setting it up now will report when i test
yogeshsuhagiya   [7 days ago]
@ZigaBBS Don't think so, there is something wrong with core code.
yogeshsuhagiya   [7 days ago]
VSF team developed and used RouterManager, might be this will be the reason.
yogeshsuhagiya   [7 days ago]
They replaced
import routes from './router'
with
import { RouterManager } from '@vue-storefront/core/lib/router-manager' (edited)
Jens   [7 days ago]
This is no problem, I suggest copying the latest default.json to local.json so you don’t forget anything.
Jens   [7 days ago]
As the default file has changed a bit, see: https://github.com/DivanteLtd/vue-storefront/commit/be5d10d183de9a257a68251828c07ca2161290f4#diff-1e9c3d615e9ebaaaa3669b4c2fd87d00 (edited)
Jens   [7 days ago]
And:
https://github.com/DivanteLtd/vue-storefront/commit/93949bc70bc3a1303500815b71b49d1fc98ba3f6#diff-1e9c3d615e9ebaaaa3669b4c2fd87d00
reVLis   [7 days ago]
I'm just in the process of upgrading to 1.9 and experiencing the same issue: configurable products do not load in csr when clicked from category page....same url is working in SSR though...urlDispatcher is set to false.  I will start investigating and any help is welcome :smile: (edited)
yogeshsuhagiya   [7 days ago]
@Jens I've added all latest changes of default.json file into local.json file but still issue is there
ZigaBBS   [7 days ago]
Same...
pkarw (Core Team)   [7 days ago]
please reindex the products with the latest version of mage2vs then run the yarn db rebuild in the vs-api and make sure the product.url and category.url fields are set in the Elasticsearch - if so, it should work jsut fine
pkarw (Core Team)   [7 days ago]
You can also set config.useUrlDispatcher = false in order to have the backward compatibility
pkarw (Core Team)   [7 days ago]
@degi - working on cookbook this is something that we probably emphasise in the docs -> I mean runing the yarn db rebuild after importin the products / upgrading to 1.9 as the url_path has keyword  data type and won’t work on defualt string
Mateus Nunes   [6 days ago]
@pkarw (Core Team)  i had the same problem that @Jailson Dias  was describing, the funny thing is: on the vsf If im on the home page And click on the product the CSM error do pop up, but If go direct link ( open a new tab And paste the product URL ) It works Just fine, show the product without any errors, If you have any ideia on How to fix this or any clue on what might be happening please let me know here
reVLis   [6 days ago]
I still could not pinpoint the issue. Regardless if using UrlDispatcher or not: SSR is working for all products, CSR is working for all products when clicked FROM SEARCH. CSR is working for SImple Products when clicked from category. CSR is not working for nearly all configurable products from category - though i have a handful of configurable products that do work - could not make out any difference....definately something funky going on here (edited)
yogeshsuhagiya   [6 days ago]
@reVLis I got the solution, DM me
Jens   [6 days ago]
@yogeshsuhagiya Could you share the solution in form of a PR maybe?
yogeshsuhagiya   [6 days ago]
@Jens
Step 1: Open vue-storefront/core/modules/catalog/store/product/actions.ts
Step 2: Goto line number 601 in fetch method
Step 3: Find following code inside fetch method
``
if (product.visibility === 1) { // not visible individually (https://magento.stackexchange.com/questions/171584/magento-2-table-name-for-product-visibility)
throw new Error(Product query returned empty result product visibility = ${product.visibility})
}
*Step 4:* Comment out that code or just remove it

*Note: It's no a genuine solution, it's only temporary fix* (edited)

reVLis   [6 days ago]
This works with the side-effect of having configurable children accessible via url

Jens   [6 days ago]
@yogeshsuhagiya Could you please make a GH issue describing your problem, with this temporary solution for others experiencing the same? I’m not experiencing it so I don’t think I could make it clear enough

reVLis   [6 days ago]
In short: one cannot access configurable products when the visibility of the configurable children is set to "not visible individually" in magento (edited)

yogeshsuhagiya   [6 days ago]
@Jens I think it will be hopefully fix in new release of VSF and some issues are already there in GH relate to this issue

reVLis   [6 days ago]
@yogeshsuhagiya can you share which GH-issues that are? Because else I will create one for this specific problem to raise awareness...

yogeshsuhagiya   [6 days ago]
@reVLis can you please open a GH issue?

reVLis   [6 days ago]
Will do

pkarw (Core Team)   [6 days ago]
@reVLis this issue occurs when just >one< of the configurable_children is not visible, correct?

pkarw (Core Team)   [6 days ago]
very valuable insights, thanks Guys!

pkarw (Core Team)   [6 days ago]
Please do create an issue or even beter - aPR with a fix :smile: I belive it may be caused by merging the configurable_children with the current product >before< checking it status - i mean `product/configure` or `cofnigureProductAsync` helper is being called from the `product/list`https://github.com/DivanteLtd/vue-storefront/blob/2646de2905174c4d20c6f3365769b0a90cef353a/core/modules/catalog/store/product/actions.ts#L309
core/modules/catalog/store/product/actions.ts:309
            ```let selectedVariant = configureProductAsync(context, { product: product, configuration: configuration, selectDefaultVariant: false })```
DivanteLtd/vue-storefrontAdded by GitHub

pkarw (Core Team)   [6 days ago]
I think that we could check all the configurable_children visibiltiy before this line (309) and then set the `product.visibility` like a kind of `max(product.visibility, configurable_children.visibility)` - which will disable the product ONLY when all the configurable_children + the product itself are not visible (edited)

pkarw (Core Team)   [6 days ago]
However, the issue @Mateus Nunes described can be caused by the other reason - both SSR and CSR (browser) modes must have the access to the `config.elasticsearch.host` address. Would be great to ahve a Feature request to add a proper error message in case is not accessible (currently we don’t have any error message in a case of Ealsticsearch not accessible)

reVLis   [6 days ago]
If we just do not overwrite Parent-Visibility with Child visibility on line 309, then everything should work fine!? I think visibility of the configurable should be derived only from itself and not any children
e.g. :
```let selectedVariant = configureProductAsync(context, { product: product, configuration: configuration, selectDefaultVariant: false })
delete selectedVariant['visibility']
Object.assign(product, selectedVariant)```
(edited)

pkarw (Core Team)   [6 days ago]
OK, I belive it’s better to use the `omit`

pkarw (Core Team)   [6 days ago]
like in here: https://github.com/DivanteLtd/vue-storefront/blob/2646de2905174c4d20c6f3365769b0a90cef353a/core/pages/Product.js#L173
core/pages/Product.js:173
        ```this.$store.dispatch('product/setCurrent', omit(product, ['name']))```
DivanteLtd/vue-storefrontAdded by GitHub

pkarw (Core Team)   [6 days ago]
Could You please propose a PR for that?

reVLis   [6 days ago]
Will do...it's hotfix stuff for 1.9.1, right?

pkarw (Core Team)   [6 days ago]
Yeah, sounds like a hotfix

pkarw (Core Team)   [6 days ago]
Thanks man!

Thanks @revlis-x !

Just for the records and backlinks I had this problem described in the forum too (the fix works):
https://forum.vuestorefront.io/t/product-links-route-to-home-with-offline-message/296


Health Monitors not updating

First push up this declaration containing a health monitor in the common partition:
{
    "class": "AS3",
    "action": "deploy",
    "declaration": {
        "class": "ADC",
        "schemaVersion": "3.12.0",
        "id": "0",
        "Common": {
            "class": "Tenant",
            "Shared": {
                "class": "Application",
                "template": "shared",
                "test_http_monitor": {
                    "class": "Monitor",
                    "monitorType": "http",
                    "send": "GET /keepalive.txt",
                    "receive": "200"
                }
            }
        }
    }
}

Then push up this declaration to create a VIP with a pool using the health monitor in the Common/Shared partition:
{
    "class": "AS3",
    "action": "deploy",
    "declaration": {
        "class": "ADC",
        "schemaVersion": "3.12.0",
        "id": "0",
        "shortlived": {
            "class": "Tenant",
            "myhui1085_dev": {
                "class": "Application",
                "template": "generic",
                "myhui1085_dev": {
                    "class": "Service_HTTP",
                    "snat": "self",
                    "virtualAddresses": [
                        "<IP_ADDRESS>"
                    ],
                    "pool": "myhui1085_dev_pool"
                },
                "myhui1085_dev_pool": {
                    "class": "Pool",
                    "monitors": [{
                        "bigip": "/Common/Shared/test_http_monitor"
                    }],
                    "loadBalancingMode": "least-connections-member",
                    "members": [
                                                {
                            "adminState": "enable",
                            "servicePort": 80,
                            "serverAddresses": [
                                "<IP_ADDRESS>"
                            ],
                            "hostname": "test_server2"
                        }
                    ]
                }
            }
        }
    }
}

Now, Finally push up this declaration to use a health monitor created in the shortlived tenant:
{
    "class": "AS3",
    "action": "deploy",
    "declaration": {
        "class": "ADC",
        "schemaVersion": "3.12.0",
        "id": "0",
        "shortlived": {
            "class": "Tenant",
            "myhui1085_dev": {
                "class": "Application",
                "template": "generic",
                "myhui1085_dev": {
                    "class": "Service_HTTP",
                    "snat": "self",
                    "virtualAddresses": [
                        "<IP_ADDRESS>"
                    ],
                    "pool": "myhui1085_dev_pool"
                },
                "myhui1085_dev_pool": {
                    "class": "Pool",
                    "monitors": [{
                        "use": "test2_http_monitor"
                    }],
                    "loadBalancingMode": "least-connections-member",
                    "members": [
                                                {
                            "adminState": "enable",
                            "servicePort": 80,
                            "serverAddresses": [
                                "<IP_ADDRESS>"
                            ],
                            "hostname": "test_server2"
                        }
                    ]
                },
                "test2_http_monitor": {
                    "class": "Monitor",
                    "monitorType": "http",
                    "send": "GET /keepalive.txt",
                    "receive": "200"
                }
            }
        }
    }
}

What I have experienced is that the health monitor doesn't update to the new monitor that i have defined in the final declaration.

In the future, please use the templates when submitting an issue.
Most likely the monitor is actually changed, and the change can be confirmed with tmsh. The BIG-IP GUI has a bug where it will continue to show the old monitor, even though tmsh shows the new monitor. Can you confirm that this is what you are encountering?

This is correct. If i look at the AS3 declaration as well as tmsh I can see that the monitor was updated. I would also like to point out that if i try to remove the "test_http_monitor" in my example after updating to the new monitor, i can't deleteit. Even if i remove all configurations except that "test_http_monitor" monitor, i can't delete it as the f5 says it's in use.

We are experiencing the same issue, trying to delete a monitor that is no longer associated to anything fails with error in-use.
Jun  9 11:29:22 <hostname> err mcpd[26081]: 01070083:3: Monitor /<path>/hm_2f4ccc65-183d-46aa-a688-8dec3f5257e7 is in use.

@penguin2839 @m-kratochvil, The issue you are experience where you cannot delete the tenant is different from the BIG-IP GUI bug. It is a bug in tmsh transactions. More details on that issue can be found in #110.


Long video support ？

The current code reads all video frames at once,
# Returns an array containing all images read from a compressed video file.
video_data = media.read_video(path_input)

This is suitable for short videos of 10 seconds. If the video is longer than 10 minutes.
The GPU VRAM or memory will crash.
How to do it for long videos ? Thank you.

Hi, thank you for asking!
For longer videos, I think our model should work well. Even though current code reads all video frames at once, the whole video is processed in a streaming manner within the ChronoDepthPipeline. Like when n_token=10, our model processes 10-frame segments sequentially. This approach ensures that our model remains efficient in managing GPU VRAM, even for longer video durations.
Let me know whether your issue has been solved.

Hi, Thanks for the great job!
The sample video worked fine, but it crashes after about 2 minutes of video.
Loading pipeline components...: 100%|█| 5/5 [00:00<00:00, 13.38
Killed

What specs are required for it to work?
Or can you give me a solution?
Thank you.

Hi, thanks for your interest!
There are several ways to save GPU memories:

pass the arguments --cpu_offload=model
decrease the '--chunk_size', e. g.  '--chunk_size=1'
reduce the '--max_res' but it will slightly introduce decrease on performance


I want to get WidgetRef, using useWidgetRef hook.

I request to make new hook, useWidgetRef.
There is useContext() in flutter hook which returns BuildContext.
Like this, I want to use useWidgetRef hook which returns WidgetRef.
Then, this hook should be used in only HookConsumerWidget.

You can do WidgetRef useWidgetRef() => useContext as WidgetRef;

I don't plan on adding such a hook for now. This can be revisited in the future though

Maybe a useRiverpod() hook which returns the WidgetRef could be a neat alternative to to the HookConsumerWidget?


[Documentation] Github Pages returning 404 on (most) documentation sections

Hi, I'm trying out the library but I'm having troubles with the documentation website. As mentioned, most of the pages there return a 404 error.
An example: https://tcapelle.github.io/timeseries_fastai/models.html

Thanks to let me know, I think is a problem of nbdev.
I will put the sidebar on custom to solve this issue.

shuold be good now.
Anyway, I will suggest to read the jupyter notebooks directly instead of the doc, so you can play with the code.


Adding website link and Profile link to mail

It would easier for users to click on the links from mail and access the website.
Also updating title attribute of IMG tag and adding link to image if needed.

@RishabhNaik can you take this issue

@RishabhNaik can you take this issue

Yes

Yes

#152 #153

#152 #153


add closable prop to modal

fixes #17

adding another with default: true to maintain existing default functionality


Accesing "this" inside a function

If I call
 (js-invoke (js-closure (lambda (props)
                                            (let* ((this (js-eval "this"))
                                                   (result (js-invoke _react-component 'call this props)))
                                              (if (js-undefined? result)
                                                  this
                                                  result))))
                                         'bind
                                         _react-component))

this returns the window object, not the this object I passed in.
Would I need to make a js-closure-with-this lib func that passes in this and then args?

Just wanted to say "me too" here; I workaround by defining a wrapper function thiswrap https://github.com/yuniscm/yuniappjs-pkg/blob/e54a2ce2124c4ad85a7063d7640cd1cd3346789e/entry_point.js#L69 and receiving JavaScript this through callback to generate this-accessing-JS-function
;; the-render will be a JS function
(define the-render (js-call thiswrap (js-closure (lambda (the-this) (let ((props (js-ref the-this "props"))) ...))))

Wish I could pick your brain after looking through that repo! Really neat (over-my-head-mostly) stuff!


Remove jake (hopefully for real this time)

A couple of our build tasks will need to be updated, too; but I think this should be all that's needed in TS itself.

Oooooo that feeling when you put async functions in the gulpfile but node 6 doesn't support them natively. @rbuckton got a preference on how to handle that?

I would like to keep jake around until
> gulp
> gulp

Has the second invocation run fast, just like jake does today.

Well...
Now that's true.

Do you need to add the change to support using LKG on CI that we added to jake?

I'm doing something a little different - one second, just validating it a bit.

There we go - rather than disabling lkg on CI (which leads to frustrating test diffs locally), I'm now running a sanity check built/local build on CI only after tests pass.

Node 6 is now out in favour of node 11, so that's one more obstacle out of the way.

Alright, looks like everything is now green. Anyone wanna peek over the CI logs to see that you're OK with the now gulp-based ones and give this the :+1:?

Alright. All build pipelines are on gulp-based scripts and the sync script for VS is setup to use gulp, too. Once this build passes now that I've synced with master and resolved the package.json conflict, I plan to merge this. That should unblock merging user suite PRs verbatim, and make the code coverage pipeline green again. This PR's been up for a week since approval, but if anyone has any final words before this merges, the next hour or so is what's left (barring catastrophic failures requiring a revert).

Wooo, this scared me.（bye jake）

@weswigham how do you have incremental builds now?

Our gulpfile is shelling out to tsc -b and not using gulp-typescript is how. :neutral_face:


i claimed my BCH

I Claimed my BCH but when I try to transfer it out it said repeat ... so I just left it in the wallet. its been a long time now. I was thinking maybe one day electrum cash may one day have a support. what do you think can I claim this I have the string and private keys.

i don't understand your issue.  What said "repeat"?  How did you claim?  Transfer out of where?  Please be more specific exactly whats going on here.

I had my bitcoin in my electrum wallet at the time of the fork. I opened a
electrum cash wallet and entered in my private ket and it shows I have a
balance but it wont let me send anything out of the wallet. it gives me a
error
On Tue, Dec 19, 2017 at 9:42 PM, fyookball<EMAIL_ADDRESS>wrote:

i don't understand your issue. What said "repeat"? How did you claim?
Transfer out of where? Please be more specific exactly whats going on here.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/fyookball/electrum/issues/355#issuecomment-352970268,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AhFzgRNhoEyx-4lcyKBpDzsFU0n3Yv-Rks5tCJ4ygaJpZM4RHzFO
.

--
[image: --]
james Demongin 1
[image: https://]about.me/jamesdemongin1
https://about.me/jamesdemongin1?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links


I wish /docs/admin/federation/index/ existed

I wish /docs/admin/federation/index/ existed
Seems to me that the link from this page http://kubernetes.io/docs/admin/dns/ is pointing to /docs/admin/federation/index.md, but the page does not exist.

Dup of #901?

Link fixed: http://kubernetes.io/docs/admin/federation/


python3Packages.pre-commit: 2.15.0 -> 2.16.0

Motivation for this change
Update to latest upstream release 2.16.0
Change log: https://github.com/pre-commit/pre-commit/releases/tag/v2.16.0
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 148123 run on x86_64-linux 1

3 packages built:

pre-commit (python39Packages.pre-commit)
python38Packages.pre-commit
wlc

I think this breaks pre-commit, since the generated hook now looks like this:
#!/usr/bin/env bash
# File generated by pre-commit: https://pre-commit.com
# ID: 138fd403232d2ddd5efb44317e38bf03

# start templated
INSTALL_PYTHON=/nix/store/5bh6rpya1ar6l49vrhx1rg58dsa42906-python3-3.9.6/bin/python3.9
ARGS=(hook-impl --config=.pre-commit-config.yaml --hook-type=pre-commit)
# end templated

HERE="$(cd "$(dirname "$0")" && pwd)"
ARGS+=(--hook-dir "$HERE" -- "$@")

if [ -x "$INSTALL_PYTHON" ]; then
    exec "$INSTALL_PYTHON" -mpre_commit "${ARGS[@]}"
elif command -v pre-commit > /dev/null; then
    exec pre-commit "${ARGS[@]}"
else
    echo '`pre-commit` not found.  Did you forget to activate your virtualenv?' 1>&2
    exit 1
fi

INSTALL_PYTHON is pointing to a Python that doesn't know anything about a pre_commit module.

These changes in particular: https://github.com/pre-commit/pre-commit/commit/e622f793c3de2276d6975358c7e35e9b890fbdfd#diff-0b529f245bb818d1b64af9b1703b9f645b7ac669aad9e20970c8d0b9515e7e3dR13-R20 make it so that a new hook-tmpl patch is necessary since sys.executable likely won't be the same python that contains the dependencies needed to run the pre_commit module (I think).

PR'd it: https://github.com/NixOS/nixpkgs/pull/148353


Exploit the more complete DatasetDescriptor

UI changes wrt https://github.com/CCI-Tools/cate/pull/1006:
Datasets availablitiy and capabilities are now derived from obtained (ODP store) and computed (Zarr store) verification_flags (called capabilities in Cate App):

"open"  - dataset can be opened
"constrain_time" - dataset can be opened and time can be constrained
"constrain_region" - dataset can be opened and region (bbox) can be constrained
"write_zarr" - dataset can be opened and cached

@dzelge FYI!


Add developer getting started guide (#2837)

Pull request opened by the merge tool on behalf of #2837

Merged into branches iso3 in ce1beac78aadcd99e3b7c84f48876952a702d5af


esp doesn't start up to wifi configuration mode

Hi All,
I'm having continuous issues with fs, it doesn't seem to successfully mount whatever I do.
This is causing it not to work, any suggestions?
Tried:

reinstalling arduino ide
updated esp8266 in board manager
changed the version of IwIP variant to v1.4

01:33:01.421 -> failed to mount FS

For those looking for a solution to the above, it seems to be fixed by downgrading esp8266 board between 2.3 and 2.4, currently at 2.4.2, but it still jump straight to 15:12:07.025 -> *WM: Using last saved values, should be faster without trying to configure the wifi?
connecting gpio 10 to ground isn't resetting esp as well?

Commenting out this line seemed to allow the reset to happen. After the device is reset, uncomment this line.
  // if (resetConf)     wifiManager.resetSettings();

A few things to try to help others:

you can reset wifi settings through the Arduino ide: Tools > erase flash > all
set IwIP variant to v1.4 higher bandwidth


add ECDSA signing and verification on the secp256k1 curve

This PR adds support for ECDSA signing and verification on the secp256k1 elliptic curve.
Future work:

add GLV to efficiently compute the EC scalar multiplication
prove lemmas needed for Lib.Exponentiation (e.g., (points on the curve, aff_point_add) forms a commutative monoid)

@msprotz thanks for your comments!
That's right, this PR also introduces some changes to exponentiation.
There are three files: lib/Lib.Exponentiation (high-level specification), specs/Spec.Exponentiation (low-level specification) and code/bignum/Hacl.Impl.Exponentiation (low* implementation).
Until now, Hacl.Impl.Exponentiaion was functionally correct with respect to Lib.Exponentiation without any link to Spec.Exponentiation. It worked fine for EC scalar multiplication of Ed25519 and modular exponentiation.
The bitcoin-core/secp256k1 implementation has a nice optimization that avoids computing the modular multiplicative inverse (which is an expensive operation) for ecdsa-verify when the order of the curve < prime and prime < 2 * the order of the curve. To prove the correctness of such optimization we need to continue working with projective (= low-level specification) coordinates rather than with affine (= high-level specification) ones. That's why Hacl.Impl.Exponentiation is now proven to be functionally correct against Spec.Exponentiation.
I hope this explains the changes in the library.

@karthikbhargavan I don't see atm how one can easily write a generic specification that works for both curves.
The main difference is that our specification of secp256k1 uses projective coordinates (needed for the complete point addition and doubling formulas given in this paper), while p256 uses jacobian ones.
As for ecdsa-verify and ecdsa-sign, they basically load inputs into a field or scalar representation, compute EC scalar multiplication and serialize/check the result. As I've already said, there is a nice optimization for ecdsa-verify that avoids computing a modular multiplicative inverse (needed for converting a point from the projective to affine coordinates), so the EC scalar multiplication API no longer works with affine coordinates.
@Frosne, it seems that the above optimization also works for p256 as order_p256 < prime_p256 and prime_p256 < 2 * order_p256. If so, it would be great to implement it for p256-ecdsa-verify as well, as it saves two modular multiplicative inverse operations from _norm.

I will put it to my to-do list (:
On Tue, 29 Mar 2022, 15:47 Marina Polubelova, @.***>
wrote:

@karthikbhargavan https://github.com/karthikbhargavan I don't see atm
how one can easily write a generic specification that works for both curves.
The main difference is that our specification of secp256k1 uses projective
coordinates (needed for the complete point addition and doubling formulas
given in this paper https://eprint.iacr.org/2015/1060.pdf), while p256
uses jacobian ones.
As for ecdsa-verify and ecdsa-sign, they basically load inputs into a
field or scalar representation, compute EC scalar multiplication and
serialize/check the result. As I've already said, there is a nice
optimization
https://github.com/bitcoin-core/secp256k1/blob/912b7ccc4473b5c969b01d027b8d5dc515435eb5/src/ecdsa_impl.h#L244
for ecdsa-verify that avoids computing a modular multiplicative inverse
(needed for converting a point from the projective to affine coordinates),
so the EC scalar multiplication API no longer works with affine coordinates.
@Frosne https://github.com/Frosne, it seems that the above optimization
also works for p256 as order_p256 < prime_p256 and prime_p256 < 2 *
order_p256. If so, it would be great to implement it for p256-ecdsa-verify
as well, as it saves two modular multiplicative inverse operations from
_norm
https://github.com/project-everest/hacl-star/blob/0bd6f56e0a4b98f930ceaed1959d859428b63365/specs/ecdsap256/Spec.P256.fst#L84
.
—
Reply to this email directly, view it on GitHub
https://github.com/project-everest/hacl-star/pull/542#issuecomment-1081894178,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC2OMZZK5SKRTBA4I6QHUJDVCMCV7ANCNFSM5RRERX6A
.
You are receiving this because you were mentioned.Message ID:
@.***>


Add falsehood about accessibility of phone numbers

I signed it!

Hi,
Thank you for taking the time to improve libphonenumber!
Using phones for calls completely depends upon individual’s phone settings. These are generic issues which the caller should be aware of. As it is not related to / within the scope of our library, we would not be adding it to FALSEHOODS.md. Hence closing the PR.

Hey @kkeshava,
If it were re-worded to be more like "You can send a text message to any phone number", would it be a better fit?
Other examples include fax machines, mobile internet connections and IOT/Standalone devices like GPS trackers and eftpos terminals.

Hi,
Sending a text message to a phone number also depends on each individual's choice. It's the same with the other examples. As mentioned earlier, this is not related to / within the scope of our library.

Sorry, I was referring to the already existing Falsehood titled "You can send a text message to any phone number".
https://github.com/mpomery/libphonenumber/blame/fae515072fa7a65ad9fe465ba4c8a15fc00c8219/FALSEHOODS.md#L77
If the falsehood was similar:

You can make a call to any phone number
Some devices such as EFTPOS terminals, fax machines and mobile internet dongles may not support receiving calls. In addition, some people can not use their phones for phone calls. This may be permanent (such as a hearing disability), temporary (temporary hearing loss) or situational (when the user is in a noisy environment).
Tip: Make how you use phone numbers apparent to the user and give them the option to set their preference. Ensure that you look up this preference before attempting to communicate with the user.

Hi,
Sorry had not understood correctly about your previous comment :)
Your last suggestion seems perfect could you please make the changes accordingly, I shall approve it.

Branch updated 😊


Handle network down and add network selection

Handle network down and add network selection

Not added network selection yet but it handles at least one network node being up


Unable get the real hostname from request in a python script

It is very simple: Need get the current host request from host header using a python script and mitmdump as reverse proxy.
The flow.request.data.host can set the custom header, but need get the real host request from web browser using the same server ip address.
In the Apache settings i can make a virtual host with separated settings by hostname and port, but the mitmproxy can not get the real hostname.
I try get this using:
flow.request.get_state()
flow.request.data
flow.request.headers.get('host')
flow.request.host

But, the data is a empty byte, the state show the ip address but no the hostname.
How to get the raw request binary data to get the real hostname using a regular expression?
The modify of the header host is a big bad idea.

See https://docs.mitmproxy.org/stable/api/mitmproxy/http.html#Request.pretty_host.

pretty_host gets the setted hostname by mitmproxy but is not the real host header value in original request.


Is there any other way to get information apart from pokedex.pokemon(id) ?

I would like to do something like this:
console.log(pokedex.pokemon_abilities(5));
because i saw the csv file with that information? My question is: Is that valid or this library does not have this kind o function? If it does not, is there a way to extend it so i can implement thata myself ?

@enbermudez Right now this library doesn't have that functionality. I just made a merge request #3 that will update the current functionality to use an updated version of csvtojson, then once that gets merged I had planned on adding this functionality and also adding that kind of data to the pokemon objects themselves. I am going to start work on that in my forked repo

@tcov189 Great! I'll be following those changes

@enbermudez Hey just so you know I am going to abandon my fork for this repo in favor for this one https://github.com/jalyna/oakdex-pokedex


Fixing query error

Fixing query error
[06-Jun-2016 11:26:42] RECOVERABLE ERROR: "Object of class Database could not be converted to string"

I need a PHP syntax highlight editor ... :D


fix: create tag script error (InvalidID) when multiple sunbets for eks nodegroup

…s nodegroup

Fixes #N/A
Description
How was this change tested?
Does this change impact docs?

[ ] Yes, PR includes docs updates
[ ] Yes, issue opened: #
[x] No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@wttat  Can you update the PR with a description?

@wttat Can you update the PR with a description?

updated


feat: readable() in additon to save()

Is this possible to get ReadableStream instead of saving to the file or saving to the Buffer?

There isn’t any direct API to do that, but maybe we’ll be able to offload encoding to a different thread and make encode async at least. Streaming is probably not possible due to the nature of API skia exposes for encoding.

Thanks for the answer.

It's still desirable to have async encoding so I've opened a separate issue: #28


adds usings in enums in CSharp

follow up to #2494

Was this because we were generating multiple using statements if there were multiple enums that were escaped?

No it was because when cleaning up for the code symbol, we add an enum member attribute. But the using for that attribute was missing.


When building for baremetal targets, accept any arch that LLVM supports

@swift-ci please test

Would it be worth/possible to upstream OS none to LLVM?

Note that LLVM already accepts "none" as the OS part of the triple, but it gets parsed as "unknown OS" (OSType::UnknownOS) rather than an explicit "baremetal OS". And interestingly, UnknownOS already has a lot of the properties a "baremetal OS" should have. But yes, I agree that it would nice to formalize that.

@swift-ci please test

@swift-ci please test Windows platform


Remove DependencyInfoBlock for F-Droid

We find that there is a DependencyInfoBlock in your APK.
It's a Signing block added by AGP and encrypted with the Google public key so it can't be read by anyone else except Google. You can read more about it here and here.
While this was added a while ago, we were only enforcing it for new apps, and recently we started scanning updates too.
Could you please disable it with the following code?
android {
    dependenciesInfo {
        // Disables dependency metadata when building APKs.
        includeInApk = false
        // Disables dependency metadata when building Android App Bundles.
        includeInBundle = false
    }
}

Thanks!

@linsui thanks for pointing this out. Do I need to create another release for this?

It's better to have a new one, yes

The version code is bumped from 19 to 110. Do not make it 20 when you update the version to 2.0. :)

merged, close this :)

Thanks!


FilesystemFileType for simple Sf Form integration

This one adds very simple FilesystemFileType built on top of core FileType. It's main purpose it transforming UploadedFile objects from FileType into PendingFile.
The tests are very simple right now as TBH I probably lack a bit of form component expertise to determine all possible paths - help very welcome, I'd love to learn :)
For now the only purpose of this type is to plumber between FileType. My intent is to similarly provide FilesystemImageType (based on ImageType), whenever PendingImage (or similar) will be merged.

Closing as this was added in my refactor


feat: update the config item table column width

Brief changelog
Changed the table column widths to try to keep the configuration from changing rows and make it look more comfortable
Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Read the Contributing Guide before making this pull request.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Write necessary unit tests to verify the code.
[x] Run mvn clean test to make sure this pull request doesn't break anything.
[x] Update the CHANGES log.

Summary by CodeRabbit

New Features

Enhanced the display of configuration items by updating table column widths in the UI, improving visual layout and content alignment.

Could you please attach screenshots to illustrate the differences between the old and new table layouts?

中文

English


New install and getting weird error

Hello, just did a new, fresh install of faster whisper and I am getting the following error. This same dockerfile was working maybe a week ago. Is there any reason this is happening?
I load the model the following way
whisper_model = WhisperModel(whisper_model_path, device="cuda", compute_type="float16")
and the model_path is achieved by doing
git clone https://huggingface.co/guillaumekln/faster-whisper-large-v2
However, if I do the path as large-v2 it works...
The error I get is the following.
self.model = ctranslate2.models.Whisper(
RuntimeError: Unsupported model binary version. This executable supports models with binary version v6 or below, but the model has binary version v1936876918. This usually means that the model was generated by a later version of CTranslate2. (Forward compatibility is not guaranteed.)

Any suggestions?

Hi,
Can you try downloading the model again? It seems the files are corrupted or incomplete.


Trained tree unexpectedly contains only root

Description
There is a synthetic dataset with 1 feature and 1 target, number of the records = 120.
On the full dataset, I get just 1 trained tree without splits, only root.
But, on the dataset with fewer records, I get normal results.

You can set force_col_wise=true to remove the overhead.
[LightGBM] [Info] Total Bins 31
[LightGBM] [Info] Number of data points in the train set: 120, number of used features: 1
[LightGBM] [Info] Start training from score 15.000000
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[LightGBM] [Debug] Trained a tree with leaves = 1 and depth = 1

Environment info
lightgbm version 3.3.1
python 3.8.8
Windows 10 x64
Additional Comments
Necessary file:
issued_dataset.csv

Thanks very much for your interest in LightGBM and for creating this issue.
I created the following minimal, reproducible example by modifying your provided code in the following ways

adding import statements
reading dataset over the internet to avoid dealing with local filepaths
removing parameters until I had it down to the smallest possible set that still reproduces the problem
reducing num_boost_round

import lightgbm
import pandas

data_url = "https://github.com/microsoft/LightGBM/files/7597692/issued_dataset.csv"
df_data = pandas.read_csv(data_url)

X = df_data[['X']]
y = df_data['y']

for n in range(10, len(df_data) + 1):
    dataset = lightgbm.Dataset(
        data=X.iloc[:n],
        label=y.iloc[:n],
        categorical_feature=[],
        params={"max_bin": 256},
        free_raw_data=False
    )
    model = lightgbm.train(
        params={
            'boosting_type': 'gbrt',
            'objective': 'regression',
            'verbose': -1,
            'min_data_in_leaf': 1,
        },
        train_set=dataset,
        num_boost_round=50
    )
    print(f'{n=}, {model.num_trees()}')

I can see the same behavior you've described. Using small amounts of data, LightGBM builds num_boost_round trees. At exactly 120 observations (the full dataset), LightGBM only builds a single tree and shows the following warning at every iteration.

[LightGBM] [Warning] Stopped training because there are no more leaves that meet the split requirements

If you're curious, this warning comes from here:
https://github.com/microsoft/LightGBM/blob/b0137debe6e9cc92b65ec71b0fe8a56ea213c143/src/boosting/gbdt.cpp#L443
I expect that something about adding one more observation changed something about the distribution of either the target or the feature (there is only 1) in a way that led to this stopping condition. With so few observations (120), one sample can have a large impact.
I'll investigate this a bit more.

aha! I think I figured it out!
The target in this dataset only has two unique values.
np.unique(y)
# array([10., 20.])

Adding the final row in the dataset results in the mean of y being exactly 15.0. 60 observations where y = 10, 60 observations where y = 20.
In your dataset, X is monotonically increasing, and after that last observation is added, the mean of X is identical for y=10 and y=20.
pandas.DataFrame(
   {
       "y": y,
       "x": X.values.flatten(),
       "cumulative_sum": np.cumsum(y),
       "cumulative_mean": np.cumsum(y) / (np.array(range(120)) + 1)
   }
).groupby(["y"]).mean()

I think that in this situation, it's not possible for LightGBM to create a split with positive gain for the regression objective. Even after setting min_data_in_leaf=1, min_gain_to_split=0.0, and min_sum_hessian_in_leaf=0.0.
import lightgbm
import pandas

data_url = "https://github.com/microsoft/LightGBM/files/7597692/issued_dataset.csv"
df_data = pandas.read_csv(data_url)

X = df_data[['X']]
y = df_data['y']

n = 120
dataset = lightgbm.Dataset(
    data=X.iloc[:n],
    label=y.iloc[:n],
    categorical_feature=[],
)
model = lightgbm.train(
    params={
        'boosting_type': 'gbrt',
        'objective': 'regression',
        'verbose': -1,
        'min_data_in_leaf': 1,
        'min_gain_to_split': 0.0,
        'min_sum_hessian_in_leaf': 0.0,
    },
    train_set=dataset,
    num_boost_round=50
)
print(f'{n=}, {model.num_trees()}')

I think this plot illustrates the problem well.

import seaborn as sns
sns.scatterplot(x=X[["X"]].values.flatten(), y=y)

In this case, there's very little that a tree-based supervised learning approach can tell you about how to predict y from X.

@jameslamb Thank you for your response. Yes. This is my fault, I just realized this example is not correct.

@jameslamb  First of all, thank you for your reply. Yes, this is my fault, I made incorrect example.
This is almost the same dataset. I just added another column: 'x2'. It contains 1 and 2. And it splits 2 independent groups.
issued_dataset_2.csv
We can see 2 groups:
import pandas
import lightgbm

n = 120

df_data = pandas.read_csv('issued_dataset_2.csv')

X = df_data[['x1', 'x2']]
y = df_data['y']

idx = X['x2'] == 1
plot(X.loc[idx, 'x1'].values, y.loc[idx].values)

and
idx = X['x2'] == 2
plot(X.loc[idx, 'x1'].values, y.loc[idx].values)

But LightGBM returns the same "tree".
dataset = lightgbm.Dataset(
    data=X.iloc[:n],
    label=y.iloc[:n],
    categorical_feature=[],
)
model = lightgbm.train(
    params={
        'boosting_type': 'gbrt',
        'objective': 'regression',
        'verbose': -1,
        'min_data_in_leaf': 1,
        'min_gain_to_split': 0.0,
        'min_sum_hessian_in_leaf': 0.0,
    },
    train_set=dataset,
    num_boost_round=50
)
print(f'{n=}, {model.num_trees()}')

n=120, 1

At the same time, DecitionTreeRegressor coped with it
from sklearn.tree import DecisionTreeRegressor
from sklearn.tree import plot_tree
from sklearn.metrics import mean_absolute_error

estimator = DecisionTreeRegressor(max_depth=50)
estimator.fit(X, y)
print(mean_absolute_error(estimator.predict(X), y))

0

And it builds the tree:
estimator = DecisionTreeRegressor(max_depth=5)
estimator.fit(X, y)
plot_tree(estimator, fontsize=9, )

I think I get it. The problem is in splitting criterion. Because our data is absolutely symmetric with respect to the partition by x1 and x2. As I understand, here we could split only by 2 equal (in terms of gain) datasets.
With another criterion we won't have such problems or the 1st split should not maximize variance gain. But here, it seems this caused the issue.

here we could split only by 2 equal (in terms of gain) datasets

Yep, exactly! If you look at the nodes produced by the first split from your DecisionTreeRegressor example, note that they both have leaf value 15.0 (exactly equal to the mean of the target). LightGBM wouldn't make such a split because it would say "such a split won't change anything about the model's predictions".
If you want to exert tighter control over the tree-building process, you can force LightGBM to make a given split by using a concept called "forcedsplits".
The code below could be used to try to force LightGBM to reproduce results similar to what you saw with DecisionTreeRegressor.
import lightgbm
import pandas
import json
from sklearn.metrics import mean_squared_error

data_url = "https://github.com/microsoft/LightGBM/files/7616028/issued_dataset_2.csv"
df_data = pandas.read_csv(data_url)

X = df_data[['x1', 'x2']]
y = df_data['y']

forced_split = {
    "feature": 1,
    "threshold": 1.5,
    "right": {
        "feature": 0,
        "threshold": 25.0,
    },
    "left": {
        "feature": 0,
        "threshold": 25.0,
    }
}

with open("forced_split.json", "w") as f:
    f.write(json.dumps(forced_split))

n = 120
dataset = lightgbm.Dataset(
    data=X.iloc[:n],
    label=y.iloc[:n],
    categorical_feature=[],
)
model = lightgbm.train(
    params={
        'boosting_type': 'gbrt',
        'boost_from_average': False,
        'objective': 'regression',
        'verbose': 1,
        'min_data_in_leaf': 1,
        'min_gain_to_split': 0.0,
        'min_sum_hessian_in_leaf': 0.0,
        'forcedsplits_filename': "forced_split.json"
    },
    train_set=dataset,
    num_boost_round=500
)
print(f'{n=}, {model.num_trees()}')

That code produces a model with num_boost_round trees, and the predictions approach those you saw with DecisionTreeRegressor...LightGBM is able to split off some observations to either 10.0 or 20.0, and it locks in to predicting close to 15.0 (the target mean) for other observations.
mean_squared_error(
    model.predict(X[['x1', 'x2']]),
    y
)
# 7.5000000000000115

model.predict(X[['x1', 'x2']]

Please note that manually overriding LightGBM's tree-building decisions is a complex task that can be difficult to get right (see the discussion in #4591 and #4725, for examples).
Broadly speaking, for a wide range of datasets and use cases, gradient boosting should outperform other non-boosted tree-based models like DecisionTreeRegressor. However, for these sorts of cases you might find that DecisionTreeRegressor or even just application code with if statements will be a better choice.

I've tried commenting the following line
https://github.com/microsoft/LightGBM/blob/af5b40e1f653e25a5f3470e09de4d1c68dc59663/src/io/config_auto.cpp#L403
This would allow us to set min_gain_to_split to a negative value.
Setting min_gain_to_split to a negative value produces 50 trees with the above dataset.
So I think we may remove the check for min_gain_to_split. This allows LightGBM to do exploration in early stages when fitting dataset like Y = X1 (exclusive or with) X2, where X1 and X2 are features taking values 0 and 1.

From the Parameters description:

Note: the forced split logic will be ignored, if the split makes gain worse

I would also second allowing for the possibility to apply forced splits, even if they are worse than doing nothing. Sometimes you might be forced to have certain features in a model because of e.g. business constraints - even, if it makes the model performance worse.


Upgrade to Webpack's loader-utils v1.0.0

Hi! I started getting this deprecation warning yesterday after updating to the latest version of Webpack:
(node:990) DeprecationWarning: loaderUtils.parseQuery() received a non-string value which can be problematic, see https://github.com/webpack/loader-utils/issues/56
parseQuery() will be replaced with getOptions() in the next major version of loader-utils.
    at Object.parseQuery (/Users/fknussel/workspace/x/node_modules/loader-utils/index.js:78:3)
    at Object.module.exports (/Users/fknussel/workspace/x/node_modules/eslint-loader/index.js:150:17)
    at LOADER_EXECUTION (/Users/fknussel/workspace/x/node_modules/loader-runner/lib/LoaderRunner.js:119:14)
    at runSyncOrAsync (/Users/fknussel/workspace/x/node_modules/loader-runner/lib/LoaderRunner.js:120:4)
    at iterateNormalLoaders (/Users/fknussel/workspace/x/node_modules/loader-runner/lib/LoaderRunner.js:229:2)
    at Array.<anonymous> (/Users/fknussel/workspace/x/node_modules/loader-runner/lib/LoaderRunner.js:202:4)
    at Storage.finished (/Users/fknussel/workspace/x/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:38:15)
    at /Users/fknussel/workspace/x/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:69:9
    at /Users/fknussel/workspace/x/node_modules/graceful-fs/graceful-fs.js:78:16
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:416:3)

I've set the process.traceDeprecation flag to true on my Webpack config file to find out which loader was causing the deprecation warning. As noted on the warning, this is related to this other issue on webpack's loader-utils.
This PR updates loader-utils to its latest version and therefore removes the deprecation warning.

Released in 1.6.2


(Solution Needed) Find the value of nCr for given value of n & r

I'd like to work on this issue pls assign me.

@RKRohk

I am ready to work with this. Kindly assign

updated 17.h

Everyone is welcome to contribute to this. If you have a better solution, just make a pull request @anurag03-10


Bloblang for field values

It would be great to have a list of fields that support Bloblang as described in the Interpolation page.
I'm hoping to spin up multiple Kafka consumers with different client_ids but the client_id doesn't seem to support Bloblang:
input:
    kafka_balanced:
        addresses:
            - localhost:9092
        topics: [ topic.1 ]
        client_id: ${! uuid_v4() }
        consumer_group: example

output:
  stdout: {}

The output of running against the above configuration is as follows:
{"@timestamp":"2020-11-02T20:21:19Z","@service":"benthos","level":"ERROR","component":"benthos.input","message":"Failed to connect to kafka_balanced: kafka: invalid configuration (ClientID is invalid)"}

Does this field support Bloblang, or have I misunderstood something?

Hey @codingconcepts, unfortunately client_id doesn't support interpolations, they're mostly limited to fields that are useful to change based on messages passing through rather than cases like this that would be evaluated once on start up. A dirty work around might be to use an env var instead with client_id: ${CLIENT_ID}, and then when you execute the config set that var with something like CLIENT_ID=$(cat /proc/sys/kernel/random/uuid) benthos -c ./foo.yaml.
The fields that support interpolation already have notes on their docs, but it's not particularly clear and there's currently no way to quickly scan all interpolation fields. I'll mark this as a documentation enhancement and think about how this could be made more user friendly.

That'd be brilliant, thanks very much, @Jeffail!
Thanks for the suggestion of using env vars too, I'll see how this feels once implemented 👍


Custom responses for failed actions

We have a couple of cases where we need to know why an action failed.  We could come up with some work around with the server log or monitored properties, but that is not an ideal way to handle errors.
This is written such that existing callbacks will not be affected.  The err object must have a "properties" member for the behavior to be triggered.  I also added a test case to validate the Siren response.

To be clear you're proposing in device actions:
TestDriver.prototype.customError = function(cb) {
  cb({statusCode: 401, properties: {message: 'custom error message'}});
}

This would return the following http response:
< HTTP/1.1 401
{
  class: ['action-error'],
  properties: err.properties,
  links: [
    { rel: ['self'], href: env.helpers.url.current() }
  ]
}

Today if anything is passed back in the first parameter to the callback it returns a 500 with no body.
My only issue with the PR is syntax for returning the error. We were thinking about adding a custom error object possibly on the zetta driver that takes a status code and message.
Maybe something like:
TestDriver.prototype.customError = function(cb) {
  cb(new Driver.ActionError(401, 'custom error message'));

 // Also using js Error would also work, could set the message property and assume 500 status code.
 cb(new Error('custom error message'));

}

I'm fine with that approach, I'll make those changes.
There isn't going to be any problem with actions that previously returned a 500 error with no body having a body after this change?

We don't believe so. We'll do some testing with our clients but all so far just look for a non 500 status code.

Added the error.  Need to update the callback handler to use it still.

+1 Thanks!


[Q] How to find the version of an ancient installed graphite server

I have inherited a really old graphite server that was installed about 8-10 years ago. I'm trying to work out which version of graphite this is.  We can access the main page, but we have no accounts or logins to the dashboard.
What would be the best way to discover this by looking at the installed folders, please?
$ curl -sv http://localhost/metrics/find?query=carbon.*
* Hostname was NOT found in DNS cache
*   Trying <IP_ADDRESS>...
* Connected to localhost (<IP_ADDRESS>) port 80 (#0)
> GET /metrics/find?query=carbon.* HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Date: Wed, 06 Mar 2024 11:25:52 GMT
* Server Apache/2.4.7 (Ubuntu) is not blacklisted
< Server: Apache/2.4.7 (Ubuntu)
< Content-Length: 284
< Content-Type: text/html; charset=iso-8859-1
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

<title>404 Not Found</title>

Not Found

The requested URL /metrics/find was not found on this server.

<address>Apache/2.4.7 (Ubuntu) Server at localhost Port 80</address>

* Connection #0 to host localhost left intact

You can also try to call  curl -sv http://localhost/version
Or look for WEBAPP_VERSION in webapp/graphite/settings.py
If it's < 1.2.0 - it's exact version. If 1.2.0-dev - it's installed from master and then it's hard to say.

curl -sv http://localhost/version

This gives 404

Or look for WEBAPP_VERSION in webapp/graphite/settings.py
This is more promising:
 /opt/graphite/webapp/graphite/settings.py:29:WEBAPP_VERSION = '0.9.12'

So it looks like this is 0.9.12. Would you agree?

Yes, I would

Thank you


NETOBSERV-105 Netflow table sort test

Added tests for netflow-table sorting and found a small bug in netflow-table-raw keys.

/approve


Is there a way to get the precise time that each period is played?

Hello, I have been using this module for an auditory neuroscience experiment in my research lab, and I'm trying to increase the temporal precision of my results. Basically, I'm running jackclient-python on a raspberry pi with a Hifiberry audio card (all part of the Autopilot project). We play sounds separated by silence, and record auditory responses from hearing regions in the brain to those sounds. This is similar to hearing tests at the audiologist.
To make this experiment work, I would like to know with sub-millisecond precision the exact time that each sound comes out of the speaker. Of course I can us an oscilloscope to measure this, but this is bulky and it seems like there should be some way to get the information I need directly from jack.
I am starting jackd like this:
jackd -P75 -p16 -t2000 -dalsa -dhw:sndrpihifiberry -P -r192000 -n3 -s &

I think this means 3 periods of playback latency. While this call seems to set the length of the period to 16 frames, I think I am actually using periods of 1024 frames (because the parameter blocksize of my jack.Client is 1024), this must be set by the sound card.
Here is some pseudocode for what is going on in my process callback right now:
def process(frames):
    last_frame_time = client.last_frame_time
    frames_since_cycle_start = self.client.frames_since_cycle_start
    now = datetime.datetime.now()

    # write sound to the output ports (not shown...)

I log these three timing variables (last_frame_time, frames_since_cycle_start, and now) for every process call. Using these data, I think I can calculate offline the approximate relationship between frame times and the system clock. That way, I can calculate what time it was on the system clock at the beginning of each period (i.e., when the process callback was called). Finally, I think I can assume that sound comes out of the speaker 3 periods later.
I am looking for guidance, am I thinking about this correctly? If so, then my precision will be limited by the accuracy of frames_since_cycle_start, which I know is only approximate, and the latency between getting that estimate and getting the system clock time. Is there a better way to get the precise time that the sound on each period comes out of the speaker? Maybe there is a way to directly sample the audio clock on the raspberry pi, if I can figure out which pin it is on. Thanks for any tips!
edit: more info, this is what I see when I start jackd, which includes version information and parameter settings
jackdmp 1.9.20
Copyright 2001-2005 Paul Davis and others.
Copyright 2004-2016 Grame.
Copyright 2016-2021 Filipe Coelho.
jackdmp comes with ABSOLUTELY NO WARRANTY
This is free software, and you are welcome to redistribute it
under certain conditions; see the file COPYING for details
JACK server starting in realtime mode with priority 75
self-connect-mode is "Don't restrict self connect requests"
creating alsa driver ... hw:sndrpihifiberry|-|1024|3|192000|0|0|nomon|swmeter|soft-mode|32bit
configuring for 192000Hz, period = 1024 frames (5.3 ms), buffer = 3 periods
ALSA: final selected sample format for playback: 32bit integer little-endian
ALSA: use 3 periods for playback

Getting reliable information about time is often quite hard.
If you have access to an oscilloscope, you should definitely use it to check if your assumptions about the API and your code are correct.
AFAIU, client.last_frame_time is the way to get an exact time in the audio callback. I have the feeling that you should not use datetime.now() in there. I that case you don't need client.frames_since_cycle_start either, which seems sketchy anyway.
In the non-audio thread, you should use client.frame_time and compare that to client.last_frame_time from the audio thread.
I'm not sure what exactly you need, can you describe that in more detail?

While this call seems to set the length of the period to 16 frames, I think I am actually using periods of 1024 frames (because the parameter blocksize of my jack.Client is 1024)

This sounds wrong.
The "period" should be equal to the "blocksize". It's the same thing.
It seems like your -p16 sets --port-max, which is probably not what you intended?
I guess you wanted the ALSA backend option -p, a.k.a. --period?
Using the long options surely would reduce potential confusion, but either way you have to use the ALSA options after -dalsa!
This may not be immediately obvious, I certainly learned it the hard way.
I guess you need something like this (I removed a few options where I didn't know what they are supposed to do):
jackd --driver alsa --period 16 --device hw:sndrpihifiberry --rate 192000 --nperiods 3

BTW, did you try to use --nperiods 2? If you hardware supports it, this should give you lower latency.
If not, you'll probably get xruns.
And using -s seems kinda unreliable (I never used it, though). Can't you use --realtime on the Pi?

Thanks @mgeier for this information!
Re the command line options, I received these values from a colleague, and I now understand that I was misinterpreting them, specifically I didn't understand that '-p' has a different meaning before and after '-dalsa'. In any case, I am using a blocksize/period of 1024 frames, and --nperiods of 3. I think these values are fine for me. I don't actually need particularly low latency, what I need is precise information about when sounds were played (see below).

I'm not sure what exactly you need, can you describe that in more detail?

Sure!  My goal is to synchronize the two streams within about 0.5 ms. Specifically, for each sound that was played, I need to know which sample in the brain data was taken at the time that the sound started playing.
For playing sounds: I do this with a raspberry pi running jack with a hifiberry amp2 audio output.
For recording EEG: I do this with a chip called an ADS1299. I use a Teensy to get the data off the ADS1299 and send it to a desktop PC. The sampling rate is 16 kHz.
So somehow, I need to know which sample in the EEG corresponds to the time that jack started playing sound. Approach 1 (software): store the time on the Raspberry Pi system clock at which jack played each sound. To do this I need to convert between last_frame_time and clock time. Approach 2 (hardware): If I could find a way to pulse a pin on the raspberry pi on every period/blocksize, then I could sample this signal on one of the digital inputs on my Teensy.
Thus I am wondering what would be the best way to get the clock time that each period is played, or alternatively to set a callback with jack that I could use to pulse a pin every time a period is played.

Getting reliable information about time is often quite hard. If you have access to an oscilloscope, you should definitely use it to check if your assumptions about the API and your code are correct.

Agreed! Thanks for any suggestions about what to try first, and I will use the oscilloscope to verify it.

To do this I need to convert between last_frame_time and clock time.

I'm not sure, but it might be hard to get to a reliable 0.5 ms accuracy with this approach.
I would expect quite a lot of jitter with this approach.

Approach 2 (hardware): If I could find a way to pulse a pin on the raspberry pi on every period/blocksize, then I could sample this signal on one of the digital inputs on my Teensy.

Do you have a spare audio channel?
You could try to generate a second audio signal, wire it to one of the inputs of the Teensy and try to detect it there?
I don't know what voltages and types of signal would work for that.


Uploading Correlation ID

Description
Due uploading though the CLI I received this issue (second time). Appcenter cli is a part of our CI pipe line and uploaded build definitely is unique. Probably the issue coming from backend side but unfortuynalelly I can't find a right place to create and issue.
Repro Steps
I can't provide specific apks, but it start happens 4-5 days ago. And also it happen time to time.
Expected behavior
Apk will be uploaded property to appcenter
Details

Is there a particular beacon/command that you are having trouble with?

e.g. appcenter tokens list

Which version of the App Center CLI are you using?
2.7.3
Which OS version did you experience the issue on?
Linux
What version of Node.js and NPM/Yarn are you using?

node:15.3.0-alpine3.10 docker image

17:41:30
  Response: Error: Loading release id failed: A problem occured while extracting your app. (Correlation ID: <UIDD>)

Hi @Vacxe, thanks for reaching out. Please, fill in the issue according to the bug report template.

@Vacxe thank you.
Could you check whether CLI fails to upload this specific apk every time or it happens occasionally?
Is it possible for you to check whether you can upload this apk to the portal through a browser?

@russelarms yea. It possible to upload the same apk (exactly the same artifact) via web version. If u require any additional info for investigation please let me know.

Hey @Vacxe, would you mind sharing correlation id with us? That would help us understand what is going wrong during the upload process.

@Jamminroot f79fd1ba-f306-4fa3-859b-014bdec901ea

Is there any other cases - for example, app transfer, or other critical changes?
Also, is it possible that there's a typo in correlationId - since I can't really find a transaction on the backend related to the one you provided.

Is there a chance for you to gather debug logs? For that you'd need to add --debug argument to the command.
Also, having additional debug logs from npx should help, but that would require different invocation, example is below:
DEBUG="appcenter-cli:commands:*" npx ts-node src/index.ts distribute releases list
And can you share the command you are using to upload?

Description
We got similar problem with uploading iOS builds in our CI (we are using appcenter-cli).
Sometimes uploading is ok, sometimes - we have error
Loading release id failed: A problem occured while extracting your app.
We then can upload built ipa manually without problem. Often another build from same commit uploads normally.
Correlation ids: 18711d30-da7d-4d0d-b349-532fe457d1c1, 15723071-c248-42c5-bb04-9311dc7a5f6b, bdd785a1-ce23-4682-ac77-af2bb9abf04b
Repro Steps
Cannot reproduce, occurs often, randomly.
Expected behavior
We expect that build uploads without errors

Is there a particular beacon/command that you are having trouble with?
appcenter distribute release --app APP_NAME --file path/to/appcenter.ipa --token TOKEN --group Collaborators --disable-telemetry --debug --release-notes 'RELEASE_NOTES'
Which version of the App Center CLI are you using?
2.7.3
Which OS version did you experience the issue on?
mac OS Catalina 10.15.6
What command line/shell are you using?
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin19)
What version of Node.js and NPM/Yarn are you using?
node v14.1.0, npm 6.14.4

(May be related to https://github.com/microsoft/fastlane-plugin-appcenter/issues/243 ?)

Hi all,
We implemented a fix for preventing this issue again.
@Vacxe , @PetrBodnar , could you please let us know if you still have this issue or not?

Thanks @alexandergoncharov . I let you know in case of reproduce. Cheers.

Hi all,
@Vacxe , @PetrBodnar, just a friendly ping if this issue is still reproducible? :)

Looks like it is ok now (issue is not reproduces any more), thank you :)

Awesome, Thanks for letting us know!
So, I'm going to close this issue now but please feel free to ask any questions or let us know if you faced this issue again.

I'm facing the same issue. Was the fix in the cli or the backend? @alexandergoncharov-zz

Hello @philj0st , do you get exactly the same error as in the description? It would be helpful if you could provide the command that produces the error and the corresponding logs.

For me, I fixed this issue when I changed the Release type to Beta from Entreprise in my App Settings in Appcenter.
I hope this helps you as well.


python setup.py egg_info did not run successfully.

Hello. I tried to install your project via pip install sqlite-icu. And I get an error:
... line 20, in
icu = Extension("icu", sources=["icu.c"], **pkgconfig('icu-uc icu-i18n'))
TypeError: keywords must be strings
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

What is the output of the shell command pkg-config --cflags --libs icu-uc icu-i18n on your machine? With that output, I should be able to find out what is going wrong.

Output:
Package icu-uc was not found in the pkg-config search path.
Perhaps you should add the directory containing `icu-uc.pc'
to the PKG_CONFIG_PATH environment variable
No package 'icu-uc' found
Package icu-i18n was not found in the pkg-config search path.
Perhaps you should add the directory containing `icu-i18n.pc'
to the PKG_CONFIG_PATH environment variable
No package 'icu-i18n' found

However, I could have made something wrong since I have never used pkg-config before.

To build the package, you need the ICU development library installed on you machine (libicu-dev on debian).
Things I could improve on my side:

[ ] Improve error message
[ ] Improve documentation
[ ] Provide binary packages
[ ] Check if there is any need for this package now that sqlean's unicode extension is available via the sqlite-sqlean python package.

You could also check out the link in the last point. That even provides binary packages, so you don't have to install any development packages.


reduce/move provider dependencies

For providers to be reused effectively, their dependencies on the larger tegola project should be
reduced.
For example, the import section of the gpkg provider includes:

"github.com/terranodo/tegola"
"github.com/terranodo/tegola/basic"
"github.com/terranodo/tegola/geom"
"github.com/terranodo/tegola/geom/encoding/wkb"
"github.com/terranodo/tegola/internal/log"
"github.com/terranodo/tegola/provider"

Where github.com/terranodo/tegola is imported only for tegola.WebMercator.  The geom package is a logical place for this & would eliminate this top-level dependency.
Perform similar examination of other provider dependencies .

I agree. This is part of the transition to the geom package and purging all the top level files. After the transition we should be able to remove:

github.com/terranodo/tegola
github.com/terranodo/tegola/basic


feat: added personalized link

Better late than never:

The new button is not always located correctly
Pressing that button when there's no data in the fields sets everything to undefined:
https://avishayil.github.io/corona-health-statement/?childName=undefined&childId=undefined&parentName=undefined&parentId=undefined

Yep, I suppose a fix is neccesary.
Regarding the new button location - what do you mean?


ranker.score(idx, query) runs indefinitely

I am running the search function from my Flask app. The step ranker.score(idx, query) runs indefinitely. Could you please check and help me with any pointers? Thank you.
Dataset used: data.dat attached.

Code:

Issue: The application returns results for a single search currently.
During the second search, the below steps are completed:

Index creation
Ranker selection
User input parsing
The step ranker.score() runs indefinitely during the second search.
Workaround: The Flask application requires a restart for checking different queries.

same here, please fix this

I am seeing this on Python 3.7 as well.  According to other students in my class, this functionality works correctly on Python 3.5 (which I will try soon).  (Unfortunately, since this is a homework assignment, I cannot provide any more details.)

@mikepigott Did you end up finding a solution?

As an update: running my Flask app with the "threaded" flag set to False solved the issue.
I.e., changing:
app.run()
to
app.run(threaded=False)
solved my issues.


Enhancement: Allow users to add a project from the page

Enhancement Description
What feature?
In this section the user will be able to add a project using a username and repo name input, here we will use:

GitHub API to get repo details.
A button to fetch this details to the projects.json.

Impact Area

[ ] User Interface
[X] Functionality
[ ] Performance
[X] Accessibility
[ ] Documentation
[ ] Other (please specify)

Additional Information
It could be easier for the users to add the projects from the page.
Record

[X] I agree to follow this project's Code of Conduct
[X] I want to work on this issue

Hello @rafabelts! Congratulations on creating your first issue! 🎉 We appreciate your contribution and want to assure you that we're on it. We'll do our best to resolve the issue promptly. If you have any additional information or details to share about the issue, please let us know. We're here to help! Thanks again for taking the time to report this issue. We value your feedback and look forward to working together to address it. Best regards, Project TeamWe will promptly review your changes and offer feedback. Keep up the excellent work! Kindly remember to check our contributing guidelines


reworking of CDM mapping

This is the PR to Issue #36

@aanderss: Can you please review the new CDM mapping.
ICOADS:
I set location_quality to LZ mapping.
Notes:

This code table is used for the value of report_quality. I did not taken this into account for location_quality as we want to know whether LZ from the original data is equal 1, as far as I understood. Do you agree?

location_quality's default value was 3. With the new LZ mapping, location_quality is equal *NULL in the most cases of testing suite.

C-RAID:
I set the report_quality's default value from NULL to 2.

Regarding ICOADS:
need to check, whether LZ flag is only set, when landlocked and otherwise empty.
mapping would then be
LZ = empty/*NULL => position ok => location quality = 0
LZ = 1  => bad / land position => location quality = 2

QC: if location_quality == 2: report_quality = 1
see report_quality code table


Support for dark theme

It hurts my eyes when the background is all white.

Well, it was because of https://github.com/anthonygelibert/QLColorCode, not QLStephen.qlgenerator


Support render unless key not found

render_if_key_found:
"This option or method supports rendering only if the specified key is found in the hash. If the key is not found, rendering will be skipped."
@wojtowicz
Currently the representer will return defined property with nil value even if there isn't key passed from represented.
For example
class SimpleRepresenter
   property :name
   property :title
end

SimpleRepresenter.new({ title: "Paladin" })

output:
{ name: "", title: "Paladin" }

so I expect to have an option to remove defined properly when the represented doesn't have that key
expected output:
{ title: "Paladin" }

cc @kukicola

I don't get it. Representer won't render a field if the value is nil unless you set render_nil: true.
irb(main):001> require 'simple_representer'
=> true
irb(main):002* class Representer < SimpleRepresenter::Representer
irb(main):003*   property :name
irb(main):004*   property :title
irb(main):005> end
=> [#<SimpleRepresenter::Property:0x00000001031eb280 @field=:name, @options={}>, #<SimpleRepresenter::Property:0x00000001031eb1b8 @field=:title, @options={}>]
irb(main):006> Representer.new({ title: "Paladin" }).to_json
=> "{\"title\":\"Paladin\"}"

irb(main):001> require 'simple_representer'
=> true
irb(main):002* class Representer < SimpleRepresenter::Representer
irb(main):003*   defaults render_nil: true
irb(main):004*   property :name
irb(main):005*   property :title
irb(main):006> end
=> [#<SimpleRepresenter::Property:0x00000001014b4720 @field=:name, @options={:render_nil=>true}>, #<SimpleRepresenter::Property:0x00000001014b4658 @field=:title, @options={:render_nil=>true}>]
irb(main):007> Representer.new({ title: "Paladin" }).to_json
=> "{\"name\":null,\"title\":\"Paladin\"}"

So in your case, I'm assuming you have render_nil: true as default, can't you just turn that off for specific fields? Or am I missing something here?

@comictvn This is a weird use-case, I wouldn't implement it directly in the gem. You can always use conditional fields:
class Representer < SimpleRepresenter::Representer
    defaults render_nil: true
    property :name, if: -> { represented.respond_to?(:name)  }
    property :title
 end

Representer.new({ title: "Paladin", name: nil }).to_json
=> "{\"name\":null,\"title\":\"Paladin\"}"
Representer.new({ title: "Paladin" }).to_json
=> "{\"title\":\"Paladin\"}"

@kukicola the conditional will works but not perfectly since we have a lot of fields in representer.

@comictvn
You can define an additional class method to handle that for you, or even overwrite property method
class BaseRepresenter < SimpleRepresenter::Representer
  def self.conditional_property(name)
    property  name, if: -> { represented.respond_to?(name)  }
  end
end

class Representer < BaseRepresenter
  defaults render_nil: true
  conditional_property :name
  conditional_property :title
end

it's might good if have a global condition. but might we don't have as of now.


Add timeout to a couple of validate runner tests

Mitigate #26301
Now it should fails the stuck test instead of timing out after 4h and does not report test result.

Mitigate test suite timeout for beam_PreCommit_Java_PVR_Flink_Batch

Please add a meaningful description for your change here

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

I experienced this twice when testing #26275. It happens quite frequently for commit run. e.g. also see in #26082 (https://ci-beam.apache.org/job/beam_PreCommit_Java_PVR_Flink_Batch_Commit/2621/), #26091 (https://ci-beam.apache.org/job/beam_PreCommit_Java_PVR_Flink_Batch_Commit/2618/)

Run Java PreCommit

R: @damccorm
similar to #25664

Java PreCommit failure due to #21333 not relevant. Merging for now.


Histogram

It appeared that LineChart and BarChart components were completely identical.
This new Histogram component allows to merge by choosing the plot.

@jdesboeufs


READY: (willbe) Output structure for endpoints result

Report for .list and .workspace.list will be reworked when .list gets the type property for printing as a tree

Good! Thank you.


Fix AWS metrics display

Closes #882

Coverage increased (+0.09%) to 59.915% when pulling dc2feab0eb3c3507f199fdbe32a61ceb9373d4a9 on stgleb:aws_metrics into 4130c3e1a94ef5fde13d6b20383a035a4d8f5464 on supergiant:master.


"Failure Recovery Procedure" does not work (at least with SQL 2016) in case you have no more tempdb device(s)

"Failure Recovery Procedure" does not work (at least with SQL 2016) in case you have no more tempdb device(s)
It was possible to drop "extra" files (except the 1st one) but had no chances to move (and resize!) the files to the C: drive

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0d9423e4-0573-7564-3f58-2a8df9a11d9f
Version Independent ID: 5b88aa32-b6f7-e8eb-806f-fd331ed43133
Content: Move System Databases - SQL Server
Content Source: docs/relational-databases/databases/move-system-databases.md
Product: sql
Technology: ****
GitHub Login: @stevestein
Microsoft Alias: sstein

@crimean-celica -- Sergey, thank you for your feedback.  Please consider these resources:

SQL Server on Microsoft Q&A
DBA Stack Exchange
Stack Overflow

@stevestein -- Steve, please look into this potential documentation issue.

@crimean-celica - thanks for the feedback.
It's difficult to provide details for the many environments and situations that arise when recovery is needed. The links William provided, for example, Q&A, forums, etc.. or support is likely the best option.
Thanks
#please-close


WIP: Remove per-entity tables

Summary
Fixes #(related issue)
Change Type
Mark the type of change your PR introduces:

[ ] Bug fix (resolves an issue without affecting existing features)
[x] Feature (adds new functionality without breaking changes)
[ ] Breaking change (may impact existing functionalities or require documentation updates)
[ ] Documentation (updates or additions to documentation)
[x] Refactoring or test improvements (no bug fixes or new functionality)

Testing
Outline how the changes were tested, including steps to reproduce and any relevant configurations.
Attach screenshots if helpful.
Review Checklist:

[x] Reviewed my own code for quality and clarity.
[ ] Added comments to complex or tricky code sections.
[ ] Updated any affected documentation.
[ ] Included tests that validate the fix or feature.
[ ] Checked that related changes are merged.

About the failures in artifact.go - I wonder if the path of least resistance (even with a bit of manual coding that might be thrown away later) would be to substitute the database lookups like GetArtifact with wrappers around GetEntity that would for now fulfill the same contract, in case of GetArtifact still return a pb.Artifact.


Add Javadoc Escapes to README Overview

This PR adds HTML escapes that are being used in the codesnippet injection step to the README overview injection step. This is to prevent scenarios when illegal Javadoc characters are included in the README.md (which is a valid case).

Thanks for looking at this @alzimmermsft. In the case of the unknown tags, where only some of them are unknown, this would have potentially masked the issue. At this moment, I don't believe this change is entirely necessary and before checking it in I want the other issue understood. Only then, if we feel this is still necessary, should we check it in.

/azp run azure - core - ci

/azp run java - core - ci


php artisan make:fileament-resource generate wrong code if a column is an enum

Package
filament/filament
Package Version
v3.2.76
Laravel Version
v11.7.0
Livewire Version
No response
PHP Version
v8.3.6
Problem description
If you have a  column on your model that is of type enum then the generated code by php artisan make:filament-resource UserResource -G is not correct.
this is generated
Tables\Columns\TextColumn::make('user_level')
                    ->numeric()
                    ->sortable(),

Expected behavior
That the code generated is valid and working.
Steps to reproduce
Use the repo, login select the users
Reproduction repository
https://github.com/rabol/filament-bug
Relevant log output
No response

I cloned your repo but this is the migration:
$table->integer('user_level')->default(0);

it's not an enum

On the model it’s casted to a enum, how would you otherwise do it?

You could use the enum function of a migration.
IMHO seems more an enhancement, instead of a bug.
Filament gets database schema to create the list and the form. It does not check the laravel casts.

a boolean work even if it stored in the db as an int

This isn't a bug if the column is an integer in the database indeed.


Feature/jcohen dbt20 compatibility

Mirror PR of PR #46 with the only difference being the base branch.

All updates are good and passing on our end. We can move forward with cutting the new release of the hubspot_source for dbt v0.20.0 compatibility.


conda-forge distribution

Would be nice if this there was a conda distribution on conda-forge.

Thanks, I'll look into adding a Conda distribution to the deploy process.

Heads up, I am tackling this!

Heads up, I am tackling this!

PR here https://github.com/conda-forge/staged-recipes/pull/13796
@KieranWynn do you want to be a maintainer?

PR here https://github.com/conda-forge/staged-recipes/pull/13796
@KieranWynn do you want to be a maintainer?