unsetting $_FILES superglobal

I have made a note of this issue on the closed ticekt as well: https://github.com/aelvan/Preparse-Field-Craft/issues/23 but figured I better open a new issue as well.
In Craft 3 a fix for the front-end asset uploads (more specifically the resetUloads() method in the src/PreparseField.php file) is causing the symfony/http-foundation/Request::createFromGlobals() method to fail as it cannot find $_FILES superglobal. In case of my website that causes serious problems when trying to make a payment.
Would changing unset($_FILES) to $_FILES = [] still provide a fix for this issue? Second option doesn't break the createFromGlobals() or any other functionality that relies on the $_FILES superglobal being there.

Hi there,
Any chance you had time to review this issue?
Thanks,
Iwona

Hi,
Not sure if that change would work or not. Setting up a test for it is a bit of work, so I haven't had time to look at it. I didn't experience the original issue, or wrote the fix, so if anyone wants to set up a test and confirm whether or not it works, that would be much appreciated.

I'm experiencing the same issue.
I tried to use this on a Commerce Order publish layout, because I wanted to include a specific customer primary address field in the Commerce export. My code for the preparse field I was using was:
element.customer.primaryBillingAddress.businessTaxId|default('')
But that gave me an Undefined variable: _FILES error when I tried to submit the order during the checkout. Am I doing something wrong?

We've just run into this whist trying to setup our Commerce PayPal Express integration (painful enough on its own). Removing the unset($_FILES) is the only thing that allows us to get PayPal working properly. Whats the plan? Thanks

I've changed this from unset($_FILES) to $_FILES=[] in v1.0.7.


Adding pandas to tests_require

CI on https://github.com/fastats/fastats/pull/60 and https://github.com/fastats/fastats/pull/61 is failing due to a missing pandas dep for some of the tests.
Let me know if I'm missing something obvious, but it looks like the Appveyor config is pulling from here.

Looking at the failures here it looks like the x64 builds aren't installing all of the 2nd level dependencies properly, haven't toyed with Appveyor much but happy to look into it, explicitly storing all deps is obviously not right.
@pawroman, any clues?

Yea, this change looks good, appveyor is a bit annoying. Given that travis-ci passes, i'm happy to merge unless someone wants to pick up the appveyor fix in the next couple of days?
@fastats/core ??

Closing in favour of #60


[Android] Items not added/removed when ItemsSource is a group collection

First, this is an excellent plugin! I can recommend it to everyone.
One issue however could potentially prevent someone of using it and that is missing support in the Android renderer for adding and removing items dynamically when the itemssource collection is a group collection, i.e. one having the signature:  ObservableCollection<MyGroup>, where MyGroup is a ObservableCollection<MyItem>.
For UWP everything is OK, for iOS I haven't checked yet.
I modified the code in the GridViewAdapter of the Android renderer to support this. This is what I changed and added:
    public class GridViewAdapter : RecyclerView.Adapter, Android.Views.View.IOnLongClickListener
    {
        ...

        /// <summary>
        /// Contains the flatened collection of items.
        /// </summary>
        IEnumerable<object> _flattenedItems;

        public IEnumerable Items
        {
            get
            {
                return _items;
            }
            set
            {
                var oldColleciton = _items as INotifyCollectionChanged;
                if (oldColleciton != null)
                {
                    oldColleciton.CollectionChanged -= NewColleciton_CollectionChanged;
                    // Check if items contains groups with observable (child) items.
                    // If so then deregister the 'change' listener for these child items
                    foreach (var item in _items)
                    {
                        var groupCollection = item as INotifyCollectionChanged;
                        if (groupCollection != null)
                        {
                            groupCollection.CollectionChanged -= GroupCollection_CollectionChanged;
                        }
                    }
                }
                _items = value;
                var newColleciton = _items as INotifyCollectionChanged;
                if (newColleciton != null)
                {
                    newColleciton.CollectionChanged += NewColleciton_CollectionChanged;
                }
                NotifyDataSetChanged();
                PrepareGrouping();
                if (_items != null)
                {
                    // Check if items contains groups with observable (child) items.
                    // If so then register a 'change' listener for these child items
                    foreach (var item in _items)
                    {
                        var groupCollection = item as INotifyCollectionChanged;
                        if (groupCollection != null)
                        {
                            groupCollection.CollectionChanged += GroupCollection_CollectionChanged;
                        }
                    }
                }
            }
        }

        void NewColleciton_CollectionChanged(object sender, NotifyCollectionChangedEventArgs e)
        {
            if (e.Action == NotifyCollectionChangedAction.Add)
            {
                if (e.NewItems != null)
                {
                    //TODO handle changes here, reload data etc.. if required..
                    //or brute force.. or something else
                    foreach (var item in e.NewItems)
                    {
                        var index = IndexOf(_items, item);
                        NotifyItemInserted(index);
                        // If item is a group with observable items then register a 'change' listener for these items
                        var groupCollection = item as INotifyCollectionChanged;
                        if (groupCollection != null)
                        {
                            groupCollection.CollectionChanged += GroupCollection_CollectionChanged;
                        }
                    }
                }
            }
            else if (e.Action == NotifyCollectionChangedAction.Remove)
            {
                if (e.OldItems != null)
                {
                    //If there are multiple items.
                    if (e.OldItems.Count > 1)
                    {
                        //Not very effiencent but there is no way to establish index
                        //of unsequential removed items without maintaining an extra
                        //virtual dataset which seems like more trouble then its worth.
                        NotifyDataSetChanged();
                    }
                    else
                    {
                        //Remove the single item.
                        NotifyItemRemoved(e.OldStartingIndex);
                    }
                    foreach (var item in e.OldItems)
                    {
                        // If item is a group with observable items then deregister the 'change' listener for these items
                        var groupCollection = item as INotifyCollectionChanged;
                        if (groupCollection != null)
                        {
                            groupCollection.CollectionChanged -= GroupCollection_CollectionChanged;
                        }
                    }
                }
            }
            else
            {
                NotifyDataSetChanged();

            }
            PrepareGrouping();
        }

        private void GroupCollection_CollectionChanged(object sender, NotifyCollectionChangedEventArgs e)
        {
            PrepareGrouping();
            NotifyDataSetChanged();
        }

        ...

    }

I basically added a 'collection changed' event handler for the observable collection of each group. The handler invokes PrepareGrouping() and the RecyclerView's NotifyDataSetChanged(). One could try to have an indexes list for the child items to make 'adding' and 'removing' more efficient for RecyclerView. For now NotifyDataSetChanged for me does the job.
Hope someone can use it.

Do you want to do a pull request?
Note that With the stable release of Xamarin forms own collection-view I will be winding down this development. I have no plans to remove the repo but I won't be making new features.
https://devblogs.microsoft.com/xamarin/xamarin-forms-4-3-introducing-collectionview/

I can do a pull request since also the iOS renderer (as I found out the last couple of days) exhibits the same missing support. But, as you mentioned Xamarin will eventually come out with an own implementation and therefore I am not sure whether it all is worth the effort. I just don't know when the new forms with grid view support will come out.


Weird behavior with sftp

After successfully running a job with slurm on fs0.das5 I wanted to copy back the stdout.txt produced by the job. This command:
xenon sftp --location fs0.das5.cs.vu.nl download /home/jason/stdout.txt /home/jason/stdout.txt

does not produce the expected stdout.txt file locally. It's just not there, even though no error is given.
Strangely enough, if I replace the last /home/jason/stdout.txt with something else:
xenon sftp --location fs0.das5.cs.vu.nl download /home/jason/stdout.txt /home/jason/wtf.txt

it works fine! The wtf.txt file contains a copy of stdout.txt on fs0.das5 as expected ....

Debugged and found a premature return in Xenon if source and target paths are the same nothing happens, see https://github.com/NLeSC/Xenon/blob/master/src/main/java/nl/esciencecenter/xenon/engine/util/CopyEngine.java#L285-L287


top1-5 accuracy did not achieve the expected effect（Mosi/Finetuned on UCF101/HMDB51 dataset）

Thanks for your great work!
It is worth affirming that your work has great research value. However, when I loaded the fine-tuned checkpoint in the MOSI project (with R (2+1) D-10 as the backbone), I ran r2p1d_test.yaml, the accuracy of top1-5 is approximately 72.77% and 91.49%, respectively. I don't know where I made the mistake. I haven't made any changes to the configuration file. Do I need to add some additional configuration parameter?  Or can you provide the configuration in val.log
Looking forward to your reply, thank you very much!

Hi, thanks for the interest. I will look into this problem ASAP.

嗨，感谢您的关注。我会尽快调查这个问题.
Thank you for your attention. Later, I tried to use your fine-tuned checkpoint to load val files on hmdb51 using the R (2+1) D backbone for verification (there are three val/train files in hmdb51/ucf101). I found that on the first val file, the top 1/5 results were all smaller than your results, while on the last two val files, the top 1/5 results were both larger than your results. I was wondering, is your result the average of these three val results? Looking forward to your reply! Thank you.

嗨，感谢您的关注。我会尽快调查这个问题.
Thank you for your attention. Later, I tried to use your fine-tuned checkpoint to load val files on hmdb51 using the R (2+1) D backbone for verification (there are three val/train files in hmdb51/ucf101). I found that on the first val file, the top 1/5 results were all smaller than your results, while on the last two val files, the top 1/5 results were both larger than your results. I was wondering, is your result the average of these three val results? Looking forward to your reply! Thank you.

The result was generated on the first split for both UCF and HMDB. We did not average the performance on all three splits.

嗨，感谢您的关注。我会尽快调查这个问题.
Thank you for your attention. Later, I tried to use your fine-tuned checkpoint to load val files on hmdb51 using the R (2+1) D backbone for verification (there are three val/train files in hmdb51/ucf101). I found that on the first val file, the top 1/5 results were all smaller than your results, while on the last two val files, the top 1/5 results were both larger than your results. I was wondering, is your result the average of these three val results? Looking forward to your reply! Thank you.

The result was generated on the first split for both UCF and HMDB. We did not average the performance on all three splits.

Oh, okay, that's even more confusing for me. Here's val_ 10clipsx1crops.log output (including the results of three splits):
[09/01 10:46:56][INFO] tadaconv.utils.checkpoint:  511: Load from given checkpoint file.
Checkpoint file path: /home/lzh/2022/tjq/TAdaConv/checkpoint/r2p1d_pt_hmdb_ft_hmdb_5183_public.pyth
[09/01 10:46:56][INFO] tadaconv.utils.checkpoint:  328: Keys in model not matched: []
[09/01 10:46:56][INFO] tadaconv.utils.checkpoint:  329: Keys in checkpoint not matched: []
[09/01 10:46:56][INFO] tadaconv.utils.checkpoint:  337: Model ema weights not loaded because no ema state stored in checkpoint.
[09/01 10:46:56][INFO] tadaconv.utils.checkpoint:  338: Unmatched keys in model could due to new parameters introduced,and unmatched keys in checkpoint might be caused by removing structures from the original model.Both are normal.
[09/01 10:46:56][INFO] tadaconv.datasets.base.hmdb51:   37: Reading video list from file: hmdb51_test_list.txt
[09/01 10:46:56][INFO] tadaconv.datasets.base.base_dataset:  172: Loading HMDB51 dataset list for split 'test'...
[09/01 10:46:56][INFO] tadaconv.datasets.base.base_dataset:  197: Dataset HMDB51 split test loaded. Length 15300.
[09/01 10:46:56][INFO] test:  215: Testing model for 73 iterations
[09/01 10:47:23][INFO] tadaconv.utils.logging:   89: {"cur_iter": "7", "eta": "0:00:35", "split": "test_iter", "time_diff": 0.535588}
[09/01 10:47:27][INFO] tadaconv.utils.logging:   89: {"cur_iter": "14", "eta": "0:00:33", "split": "test_iter", "time_diff": 0.552698}
[09/01 10:47:33][INFO] tadaconv.utils.logging:   89: {"cur_iter": "21", "eta": "0:00:28", "split": "test_iter", "time_diff": 0.528747}
[09/01 10:47:38][INFO] tadaconv.utils.logging:   89: {"cur_iter": "28", "eta": "0:01:19", "split": "test_iter", "time_diff": 1.731150}
[09/01 10:47:42][INFO] tadaconv.utils.logging:   89: {"cur_iter": "35", "eta": "0:00:20", "split": "test_iter", "time_diff": 0.532526}
[09/01 10:47:48][INFO] tadaconv.utils.logging:   89: {"cur_iter": "42", "eta": "0:00:17", "split": "test_iter", "time_diff": 0.536757}
[09/01 10:47:54][INFO] tadaconv.utils.logging:   89: {"cur_iter": "49", "eta": "0:00:13", "split": "test_iter", "time_diff": 0.524926}
[09/01 10:47:59][INFO] tadaconv.utils.logging:   89: {"cur_iter": "56", "eta": "0:00:09", "split": "test_iter", "time_diff": 0.518019}
[09/01 10:48:03][INFO] tadaconv.utils.logging:   89: {"cur_iter": "63", "eta": "0:00:05", "split": "test_iter", "time_diff": 0.543796}
[09/01 10:48:08][INFO] tadaconv.utils.logging:   89: {"cur_iter": "70", "eta": "0:00:01", "split": "test_iter", "time_diff": 0.485820}
[09/01 10:48:10][INFO] tadaconv.utils.logging:   89: {"split": "test_final", "top1_acc": "73.86", "top5_acc": "85.29"}
[09/01 10:53:17][INFO] tadaconv.utils.checkpoint:  511: Load from given checkpoint file.
Checkpoint file path: /home/lzh/2022/tjq/TAdaConv/checkpoint/r2p1d_pt_hmdb_ft_hmdb_5183_public.pyth
[09/01 10:53:17][INFO] tadaconv.utils.checkpoint:  328: Keys in model not matched: []
[09/01 10:53:17][INFO] tadaconv.utils.checkpoint:  329: Keys in checkpoint not matched: []
[09/01 10:53:17][INFO] tadaconv.utils.checkpoint:  337: Model ema weights not loaded because no ema state stored in checkpoint.
[09/01 10:53:17][INFO] tadaconv.utils.checkpoint:  338: Unmatched keys in model could due to new parameters introduced,and unmatched keys in checkpoint might be caused by removing structures from the original model.Both are normal.
[09/01 10:53:17][INFO] tadaconv.datasets.base.hmdb51:   37: Reading video list from file: hmdb51_test_list.txt
[09/01 10:53:17][INFO] tadaconv.datasets.base.base_dataset:  172: Loading HMDB51 dataset list for split 'test'...
[09/01 10:53:17][INFO] tadaconv.datasets.base.base_dataset:  197: Dataset HMDB51 split test loaded. Length 15300.
[09/01 10:53:17][INFO] test:  215: Testing model for 40 iterations
[09/01 10:53:52][INFO] tadaconv.utils.logging:   89: {"cur_iter": "5", "eta": "0:00:35", "split": "test_iter", "time_diff": 0.987247}
[09/01 10:53:59][INFO] tadaconv.utils.logging:   89: {"cur_iter": "10", "eta": "0:01:17", "split": "test_iter", "time_diff": 2.502493}
[09/01 10:54:04][INFO] tadaconv.utils.logging:   89: {"cur_iter": "15", "eta": "0:00:26", "split": "test_iter", "time_diff": 1.014244}
[09/01 10:54:13][INFO] tadaconv.utils.logging:   89: {"cur_iter": "20", "eta": "0:00:20", "split": "test_iter", "time_diff": 0.980417}
[09/01 10:54:18][INFO] tadaconv.utils.logging:   89: {"cur_iter": "25", "eta": "0:00:15", "split": "test_iter", "time_diff": 0.958687}
[09/01 10:54:25][INFO] tadaconv.utils.logging:   89: {"cur_iter": "30", "eta": "0:00:09", "split": "test_iter", "time_diff": 0.908797}
[09/01 10:54:30][INFO] tadaconv.utils.logging:   89: {"cur_iter": "35", "eta": "0:00:05", "split": "test_iter", "time_diff": 0.897914}
[09/01 10:54:37][INFO] tadaconv.utils.logging:   89: {"cur_iter": "40", "eta": "0:00:00", "split": "test_iter", "time_diff": 0.755412}
[09/01 10:54:38][INFO] tadaconv.utils.logging:   89**: {"split": "test_final", "top1_acc": "73.86", "top5_acc": "85.03"}**
[09/01 10:59:38][INFO] tadaconv.utils.checkpoint:  511: Load from given checkpoint file.
Checkpoint file path: /home/lzh/2022/tjq/TAdaConv/checkpoint/r2p1d_pt_hmdb_ft_hmdb_5183_public.pyth
[09/01 10:59:38][INFO] tadaconv.utils.checkpoint:  328: Keys in model not matched: []
[09/01 10:59:38][INFO] tadaconv.utils.checkpoint:  329: Keys in checkpoint not matched: []
[09/01 10:59:38][INFO] tadaconv.utils.checkpoint:  337: Model ema weights not loaded because no ema state stored in checkpoint.
[09/01 10:59:38][INFO] tadaconv.utils.checkpoint:  338: Unmatched keys in model could due to new parameters introduced,and unmatched keys in checkpoint might be caused by removing structures from the original model.Both are normal.
[09/01 10:59:38][INFO] tadaconv.datasets.base.hmdb51:   37: Reading video list from file: hmdb51_test_list.txt
[09/01 10:59:38][INFO] tadaconv.datasets.base.base_dataset:  172: Loading HMDB51 dataset list for split 'test'...
[09/01 10:59:38][INFO] tadaconv.datasets.base.base_dataset:  197: Dataset HMDB51 split test loaded. Length 15300.
[09/01 10:59:38][INFO] test:  215: Testing model for 40 iterations
[09/01 11:00:13][INFO] tadaconv.utils.logging:   89: {"cur_iter": "5", "eta": "0:00:34", "split": "test_iter", "time_diff": 0.967602}
[09/01 11:00:19][INFO] tadaconv.utils.logging:   89: {"cur_iter": "10", "eta": "0:00:58", "split": "test_iter", "time_diff": 1.893316}
[09/01 11:00:25][INFO] tadaconv.utils.logging:   89: {"cur_iter": "15", "eta": "0:00:26", "split": "test_iter", "time_diff": 1.017440}
[09/01 11:00:33][INFO] tadaconv.utils.logging:   89: {"cur_iter": "20", "eta": "0:00:35", "split": "test_iter", "time_diff": 1.672724}
[09/01 11:00:38][INFO] tadaconv.utils.logging:   89: {"cur_iter": "25", "eta": "0:00:15", "split": "test_iter", "time_diff": 0.990686}
[09/01 11:00:46][INFO] tadaconv.utils.logging:   89: {"cur_iter": "30", "eta": "0:00:10", "split": "test_iter", "time_diff": 0.915038}
[09/01 11:00:51][INFO] tadaconv.utils.logging:   89: {"cur_iter": "35", "eta": "0:00:05", "split": "test_iter", "time_diff": 0.908114}
[09/01 11:00:56][INFO] tadaconv.utils.logging:   89: {"cur_iter": "40", "eta": "0:00:00", "split": "test_iter", "time_diff": 0.777181}
[09/01 11:00:57][INFO] tadaconv.utils.logging:   89: {"split": "test_final", "top1_acc": "38.89", "top5_acc": "67.25"}
The last result is the first val split
The configuration content in val.log is as follows：
[09/01 10:46:17][INFO] test:  197: Test with config:
[09/01 10:46:17][INFO] test:  198: {
"TASK_TYPE": "classification",
"PRETRAIN": {
"ENABLE": false
},
"LOCALIZATION": {
"ENABLE": false
},
"TRAIN": {
"ENABLE": false,
"DATASET": "HMDB51",
"BATCH_SIZE": 210,
"LOG_FILE": "training_log.log",
"EVAL_PERIOD": 5,
"NUM_FOLDS": 30,
"AUTO_RESUME": true,
"CHECKPOINT_PERIOD": 10,
"INIT": "",
"CHECKPOINT_FILE_PATH": "/home/lzh/2022/tjq/TAdaConv/checkpoint/r2p1d_pt_hmdb_ft_hmdb_5183_public.pyth",
"CHECKPOINT_TYPE": "pytorch",
"CHECKPOINT_INFLATE": false,
"CHECKPOINT_PRE_PROCESS": {
"ENABLE": false
},
"FINE_TUNE": true,
"ONLY_LINEAR": false,
"LR_REDUCE": false,
"TRAIN_VAL_COMBINE": false,
"LOSS_FUNC": "cross_entropy"
},
"TEST": {
"ENABLE": true,
"DATASET": "HMDB51",
"BATCH_SIZE": 210,
"NUM_SPATIAL_CROPS": 1,
"SPATIAL_CROPS": "cc",
"NUM_ENSEMBLE_VIEWS": 1,
"LOG_FILE": "val.log",
"CHECKPOINT_FILE_PATH": "",
"CHECKPOINT_TYPE": "pytorch",
"AUTOMATIC_MULTI_SCALE_TEST": true
},
"VISUALIZATION": {
"ENABLE": false,
"NAME": "",
"FEATURE_MAPS": {
"ENABLE": false,
"BASE_OUTPUT_DIR": ""
}
},
"SUBMISSION": {
"ENABLE": false,
"SAVE_RESULTS_PATH": "test.json"
},
"DATA": {
"DATA_ROOT_DIR": "/data1/hmdb51/",
"ANNO_DIR": "/data1/hmdb51_annotations/hmdb51/",
"NUM_INPUT_FRAMES": 16,
"NUM_INPUT_CHANNELS": 3,
"SAMPLING_MODE": "interval_based",
"SAMPLING_RATE": 4,
"TRAIN_JITTER_SCALES": [
168,
224
],
"TRAIN_CROP_SIZE": 112,
"TEST_SCALE": 224,
"TEST_CROP_SIZE": 112,
"MEAN": [
0.45,
0.45,
0.45
],
"STD": [
0.225,
0.225,
0.225
],
"MULTI_LABEL": false,
"ENSEMBLE_METHOD": "sum",
"TARGET_FPS": 30,
"MINUS_INTERVAL": false,
"FPS": 30
},
"MODEL": {
"NAME": "R2Plus1D",
"EMA": {
"ENABLE": false,
"DECAY": 0.99996
}
},
"VIDEO": {
"BACKBONE": {
"DEPTH": 10,
"META_ARCH": "ResNet3D",
"NUM_FILTERS": [
64,
64,
128,
256,
512
],
"NUM_INPUT_CHANNELS": 3,
"NUM_OUT_FEATURES": 512,
"KERNEL_SIZE": [
[
3,
7,
7
],
[
3,
3,
3
],
[
3,
3,
3
],
[
3,
3,
3
],
[
3,
3,
3
]
],
"DOWNSAMPLING": [
true,
false,
true,
true,
true
],
"DOWNSAMPLING_TEMPORAL": [
false,
false,
true,
true,
true
],
"NUM_STREAMS": 1,
"EXPANSION_RATIO": 2,
"BRANCH": {
"NAME": "R2Plus1DBranch"
},
"STEM": {
"NAME": "R2Plus1DStem"
},
"NONLOCAL": {
"ENABLE": false,
"STAGES": [
5
],
"MASK_ENABLE": false
},
"INITIALIZATION": null
},
"HEAD": {
"NAME": "BaseHead",
"ACTIVATION": "softmax",
"DROPOUT_RATE": 0.5,
"NUM_CLASSES": 51
}
},
"OPTIMIZER": {
"ADJUST_LR": false,
"BASE_LR": 0.00075,
"LR_POLICY": "cosine",
"MAX_EPOCH": 300,
"MOMENTUM": 0.9,
"WEIGHT_DECAY": "1e-3",
"WARMUP_EPOCHS": 10,
"WARMUP_START_LR": 7.5e-05,
"OPTIM_METHOD": "adam",
"DAMPENING": 0.0,
"NESTEROV": true,
"BIAS_DOUBLE": false,
"NEW_PARAMS": [],
"NEW_PARAMS_MULT": 10,
"NEW_PARAMS_WD_MULT": 1,
"LAYER_WISE_LR_DECAY": 1.0,
"COSINE_AFTER_WARMUP": false,
"COSINE_END_LR": "1e-6"
},
"BN": {
"WB_LOCK": false,
"FREEZE": false,
"WEIGHT_DECAY": 0.0,
"MOMENTUM": 0.1,
"EPS": "1e-3",
"SYNC": false
},
"DATA_LOADER": {
"NUM_WORKERS": 9,
"PIN_MEMORY": false,
"ENABLE_MULTI_THREAD_DECODE": true,
"COLLATE_FN": null
},
"NUM_GPUS": 3,
"SHARD_ID": 0,
"NUM_SHARDS": 1,
"RANDOM_SEED": 0,
"OUTPUT_DIR": "output/r2p1d_mosi_ft_hmdb_test_split03",
"OUTPUT_CFG_FILE": "configuration.log",
"LOG_PERIOD": 10,
"DIST_BACKEND": "nccl",
"LOG_MODEL_INFO": true,
"LOG_CONFIG_INFO": true,
"OSS": {
"ENABLE": false,
"KEY": null,
"SECRET": null,
"ENDPOINT": null,
"CHECKPOINT_OUTPUT_PATH": null,
"SECONDARY_DATA_OSS": {
"ENABLE": false,
"KEY": null,
"SECRET": null,
"ENDPOINT": null,
"BUCKETS": [
""
]
}
},
"AUGMENTATION": {
"COLOR_AUG": true,
"BRIGHTNESS": 0.5,
"CONTRAST": 0.5,
"SATURATION": 0.5,
"HUE": 0.25,
"GRAYSCALE": 0.3,
"CONSISTENT": true,
"SHUFFLE": true,
"GRAY_FIRST": true,
"RATIO": [
0.857142857142857,
1.1666666666666667
],
"USE_GPU": false,
"MIXUP": {
"ENABLE": false,
"ALPHA": 0.0,
"PROB": 1.0,
"MODE": "batch",
"SWITCH_PROB": 0.5
},
"CUTMIX": {
"ENABLE": false,
"ALPHA": 0.0,
"MINMAX": null
},
"RANDOM_ERASING": {
"ENABLE": false,
"PROB": 0.25,
"MODE": "const",
"COUNT": [
1,
1
],
"NUM_SPLITS": 0,
"AREA_RANGE": [
0.02,
0.33
],
"MIN_ASPECT": 0.3
},
"LABEL_SMOOTHING": 0.0,
"SSV2_FLIP": false
},
"PAI": false,
"USE_MULTISEG_VAL_DIST": false
}

It is important to note that our model is trained on the first split, so it is only valid when it is evaluated on the test set of the first split.
For the problem, we suspect that we might have unmatched test lists, and we provide ours here for your information.

It is important to note that our model is trained on the first split, so it is only valid when it is evaluated on the test set of the first split.
For the problem, we suspect that we might have unmatched test lists, and we provide ours here for your information.

The accuracy (in the test file you provided) is still unsatisfactory, and the results are as follows
Checkpoint file path: /home/lzh/2022/tjq/TAdaConv/checkpoint/r2p1d_pt_hmdb_ft_hmdb_5183_public.pyth
[09/05 17:01:57][INFO] tadaconv.utils.checkpoint:  328: Keys in model not matched: []
[09/05 17:01:57][INFO] tadaconv.utils.checkpoint:  329: Keys in checkpoint not matched: []
[09/05 17:01:57][INFO] tadaconv.utils.checkpoint:  337: Model ema weights not loaded because no ema state stored in checkpoint.
[09/05 17:01:57][INFO] tadaconv.utils.checkpoint:  338: Unmatched keys in model could due to new parameters introduced,and unmatched keys in checkpoint might be caused by removing structures from the original model.Both are normal.
[09/05 17:01:57][INFO] tadaconv.datasets.base.hmdb51:   37: Reading video list from file: hmdb51_test_list.txt
[09/05 17:01:57][INFO] tadaconv.datasets.base.base_dataset:  172: Loading HMDB51 dataset list for split 'test'...
[09/05 17:01:57][INFO] tadaconv.datasets.base.base_dataset:  197: Dataset HMDB51 split test loaded. Length 15300.
[09/05 17:01:57][INFO] test:  215: Testing model for 40 iterations
[09/05 17:02:42][INFO] tadaconv.utils.logging:   89: {"cur_iter": "5", "eta": "0:00:36", "split": "test_iter", "time_diff": 1.002031}
[09/05 17:02:54][INFO] tadaconv.utils.logging:   89: {"cur_iter": "10", "eta": "0:04:17", "split": "test_iter", "time_diff": 8.309224}
[09/05 17:02:59][INFO] tadaconv.utils.logging:   89: {"cur_iter": "15", "eta": "0:00:25", "split": "test_iter", "time_diff": 0.966426}
[09/05 17:03:13][INFO] tadaconv.utils.logging:   89: {"cur_iter": "20", "eta": "0:00:20", "split": "test_iter", "time_diff": 0.974530}
[09/05 17:03:18][INFO] tadaconv.utils.logging:   89: {"cur_iter": "25", "eta": "0:00:15", "split": "test_iter", "time_diff": 0.962881}
[09/05 17:03:26][INFO] tadaconv.utils.logging:   89: {"cur_iter": "30", "eta": "0:00:10", "split": "test_iter", "time_diff": 0.929683}
[09/05 17:03:31][INFO] tadaconv.utils.logging:   89: {"cur_iter": "35", "eta": "0:00:05", "split": "test_iter", "time_diff": 0.947996}
[09/05 17:03:37][INFO] tadaconv.utils.logging:   89: {"cur_iter": "40", "eta": "0:00:00", "split": "test_iter", "time_diff": 0.773843}
[09/05 17:03:38][INFO] tadaconv.utils.logging:   89: {"split": "test_final", "top1_acc": "40.20", "top5_acc": "69.80"}
There is a fact that I have to mention, when decord=0.4.1, my program reported the following error
Error at decoding. 7/10. Vid index: 1130, Vid path: /data/hmdb51/videos/sit/TheBoondockSaints_sit_u_cm_np1_fr_med_29.avi
Traceback (most recent call last):
File "/home/lzh/2022/tjq/TAdaConv/tadaconv/datasets/base/base_dataset.py", line 349, in getitem
data, file_to_remove, success = self.decode(
File "/home/lzh/2022/tjq/TAdaConv/tadaconv/datasets/base/base_dataset.py", line 268, in decode_video
frames = dlpack.from_dlpack(vr.get_batch(list).to_dlpack()).clone()
File "/home/lzh/anaconda3/envs/tjq_tada/lib/python3.8/site-packages/decord/video_reader.py", line 163, in get_batch
arr = _CAPI_VideoReaderGetBatch(self._handle, indices)
File "/home/lzh/anaconda3/envs/tjq_tada/lib/python3.8/site-packages/decord/_ffi/_ctypes/function.py", line 173, in call
check_call(_LIB.DECORDFuncCall(
File "/home/lzh/anaconda3/envs/tjq_tada/lib/python3.8/site-packages/decord/_ffi/base.py", line 63, in check_call
raise DECORDError(py_str(_LIB.DECORDGetLastError()))
decord._ffi.base.DECORDError: [17:13:46] /io/decord/src/video/video_reader.cc:645: Error getting frame at: 8 with total frames: 80
Based on this（https://github.com/dmlc/decord/issues/124） link, I feel that there is a problem with the decord version。
Indeed, it runs normally when decord=0.4.0/0.6.0
The conda list is as follows:
_libgcc_mutex                    0.1
aliyun-python-sdk-core     2.13.36
aliyun-python-sdk-kms     2.16.1
ca-certificates                    2023.05.30
certifi                                 2023.7.22
cffi                                     1.15.1
charset-normalizer            3.2.0
crcmod                              1.7
cryptography                    41.0.3
decord                              0.4.1
einops                               0.6.1
idna                                   3.4
jmespath                           0.10.0
joblib                                1.3.2
ld_impl_linux-64               2.38
libffi                                  3.3
libgcc-ng                          9.1.0
libstdcxx-ng                      9.1.0
ncurses                             6.3
numpy                             1.24.4
openssl                            1.1.1v
oss2                                 2.18.1
Pillow                              10.0.0
pip                                  23.2.1
psutil                              5.9.5
pycparser                       2.21
pycryptodome              3.18.0
python                          3.8.13
readline                        8.1.2
requests                      2.31.0
setuptools                  68.0.0
simplejson                  3.11.1
sqlite                         3.38.5
tk                              8.6.12
torch                        1.12.1+cu113
torchaudio                0.12.1+cu113
torchvision               0.13.1+cu113
typing_extensions         4.7.1
urllib3                        2.0.4
wheel                      0.38.4
xz                            5.2.5
zlib                        1.2.12
Is it due to the decord version? Is there anything else I haven't noticed?
Looking forward to your reply. Thank you very much

This is caused by the change of cropping function that we use. Set the cfg.DATA.TEST_SCALE to 112 and the problem would be solved.

This is caused by the change of cropping function that we use. Set the cfg.DATA.TEST_SCALE to 112 and the problem would be solved.

After adding this parameter, testing in the test file you provided has indeed improved the effect, but it is still about 2 points short of your best result (using backbone r2p1d/r2d3s on hmdb51). The results are as follows:
[09/07 13:49:12][INFO] tadaconv.utils.checkpoint:  511: Load from given checkpoint file.
Checkpoint file path: /home/lwd/aim_tada/TAdaConv/checkpoint/r2p1d_pt_hmdb_ft_hmdb_5183_public.pyth
[09/07 13:49:12][INFO] tadaconv.utils.checkpoint:  328: Keys in model not matched: []
[09/07 13:49:12][INFO] tadaconv.utils.checkpoint:  329: Keys in checkpoint not matched: []
[09/07 13:49:12][INFO] tadaconv.utils.checkpoint:  337: Model ema weights not loaded because no ema state stored in checkpoint.
[09/07 13:49:12][INFO] tadaconv.utils.checkpoint:  338: Unmatched keys in model could due to new parameters introduced,and unmatched keys in checkpoint might be caused by removing structures from the original model.Both are normal.
[09/07 13:49:12][INFO] tadaconv.datasets.base.hmdb51:   37: Reading video list from file: hmdb51_test_list.txt
[09/07 13:49:12][INFO] tadaconv.datasets.base.base_dataset:  172: Loading HMDB51 dataset list for split 'test'...
[09/07 13:49:12][INFO] tadaconv.datasets.base.base_dataset:  197: Dataset HMDB51 split test loaded. Length 1530.
[09/07 13:49:12][INFO] test:  215: Testing model for 4 iterations
[09/07 13:49:33][WARNING] tadaconv.utils.meters:  147: clip count 0: 2, 1: 2 ~= num clips 1
[09/07 13:49:33][INFO] tadaconv.utils.logging:   89: {"split": "test_final", "top1_acc": "47.84", "top5_acc": "75.49"}
[09/07 13:49:39][INFO] tadaconv.datasets.base.base_dataset:  197: Dataset HMDB51 split test loaded. Length 15300.
[09/07 13:49:39][INFO] test:  215: Testing model for 40 iterations
[09/07 13:50:08][INFO] tadaconv.utils.logging:   89: {"cur_iter": "5", "eta": "0:00:21", "split": "test_iter", "time_diff": 0.595396}
[09/07 13:50:14][INFO] tadaconv.utils.logging:   89: {"cur_iter": "10", "eta": "0:00:20", "split": "test_iter", "time_diff": 0.647939}
[09/07 13:50:17][INFO] tadaconv.utils.logging:   89: {"cur_iter": "15", "eta": "0:00:17", "split": "test_iter", "time_diff": 0.656485}
[09/07 13:50:23][INFO] tadaconv.utils.logging:   89: {"cur_iter": "20", "eta": "0:00:13", "split": "test_iter", "time_diff": 0.644233}
[09/07 13:50:33][INFO] tadaconv.utils.logging:   89: {"cur_iter": "25", "eta": "0:01:46", "split": "test_iter", "time_diff": 6.626712}
[09/07 13:50:36][INFO] tadaconv.utils.logging:   89: {"cur_iter": "30", "eta": "0:00:06", "split": "test_iter", "time_diff": 0.614449}
[09/07 13:50:41][INFO] tadaconv.utils.logging:   89: {"cur_iter": "35", "eta": "0:00:03", "split": "test_iter", "time_diff": 0.557151}
[09/07 13:50:44][INFO] tadaconv.utils.logging:   89: {"cur_iter": "40", "eta": "0:00:00", "split": "test_iter", "time_diff": 0.467891}
[09/07 13:50:44][INFO] tadaconv.utils.logging:   89: {"split": "test_final", "top1_acc": "50.13", "top5_acc": "76.80"}
The configuration file is as follows：
[09/07 13:49:10][INFO] test:  197: Test with config:
[09/07 13:49:10][INFO] test:  198: {
"TASK_TYPE": "classification",
"PRETRAIN": {
"ENABLE": false
},
"LOCALIZATION": {
"ENABLE": false
},
"TRAIN": {
"ENABLE": false,
"DATASET": "HMDB51",
"BATCH_SIZE": 210,
"LOG_FILE": "training_log.log",
"EVAL_PERIOD": 5,
"NUM_FOLDS": 30,
"AUTO_RESUME": true,
"CHECKPOINT_PERIOD": 10,
"INIT": "",
"CHECKPOINT_FILE_PATH": "/home/lwd/aim_tada/TAdaConv/checkpoint/r2p1d_pt_hmdb_ft_hmdb_5183_public.pyth",
"CHECKPOINT_TYPE": "pytorch",
"CHECKPOINT_INFLATE": false,
"CHECKPOINT_PRE_PROCESS": {
"ENABLE": false
},
"FINE_TUNE": true,
"ONLY_LINEAR": false,
"LR_REDUCE": false,
"TRAIN_VAL_COMBINE": false,
"LOSS_FUNC": "cross_entropy"
},
"TEST": {
"ENABLE": true,
"DATASET": "HMDB51",
"BATCH_SIZE": 384,
"NUM_SPATIAL_CROPS": 1,
"SPATIAL_CROPS": "cc",
"NUM_ENSEMBLE_VIEWS": 1,
"LOG_FILE": "val.log",
"CHECKPOINT_FILE_PATH": "",
"CHECKPOINT_TYPE": "pytorch",
"AUTOMATIC_MULTI_SCALE_TEST": true
},
"VISUALIZATION": {
"ENABLE": false,
"NAME": "",
"FEATURE_MAPS": {
"ENABLE": false,
"BASE_OUTPUT_DIR": ""
}
},
"SUBMISSION": {
"ENABLE": false,
"SAVE_RESULTS_PATH": "test.json"
},
"DATA": {
"DATA_ROOT_DIR": "/data1/hmdb51/",
"ANNO_DIR": "/data1/hmdb51_annotations/hmdb51/",
"NUM_INPUT_FRAMES": 16,
"NUM_INPUT_CHANNELS": 3,
"SAMPLING_MODE": "interval_based",
"SAMPLING_RATE": 4,
"TRAIN_JITTER_SCALES": [
168,
224
],
"TRAIN_CROP_SIZE": 112,
"TEST_SCALE": 112,
"TEST_CROP_SIZE": 112,
"MEAN": [
0.45,
0.45,
0.45
],
"STD": [
0.225,
0.225,
0.225
],
"MULTI_LABEL": false,
"ENSEMBLE_METHOD": "sum",
"TARGET_FPS": 30,
"MINUS_INTERVAL": false,
"FPS": 30
},
"MODEL": {
"NAME": "R2Plus1D",
"EMA": {
"ENABLE": false,
"DECAY": 0.99996
}
},
"VIDEO": {
"BACKBONE": {
"DEPTH": 10,
"META_ARCH": "ResNet3D",
"NUM_FILTERS": [
64,
64,
128,
256,
512
],
"NUM_INPUT_CHANNELS": 3,
"NUM_OUT_FEATURES": 512,
"KERNEL_SIZE": [
[
3,
7,
7
],
[
3,
3,
3
],
[
3,
3,
3
],
[
3,
3,
3
],
[
3,
3,
3
]
],
"DOWNSAMPLING": [
true,
false,
true,
true,
true
],
"DOWNSAMPLING_TEMPORAL": [
false,
false,
true,
true,
true
],
"NUM_STREAMS": 1,
"EXPANSION_RATIO": 2,
"BRANCH": {
"NAME": "R2Plus1DBranch"
},
"STEM": {
"NAME": "R2Plus1DStem"
},
"NONLOCAL": {
"ENABLE": false,
"STAGES": [
5
],
"MASK_ENABLE": false
},
"INITIALIZATION": null
},
"HEAD": {
"NAME": "BaseHead",
"ACTIVATION": "softmax",
"DROPOUT_RATE": 0.5,
"NUM_CLASSES": 51
}
},
"OPTIMIZER": {
"ADJUST_LR": false,
"BASE_LR": 0.00075,
"LR_POLICY": "cosine",
"MAX_EPOCH": 300,
"MOMENTUM": 0.9,
"WEIGHT_DECAY": "1e-3",
"WARMUP_EPOCHS": 10,
"WARMUP_START_LR": 7.5e-05,
"OPTIM_METHOD": "adam",
"DAMPENING": 0.0,
"NESTEROV": true,
"BIAS_DOUBLE": false,
"NEW_PARAMS": [],
"NEW_PARAMS_MULT": 10,
"NEW_PARAMS_WD_MULT": 1,
"LAYER_WISE_LR_DECAY": 1.0,
"COSINE_AFTER_WARMUP": false,
"COSINE_END_LR": "1e-6"
},
"BN": {
"WB_LOCK": false,
"FREEZE": false,
"WEIGHT_DECAY": 0.0,
"MOMENTUM": 0.1,
"EPS": "1e-3",
"SYNC": false
},
"DATA_LOADER": {
"NUM_WORKERS": 8,
"PIN_MEMORY": false,
"ENABLE_MULTI_THREAD_DECODE": true,
"COLLATE_FN": null
},
"NUM_GPUS": 4,
"SHARD_ID": 0,
"NUM_SHARDS": 1,
"RANDOM_SEED": 0,
"OUTPUT_DIR": "output/r2p1d_mosi_ft_hmdb_test_split66",
"OUTPUT_CFG_FILE": "configuration.log",
"LOG_PERIOD": 10,
"DIST_BACKEND": "nccl",
"LOG_MODEL_INFO": true,
"LOG_CONFIG_INFO": true,
"OSS": {
"ENABLE": false,
"KEY": null,
"SECRET": null,
"ENDPOINT": null,
"CHECKPOINT_OUTPUT_PATH": null,
"SECONDARY_DATA_OSS": {
"ENABLE": false,
"KEY": null,
"SECRET": null,
"ENDPOINT": null,
"BUCKETS": [
""
]
}
},
"AUGMENTATION": {
"COLOR_AUG": true,
"BRIGHTNESS": 0.5,
"CONTRAST": 0.5,
"SATURATION": 0.5,
"HUE": 0.25,
"GRAYSCALE": 0.3,
"CONSISTENT": true,
"SHUFFLE": true,
"GRAY_FIRST": true,
"RATIO": [
0.857142857142857,
1.1666666666666667
],
"USE_GPU": false,
"MIXUP": {
"ENABLE": false,
"ALPHA": 0.0,
"PROB": 1.0,
"MODE": "batch",
"SWITCH_PROB": 0.5
},
"CUTMIX": {
"ENABLE": false,
"ALPHA": 0.0,
"MINMAX": null
},
"RANDOM_ERASING": {
"ENABLE": false,
"PROB": 0.25,
"MODE": "const",
"COUNT": [
1,
1
],
"NUM_SPLITS": 0,
"AREA_RANGE": [
0.02,
0.33
],
"MIN_ASPECT": 0.3
},
"LABEL_SMOOTHING": 0.0,
"SSV2_FLIP": false,
"COLOR_P": 0.0,
"AUTOAUGMENT": {
"ENABLE": true,
"BEFORE_CROP": true,
"TYPE": "rand-m9-n4-mstd0.5-inc1"
}
},
"PAI": false,
"USE_MULTISEG_VAL_DIST": false
}
Are there any other parameters that I need to pay attention to? For example, AUTOAUGMENT?
Looking forward to your reply, thank you very much

I do not see a problem in your current config. Autoaugment only affects the training process.
We have run on our end that we could achieve the exact 51.83 using the checkpoint on HMDB51. Are you using our list already?

On our side, we have reproduced a result similar to yours using decord==0.6.0. We used decord==0.4.0 for producing 51.83 on HMDB51.

On our side, we have reproduced a result similar to yours using decord==0.6.0. We used decord==0.4.0 for producing 51.83 on HMDB51.

Yes, I am using the test list you provided. You are right. The results I obtained above were at decord=0.6.0. When I converted decord=0.4.0, my results were as follows:
[09/07 15:25:04][INFO] tadaconv.utils.logging:   89: {"split": "test_final", "top1_acc": "51.76", "top5_acc": "78.04"}
It is very close to yours result.  This makes me extremely excited, and I am also very grateful for your patient guidance!
However, I attempted to test using the r2d3ds backbone on the hmdb51 dataset, but the results still differ by two points from yours. (I added TEST SCALE: 112 and decord=0.4.0 and the test list you provided), and the results are as follows:
Checkpoint file path: /home/lwd/aim_tada/TAdaConv/checkpoint/r2d3ds_pt_hmdb_ft_hmdb_4693_public.pyth
[09/07 16:26:19][INFO] tadaconv.utils.checkpoint:  328: Keys in model not matched: []
[09/07 16:26:19][INFO] tadaconv.utils.checkpoint:  329: Keys in checkpoint not matched: []
[09/07 16:26:19][INFO] tadaconv.utils.checkpoint:  337: Model ema weights not loaded because no ema state stored in checkpoint.
[09/07 16:26:19][INFO] tadaconv.utils.checkpoint:  338: Unmatched keys in model could due to new parameters introduced,and unmatched keys in checkpoint might be caused by removing structures from the original model.Both are normal.
[09/07 16:26:19][INFO] tadaconv.datasets.base.hmdb51:   37: Reading video list from file: hmdb51_test_list.txt
[09/07 16:26:19][INFO] tadaconv.datasets.base.base_dataset:  172: Loading HMDB51 dataset list for split 'test'...
[09/07 16:26:19][INFO] tadaconv.datasets.base.base_dataset:  197: Dataset HMDB51 split test loaded. Length 15300.
[09/07 16:26:19][INFO] test:  215: Testing model for 20 iterations
[09/07 16:27:05][INFO] tadaconv.utils.logging:   89: {"cur_iter": "5", "eta": "0:00:12", "split": "test_iter", "time_diff": 0.810198}
[09/07 16:27:23][INFO] tadaconv.utils.logging:   89: {"cur_iter": "10", "eta": "0:02:44", "split": "test_iter", "time_diff": 14.951716}
[09/07 16:27:25][INFO] tadaconv.utils.logging:   89: {"cur_iter": "15", "eta": "0:00:03", "split": "test_iter", "time_diff": 0.504766}
[09/07 16:27:44][INFO] tadaconv.utils.logging:   89: {"cur_iter": "20", "eta": "0:00:00", "split": "test_iter", "time_diff": 0.182867}
[09/07 16:27:45][INFO] tadaconv.utils.logging:   89: {"split": "test_final", "top1_acc": "44.64", "top5_acc": "72.55"}
May I ask if this result is normal, or what additional parameters do I need to add for this backbone network

Could you try using the code of this version to reproduce the result and see whether it is normal?


Norwegian translations

Localization for Norwegian (bokmål).

Thank you! I'll integrate it in 4.0.14 👍


Execution does not begin

Hi,
I have wamp server 3.0 with xdebug. If I visit my localhost:9000 the browser hangs, waiting for debug, ok, but back to atom editor the execution does not begin, and of course does not break at breakpoints. The cursor show a "forbidden" icon when I hover over "continue", "step over", ...  buttons.
Sorry for the picture, the screenshot removes the cursor.

I'm going to close this for now. If you are still experiencing this issue please reopen with your xdebug and php-debug configuration.


Update Project README

Update the benchmark project README.md to be more public-user-focused.

No longer need to refer to Bencher
Are there DH standards documented?

This has already been done.  it's also an ongoing project as more features are added.


Fix distributed logger

This seems to fix #627, I'm not exactly sure why, but I'm not super familiar with how atexit works in a distributed context (assuming that's the reason).
It'd be nice if we could add some sort of assertion, but maybe not necessary as global scope is decluttered.

It would be good to get confirmation that this is not just hiding a bug. cc @simonbyrne

It would be good to get confirmation that this is not just hiding a bug. cc @simonbyrne

Yes... Better to rule out potential bugs.
A while ago I set up a distributed simulation which had duplicate log info, it turned out that is due to an incorrect MPI setup and I was in fact running two jobs from the two processors I requested (instead of a distributed job on the two processor)...
I hope this time it is just a logger problem...

bors r+


[WIP] Blazor on the server

Putting this out there for early feedback.
Things are basically in the right places, and rendering and event handling works. I still have some issues to work out with how our client code deals with HTTP clients.
I haven't made any serious attempts at error handling yet.
TODO:

[x] Error Handling
[x] Refactor Renderer
[ ] Refactor UriHelper
[ ] Move enablenavigationinterception earlier
[ ] Remove UriHelper interop hack
[x] Allow Server-Side startup to consume client-side startup
[x] Tests For Server Side Sample
[ ] Run E2E Tests on Server Side

Planned to log issues for follow-up:

[ ] Allow server boot script to specify a non-default URL for startup
[ ] Add object pool for serialization
[ ] Make server boot script work with non-default pathbase
[ ] TODO: Also support browsers that don't have TextDecoder (e.g., Edge)

The more I see of server-side Blazor, the less I like the idea.  The whole point of Blazor is to run the CLR in the browser.  Turning it into Yet Another Server Framework not only undermines that purpose, but it lowers the quality of Blazor by introducing network latency into DOM rendering where there was none before.
And yes, I understand that this is optional, but there are still three problems that it causes anyway:

The architecture still has to be able to support it, even though it's optional.  This means various breaking changes, including completely breaking JS interop and making everything necessarily async, regardless of whether or not the underlying JavaScript code requires asynchrony, because the communication with the server would require asynchrony even if you choose not to use it.  This is not in any way a good thing, and should be in and of itself a good enough reason to kill the idea.
Working on it takes scarce resources away from other things that ought to be a higher priority.  We still don't even have debugging, for example!  Not to mention the SignalR client that's been scheduled for the next release for every single release now.
If you make it possible for people to do, some people will do it.  Any server-side Blazor program someone writes is something that can't be used offline as a PWA, because it's tied to the server.

For about decade now, ever since the smartphone went mainstream, we've been moving in the right direction as an industry, moving computation off of servers as much as possible and back to local devices.  WASM gives us the ability to do so in the browser itself, at a greater scope than has ever been possible before.  Putting Blazor on the server would buck this very important trend when we should be encouraging it and becoming leaders in it.
Please reject this PR, and the fundamental concept it's based upon.  It's the wrong direction for Blazor and for the wider industry.

Thanks for your feedback, @masonwheeler!
We know that server-side Blazor isn't going to be for everyone, or for every project. Our plan is to let people try out the server model in the 0.5.0 release and hopefully let us know how useful it is or isn't for the actual application scenarios people have in mind. If both models turn out to be useful, it might be a way of doubling the appeal of Blazor at little extra engineering cost, while also providing significant extra flexibility to developers. We just have to see when people start building with it.
On your specific concerns:

making everything necessarily async, regardless of whether or not the underlying JavaScript code requires asynchrony

Yes, you're right, it does force us to change that. However:

This change would be needed for a possible Electron model too, so there's a really big payoff from it
If you're going to use WebAssembly hosting exclusively, then you are still free to do synchronous interop! The updated interop APIs allow for either synchronous or async calls. You the developer just have to choose whether you want to support out-of-process execution (for server and electron), and if you do you need to limit yourself to async interop calls. If you are committed to WebAssembly hosting only, you're totally free to continue making sync calls.

Working on it takes scarce resources away from other things that ought to be a higher priority. We still don't even have debugging, for example! Not to mention the SignalR client that's been scheduled for the next release for every single release now.

We think the payoff from this might be big, hence prioritising the investigation now.
On debugging, we do have initial support in the 0.5.0 release. We've made great progress with this in the last couple of weeks!
On SignalR, hopefully this is transitioning to be a community-led effort.

Any server-side Blazor program someone writes is something that can't be used offline as a PWA, because it's tied to the server.

Yes, and there are things that server-side Blazor apps can do trivially that client-side ones can't (e.g., talk directly to a database, or use NuGet packages that do things that probably will never work on the client such as GPU-accelerated computation). There are pros and cons with both hosting models, so we can best understand what's useful by letting people try each of them.
In summary, I agree with a lot of your points and we will take your overall position on board. Please continue to give us feedback about this and other aspects of Blazor, especially once you've tried out the server hosting model!

This change would be needed for a possible Electron model too, so there's a really big payoff from it

Hmm... what's the value proposition there?  Because I'm having a hard time seeing one.

The purpose of Electron is to enable web developers to write desktop apps in their familiar technologies of JavaScript and HTML.
The purpose of Blazor is to enable C# developers to write web apps in their familiar technologies and take JavaScript out of the picture as much as possible.
Combining the two feels like Rube Goldberg's preferred solution for writing a desktop app in C# 😛
The only unique thing this brings to the picture is the ability to build the UI in HTML, but that feels like a step backwards compared to the power of virtually every platform's native widget set, or even existing cross-platform solutions such as Xamarin Forms.  (Which, unlike Electron, is not restricted to desktop systems.)

So I have to wonder, what benefit does Blazor + Electron bring?

If you're going to use WebAssembly hosting exclusively, then you are still free to do synchronous interop!

That's a relief.  The initial discussion of this "feature" seemed to indicate othewise.

Yes, and there are things that server-side Blazor apps can do trivially that client-side ones can't (e.g., talk directly to a database, or use NuGet packages that do things that probably will never work on the client such as GPU-accelerated computation).

Sure, but those are things that servers have always been able to do without Blazor.  This is what I mean by  "turning it into Yet Another Server Framework."  Executing C# in the browser is something new and exciting that advances the state of the art in web development.  Executing C# on the server... well... just isn't.

Some people prefer client and some prefer server version. We have already seen arguments from both sides. Now it looks that server version is almost ready. Let's try it.
In the mean time I have a few questions about this technology.

Lets assume we have an input box with bind expression. We don't have any function attached to OnKeyDown or similar events. User types something into this input box. Is this fully handled in the browser or we have client-server communication after every key press? Hopefully we will have some communication only when variable/property have to be updated in OnChange event.

User resizes browser window or changes the orientation of mobile device. Is this fully handled in the browser or visible page fragment is rerendered on the server and send to the browser?

Web page is longer (higher) than browser window. What do you send to the browser? Full page or only visible part?

Probably related to the above. I have a fixed height div element with long table inside. I see scrollbar and I can scroll this table inside div element. Do you send whole table to the browser and scrolling is fully handled in the browser?

@Andrzej-W Let’s talk about all that stuff once 0.5.0 is out, as it will all probably be quite clear once you see it. Plus I don’t want to derail this PR with a massive conversation about meta-issues. Let’s keep conversation on this PR about the code in the PR itself. Hope that’s ok!

You are right @SteveSandersonMS , sorry.

Lots of updates. This is still a rough cut - see the top post for TODO list.

OK, so a few issues that are remaining.

I rewrote the URI helper classes to be much more simpler. If you don't like what I've done let's have a discussion about what you want instead 😆. It will be somewhat difficult to rationalize the current design approach of BrowserUriHelper with what we want from the server-side version, so we might have to live with duplication if we don't have any bright ideas.
0a. I haven't really written any unit tests for the new stuff, waiting for you to do a round of feedback first.
I haven't deferred enableNavigationInterception. I feel like the code is pretty simple without doing this. Let me know what you think.
UH OH We have timing issues now with selenium. I had to skip the 'fetch data' test because of the following.

            Browser.FindElement(By.LinkText("Fetch data")).Click();
            Assert.Equal("Weather forecast", Browser.FindElement(By.TagName("h1")).Text);

The 'click' is now processed synchronously. This means that it's possible for the second line to find the old state of the page before we get the new render frames.
Even worse! The reference returned by FindElement can be invalidated before we are able to access the Text property which will cause it to throw an exception.
The solution to this is to have each navigation that we care about in tests have a selector that will be more unique so that tests are more resilient. However this means we won't be able to convert our existing tests to server-side and have them 'just work' because they rely on synchrony or timing in some cases.
I haven't done any work yet to try and bulk-convert tests, but I'd like to defer that until after this PR.

I rewrote the URI helper classes to be much more simpler. If you don't like what I've done let's have a discussion about what you want instead

Having the base class looks good! One bit I wonder if we can simplify further is avoiding the registration of "notifylocationchanged" callbacks, perhaps as follows.
In general, I'd be surprised if we can get away without providing some way of accessing the IServiceProvider in incoming JS interop calls. People will have application-specific needs for this, e.g., to mutate a scoped AppState in some way when some JS event occurs. And if we do that, then the IUriHelper situation becomes simpler, as we can put NotifyLocationChanged as a static on the URI helper base class and always call that one, not needing to register the callback target any more. This static can then access the IUriHelper from the service provider, cast to UriHelperBase and invoke its SetAbsoluteUri/TriggerOnLocationChanged.
I tried spiking this up (having another magic asynclocal in the .Blazor project called ServicesAccessor.CurrentServices) and it worked out fine. I know this has the drawback that it gives people a way of hurting their unit testability by doing static service locator patterns where they should be using DI injection elsewhere in their code, but we could limit it to be internal if we really want.
If you want my implementation of this, please let me know and I'll append it to this PR. Don't want to do so without your say-so because I don't want to trample on your work and of course you're easily able to implement it too.
Another whole approach would be to implement instance method calls in JS interop. Again that's something we'll end up doing inevitably, but I'm reluctant to do it right now because we still have ongoing discussions with the Mono folks about precisely how this is going to work, and if we go forwards unilaterally we might make it harder to work that out.

I haven't really written any unit tests for the new stuff, waiting for you to do a round of feedback first.

Sounds fine

I haven't deferred enableNavigationInterception

It looks fine as-is. We might want to reduce the number of interop round trips involved in initializing this - currently it's 3 but could be reduced to 1 if the enableNavigationInterception returned the current uri/baseUri. I did that in the spike I mentioned above and it looks OK, even though it's kind of conflating theoretically separate things.

UH OH We have timing issues now with selenium

Yeah that is awkward, but there's a good chance we can make the existing tests work out OK. I think Selenium has some way of retrying assertions briefly if they fail, as a way of dealing with this scenario. Sometimes it's still necessary to tweak how the tests are written, but we can do it.
Let's worry about getting the whole suite of E2E tests running on server rendering after this PR goes in, as you suggested.

the failures here are down to flakiness and timing. I'm going to merge this and see what I can do to harden these tests a bit as a follow up.


Merge prTesting_201904151052 branch to master branch

Docs Build status updates of commit 624d7a3:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details
Dependent Files

split/updateTestCon.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


kube-prometheus: fix correct selector

When reviewing #428, it seemed odd that before and after #428 the Prometheus instance had the correct configuration.
It turns out that we never had the correct serviceMonitorSelector, however, if a LabelSelector is empty it selects everything, which is why it still worked. Because we used a wrong key name this previous selector was parsed to be an empty and thus an "everything" selector.
Changing the selector to the correct one caused the Prometheus and Alertmanagers not to be configured correctly, but the labels just had to be adapted.
@mxinden @fabxc @gouthamve

Grafana is not a scrape target, I don't even think it exposes any metrics, at least not in Prometheus format.
example/node-exporter-monitor.yaml doesn't seem to be used anywhere, I'll remove it.


Workspace opensourcing

List of tasks that would allow the workspace to be public

[ ] Solidify build scripts, identify supported platforms
[ ] Review installation documentation #139
[x] Other documentation links #229
[ ] Limited CI builds on public facing repo? #136
[ ] Quality of life improvements
[ ] General Roadmap of the next 6th months of development
[ ] We'd need a logo :-), trademark, legal protections
[x] Licenses
[ ] Patenting
[ ] Contributions
[ ] Questions and requests
[ ] Demos

Preparing release repositories: https://github.com/ros2-gbp/ros2-gbp-github-org/pull/68
I have skipped infrastructure, drake, and delphyne ones for the first cycle.


Default rolls 2d6 not 1d6

According to the 'Standard Notation' and the readme default should be 1d6.

Added a line of explanation to the README - let me know if it doesn't makes sense?
Figured updating the default behavior would seem odd for existing uses, so instead documented the original intent 👍


Added support for windows 7 (local GPO) - wip

To be perfectly honest: PowerShell is not feeling home to me, so please handle with care. It seems to be working for me, the exclusion is added to the options.
Caveats:
You cannot delete the path from the options via GUI (they will just reappear, apparently local GPO will just override that option). It can be removed via regedit.exe though.
I don't have a windows 8 system for test available right now, so hopefully I did not break the existing functionality.

Thanks a lot, this is awesome! :tada: I made a small comment to make it runs on Windows 10, but if you put that in we suddenly have Windows 7 support!


video and audio not work in Android 7 with chrome v51

My app work in android 8,9 but in android 7 all videos and audios in local storage not work. I have uninstalled the webview plugin and the videos and audios work correctly in the different versions. Which is an error in this plugin.

What is your plugin version?
Does it work if you update Chrome?

I have last versión ionic webview. The smartphone is Meizu.
El mar., 4 jun. 2019 18:46, jcesarmobile<EMAIL_ADDRESS>escribió:

What is your plugin version?
Does it work if you update Chrome?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ionic-team/cordova-plugin-ionic-webview/issues/372?email_source=notifications&email_token=AF7LR6YYNKABGTUUOLER2BLPY2L6HA5CNFSM4HSZNFS2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW5FVUY#issuecomment-498752211,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF7LR62IMNNBJA2YG2DRHPDPY2L6HANCNFSM4HSZNFSQ
.

Some chrome versions had problems, but I can't reproduce now, so closing.


Ignore 'special files' (originally: Ignore the /proc filesystem)

Just ran: dua /. After a while, it printed:
141.02 TB /    <3 IO Error(s)>

...my local disk is not that large. By, uh, several orders of magnitude.
Going to / and running dua:
   0.00  B lost+found
   0.00  B mnt
   0.00  B root
   0.00  B srv
   2.54 KB dev
   3.52 KB .scripts
  85.26 KB run
   1.54 MB tmp
  30.85 MB etc
  81.55 MB boot
 657.30 MB sys
   1.85 GB bin
   1.85 GB sbin
  10.86 GB lib
  10.86 GB lib64
  18.13 GB opt
  19.76 GB usr
 118.48 GB home <1 IO Error(s)>
 130.30 GB var
 140.74 TB proc <2 IO Error(s)>
 141.05 TB total        <3 IO Error(s)>

Seems like /proc is reporting very weird sizes.
By the way, the different between dua / and cd /; dua was surprising, even though correct as per the help message.

This is dua 1.2.2.

@passcod  Thanks a lot for letting me know! Right now, it is only being tested on OSX.
Regarding the difference in behaviour between dua and dua / - it's actually a feature. One might be inclined to do dua ~/* and expect to see everything, even though this will skip hidden directories. The dua usecase is tailored to you being somewhere on the disk and wondering: 'How much space is used here, just show it all...'. That way its just more convenient, too.
Could you do cd / && dua i and drill in to find our which part of /proc is misreported?
I think I could determine if a file is actually a device and skip it.
Thanks a bunch!

The culprit is /proc/kcore, which is a virtual file that "contains" all possible memory. Linux supports up to 128TB of memory, so that's where that comes from. I'm not sure that ever counting procfs mounts is useful unless queried specifically, though: they'll always be zero-sized on disk, even if the files within do have a size.

Thank you! By the looks of it, the output is actually correct. I would be inclined to not call this one a bug, but only if du does the same. If du is smarter and ignores these special files, I think dua should do the same.
Could you let me know what du -sch /proc (or similar) produces?

# du -sch /proc
du: cannot read directory '/proc/1466/task/1466/net': Invalid argument
du: cannot read directory '/proc/1466/net': Invalid argument
du: cannot access '/proc/19393/task/19393/fd/4': No such file or directory
du: cannot access '/proc/19393/task/19393/fdinfo/4': No such file or directory
du: cannot access '/proc/19393/fd/3': No such file or directory
du: cannot access '/proc/19393/fdinfo/3': No such file or directory
0       /proc
0       total

Thank you, I believe dua should handle such files differently, and it should be possible to ignore them by checking the file flags in the file meta-data.

In dua v2.6.0, there is the -x flag to stay on the current file system and not cross mount points. This might already help with this issue.
Otherwise it's the question if something [in the unix specific metadata](file:///Users/sthiel/.rustup/toolchains/stable-x86_64-apple-darwin/share/doc/rust/html/std/os/unix/fs/trait.MetadataExt.html) can be used to differentiate them.
In case you get to test it and find it working, please let me know.

I'd forgotten all about this, and 2.5.0 (what arch installs currently) already reports zero for /proc, so all's fine.


Results KeyError

I get a KeyError when running the GDPbyIndustry function. It doesn't seem like it can be an issue on my end unless I am missing something simple
My code:
import datapungibea as dpb
bea = dpb.data('ID')
gdp_data = bea.GDPbyIndustry('211','2','A','2018')
print(gdp_data)
Output:
gdp_data = bea.GDPbyIndustry('211','2','A','2018')
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/datapungibea/api.py", line 36, in wrapper
return(getattr(delegate_object, called_method)(*args, **kwargs))
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/datapungibea/drivers.py", line 890, in GDPbyIndustry
output       = self._cleanOutput(query,retrivedData) #a dict of a df or df and meta (tablePretty)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/datapungibea/drivers.py", line 904, in _cleanOutput
df_output =  pd.DataFrame(retrivedData.json()['BEAAPI']['Results']['Data'])
KeyError: 'Results'

Hi lydonjj, thank you for your message.  Sorry, I only saw this today.  It seems that BEA changed the API, the code was fixed to account for this change.  The package was updated to account for this.


feat: PIZ-5 new report page

🤔 Why?
The stakeholders have requested the implementation of three new reports that they need to visualize within the app.
🛠 What I changed:

Implemented the new page that calls the new reports endpoint and presents the information.
Change the other pages navbars to include the reports page

🗃 Trello Issues:

PIZ5 - Create a Report Endpoint

 PIZ5 - Create a Report Endpoint


Compare only command output and improve diff logic

I consolidated and shifted this pull request into another as per the request to utilize the latest updates in CLT for improved outcomes in failed tests. This will also simplify future merges due to multiple changes: https://github.com/manticoresoftware/clt/pull/7


Added icons and logo.

👷 Deploy Preview for san-tagoy-xyz processing.

Name
Link

🔨 Latest commit
ee3d86fa2cb8ae2d72f54b354787f15e191d975c

🔍 Latest deploy log
https://app.netlify.com/sites/san-tagoy-xyz/deploys/6319a52f5f3679000869e6cf


Fix floatingLabel animation

This is commit fixes the expected animation for the floating label,
which should change the fontSize and line height when the field is
focused.
This is basically reverting commit 3e3ba4e, which was theoretically fixing #2248, but the true is that the root cause of that issue was not in that line, and instead it caused a regression were the floatingLabel is not resizing to an smaller size on focusing the input.
The current behaviour without this fix is:

The expected behaviour is (making the label smaller when focusing):

With this fix you're still able to change the color of the label by applying style directly to it (which I understand was what initially was reported as an issue in #2248):

@SupriyaKalghatgi can you take a look at this? Is a trivial fix and it affects the UX for people that upgraded to v2.8.1

@Angelk90 there are two ways that should work, but the behavior will depend on the version of native-base you are using:

Add the color style directly to your Label in this way: <Label style={{color: 'blue'}}>.
1.1 If you're using 2.8.1 this will change the label color for both floating/not floating, but the font-size won't decrease when floating.
1.2 If you're using 2.8.0 this will change the label color for not floating, but not when floating
1.3 If this commit is merged it'll behave as expected
Eject the theme and change the inputColorPlaceholder property.
2.1 If you're using 2.8.1 it'll behave the same as point 1.1 above
2.2 If you're using 2.8.0 it'll behave as expected
2.3 If this commit is merged it'll behave as expected

@SupriyaKalghatgi I amended the commit, and it now fixes #2248 and removes the regression for not changing the label font size when floating. The root cause of the issue was that StyleSheet.flatten should have been called with an array, otherwise it just returns the first argument and therefore was disregarding the actual Label styles coming from the props.

@elyalvarado : The button in the input field for the clear? Can you post an example, for example on snack.expo.io?

@Angelk90, I see what you mean now.
The button clearing the fields that you see at the right end of it in my gifs is the native iOS clear button, which I enabled by passing  clearButtonMode: 'while-editing'  as prop to the Input, which in turns passes it to the underlying react-native TextInput component (see React Native Documentation.
Unfortunately, that property is iOS specific, and you can't style it.

@elyalvarado I will take a look on this


Error on migrate in postgresql database

SQLSTATE[42830]: Invalid foreign key: 7 ERROR:  there is no unique constraint that matches the given columns in the referred table «currencies» (Connection: pgsql, SQL: alter table "accounts" add constraint "accounts_currency_code_foreign" foreign key ("currency_code") references "currencies" ("code") on delete set null).
The same error in the creation of the accounts, company_defaults and contacts tables.

Hmm. I suppose PostgreSQL is not as lenient or feature rich regarding composites. Can you try updating the foreign key references in each table:
From this:
$table->foreign('currency_code')->references('code')->on('currencies')->nullOnDelete();

To this:
$table->foreign(['currency_code', 'company_id'])->references(['code', 'company_id'])->on('currencies')->nullOnDelete();

I've investigated the foreign key constraint issue with PostgreSQL and found a solution. The issue is due to PostgreSQL's strict requirements for composite foreign key constraints.
Temporary Fix
You'll need to adjust the foreign key definition in the accounts table (and others) to match the order of columns in the currencies table's composite unique index. Here's what you should change:
In the accounts table migration (along with others), find the foreign key definition that references the currencies table. Update it to match the order of columns as they appear in the currencies table's unique constraint. It should be updated to look like this:
$table->foreign(['company_id', 'currency_code'])
                ->references(['company_id', 'code'])
                ->on('currencies')
                ->restrictOnDelete();

Also make sure to change the currency_code column in each table to be non-nullable.
This adjustment aligns with PostgreSQL's requirements and should resolve the foreign key constraint issue you're experiencing.
I will be updating the application with this fix sometime soon, but this change should help you overcome the immediate issue.
If you have any further questions or need more assistance, feel free to reach out.

Yes, this solves the problem.


Support use server directive for Server Actions

Is your feature request related to a problem? Please describe.
Next.js 13.4 introduced Server Actions, which require a new directive 'use server'.
Describe the solution you'd like
This new 'use server' directive should also be formatted in a similar way to how 'use client' or 'use strict' directives are formatted:
https://github.com/daidodo/format-imports/blob/db5d42e419446b442595d3ed631fad47db93382d/src/lib/format/parser/parser.ts#L143
This is what a 'use server' directive looks like at the top of a file:
'use server'; // Should be at top of file to indicate that this file should only ever run on the server

// more code...

One thing to keep in mind is that 'use server' can also be allowed in a function instead of at the top of a file:
// app/post/[id]/page.tsx

import kv from './kv';

export default function Page({ params }) {
  async function increment() {
    'use server';
    await kv.incr(`post:id:${params.id}`);
  }

  return (
    <form action={increment}>
      <button type="submit">Like</button>
    </form>
  );
}

Describe alternatives you've considered
Not formatting things and having a problem with Server Actions.
Additional context
'use client' was added for React 18 in this PR:

https://github.com/daidodo/format-imports/pull/16

Thanks for the feedback!
Yes, the directive can be handled the same way as use client. And the in-function directive is not a problem because the parser cares about top level statements only.
All suggestions and PRs will be welcomed!

Hey everyone, I would like to take a look into this 🙂

Just opened a PR here #24!

Published v7.6.3 to include the feature.


question

May I ask if the version of tk3 etc. in the code can be found in Table 1 of the paper? Still not listed

Please refer to the paper for more details.


Fixes #3824

Fixed #3824, added tests and removed deprecation warnings.

will merge after the CI builds pass


Issue in connecting with Luis app during app run

Github issues for C# /JS / Java/ Python should be used for bugs and feature requests. Use Stack Overflow for general "how-to" questions.
Sample information

Sample type: [\samples\ or \solutions]
Sample language: [dotnetcore or webapi or es6 or nodejs or typescript]
Sample name:

Describe the bug
Give a clear and concise description of what the bug is.
Error occurred in request., ConnectionError: HTTPSConnectionPool(host='https', port=443): Max retries exceeded with url: /southcXXX.api.XXXXX.microsoft.com/luis/v2.0/ap
ps/?log=true (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x000001A60DCE9948>: Failed to establish a
new connection: [Errno 11001] getaddrinfo failed'))
To Reproduce
Steps to reproduce the behavior:

Go to '...'  Run app using python app.py
Click on '....' Start chatbot emulator and open biot with http://localhost:3978/api/messages
Scroll down to '....'
See error

Expected behavior
Give a clear and concise description of what you expected to happen.
Connection to luis and then proper recognizing of intent
in config.py setup luis app and id and host
api host configured in below format
LUIS_API_HOST_NAME ="https://southXXXX.api.XXX.microsoft.com
Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.
[bug]

@penguintechlinux - This is the samples repo. The issues in this repo should pertain to specific samples in the repo. I'm closing this issue here so it can be addressed on Stack Overflow: https://stackoverflow.com/questions/60226501/issue-in-connecting-with-luis-platform-from-python-v4-botframework


Some questions when running inference

您好，感谢您的开源内容。
我在跑推理的时候，在这一步——Build PointcloudBackbone: PointNetPP频繁报错
Error executing job with overrides: ['num_gpu=1', 'note=tuning_noact']
代码问题似乎是来源于pointnetpp.yaml文件中的路径设置
理论上该文件中的path：TBD项应该是pointnetpp的pth文件吧，但是我在完成setup install和测试后没有在相关的文件夹中找到合适的pth文件
请问我的操作哪里有问题呢

You should manually download a checkpoint and then specify the path. See model weights.

不好意思，我疏忽了这点。
感谢您的回答

感谢您的帮助，我已经完成了推理过程，但是推理结果中的response输出全部是乱码
我测试了3rscan和scannet的多个scene，并试着更改了instruction，但是仍然没有得到看起来正常的结果

Can you provide more specific details so that I can reproduce your results?

一切内容都是按照您提供的README文档设置的，我使用的llm是vicnua7b，vision3d是pointnetpp
在设置了所需对应的文件路径后，我将inference.sh中的pretrained_ckpt_path设置为tuning_noact.pth
随后使用bash inference.sh开始推理，监测文档显示模型build、load、freeze全部正常
成功从tuning_noact.pth加载模型后，得到如下推理结果：
[2024-04-11 16:03:32,523][main][INFO] - {'source': '3rscan', 'scene_id': '0ad2d3a1-79e2-2212-9b99-a96495d9f7fe', 'situation': '', 'instruction': 'Describe this scene.', 'response': '~'}
其中，最后省略的~部分全部是乱码，我尝试在inference.py进行json写入时控制参数ensure_ascii=False，但并无作用
另外，我更换了default.yaml中的probe: source、scene_id、instruction进行测试，但都没有帮助
如果您需要的话，我可以为您提供推理结果的probe文件夹，其中包含了监测日志和推理保存的pth

I just instructed the model to "Describe this scene" for 3rscan:0ad2d3a1-79e2-2212-9b99-a96495d9f7fe. The response is reasonable, which begins with "In this bathroom scene, there is...".
Can you provide your logs (INFO)?

inference.log
Here is the log with the same scene and source

I suspect the reason is that you have not loaded Vicuna-7B properly. The name LlamaToken is weird and there is no log saying Loading checkpoint shards which should appear when loading Vicuna model. What files are there in your LlamaToken folder and where are they from?

The folder "LlamaToken" is used to save the vicuna7b from HuggingFace, because I have to search and download it manually due to the "ConnectionError, port=443".
And the files I obtained are all from "https://huggingface.co/lmsys/vicuna-7b-v1.5"

Weights mismatch. We use Vicuna weights of version v1.1 (see Model weights).

The files you posted on HuggingFace seem to have a slight difference from "lmsys/vicuna-7b-v1.1"
I'm downloading the files you provided, but I still wanna ask : Can both of them be used normally?

We have not checked whether our vicuna-7b-v1.1 is identical to lmsys. We manually applied the delta weights (provided by lmsys) to LLaMA weights to derive the vicuna-7b-v1.1 weights. I am not sure whether this could lead to the difference. BTW, can you show where the slight difference comes from?

I haven't tried using lmsys in the experiment yet. But when downloading the files you provided, the difference in file size caught my attention.
The file "config.json", I opened both two files and found the settings for parameters such as "bos_token_id"、“eos_token_id”、"pad_token_id" and "transformers_version" were different. That's may be the SLIGHT difference I mentioned.
I don't have time to explore the effects of these parameters. If the adjustments are to adapt to this project, then I think I have no problem for now. Thanks a lot.

感谢您的帮助，我已经完成了推理过程，但是推理结果中的response输出全部是乱码 我测试了3rscan和scannet的多个scene，并试着更改了instruction，但是仍然没有得到看起来正常的结果

hi，我能否询问一下您的环境配置是cuda 12+的版本吗

感谢您的帮助，我已经完成了推理过程，但是推理结果中的response输出全部是乱码 我测试了3rscan和scannet的多个scene，并试着更改了instruction，但是仍然没有得到看起来正常的结果

hi，我能否询问一下您的环境配置是cuda 12+的版本吗

yes, cuda 12.0

感谢您的帮助，我已经完成了推理过程，但是推理结果中的response输出全部是乱码 我测试了3rscan和scannet的多个scene，并试着更改了instruction，但是仍然没有得到看起来正常的结果

hi，我能否询问一下您的环境配置是cuda 12+的版本吗

yes, cuda 12.0

请问一下方便给个环境的txt文件吗，我也是cuda12，但是总是没发跑pointnet这个backbone

感谢您的帮助，我已经完成了推理过程，但是推理结果中的response输出全部是乱码 我测试了3rscan和scannet的多个scene，并试着更改了instruction，但是仍然没有得到看起来正常的结果

hi，我能否询问一下您的环境配置是cuda 12+的版本吗

yes, cuda 12.0

请问一下方便给个环境的txt文件吗，我也是cuda12，但是总是没发跑pointnet这个backbone

req.txt

感谢您的帮助，我已经完成了推理过程，但是推理结果中的response输出全部是乱码 我测试了3rscan和scannet的多个scene，并试着更改了instruction，但是仍然没有得到看起来正常的结果

hi，我能否询问一下您的环境配置是cuda 12+的版本吗

yes, cuda 12.0

请问一下方便给个环境的txt文件吗，我也是cuda12，但是总是没发跑pointnet这个backbone

按Readme的教程应该就能跑，我最开始有一些from调用的问题，当时好像是pointnet的文件下载到其它文件夹了，调整过来就正常了

感谢您的帮助，我已经完成了推理过程，但是推理结果中的response输出全部是乱码 我测试了3rscan和scannet的多个scene，并试着更改了instruction，但是仍然没有得到看起来正常的结果

hi，我能否询问一下您的环境配置是cuda 12+的版本吗

yes, cuda 12.0

请问一下方便给个环境的txt文件吗，我也是cuda12，但是总是没发跑pointnet这个backbone

req.txt

你这个环境配置文件是不是少了一点包

When I want to install pointnetpp, this error accur

 When I want to install pointnetpp, this error accur

i can't understand why your cuda cannot be backward compatible. Perhaps you can check to see if there are other versions that are causing this problem.

 When I want to install pointnetpp, this error accur

Maybe you can install pytorch at other versions that support cuda 12. For example, maybe try this:
conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=12.1 -c pytorch -c nvidia

 When I want to install pointnetpp, this error accur

Maybe you can install pytorch at other versions that support cuda 12. For example, maybe try this:
conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=12.1 -c pytorch -c nvidia

I successfully install pointnet++, but I cant import it:
Using /home/yiyangai/miniconda3/envs/chengxing_leo_try/lib/python3.9/site-packages
Finished processing dependencies for pointnet2==3.0.0

this error appears:
(chengxing_leo_try) (base) yiyangai@maxwell:~/Projects/chengxingxie/embodied-generalist$ python -c 'from model.pointnetpp.pointnetpp import PointNetPP'
Traceback (most recent call last):
  File "/home/yiyangai/Projects/chengxingxie/embodied-generalist/model/pointnetpp/pointnet2_utils.py", line 23, in <module>
    import pointnet2._ext as _ext
ImportError: /home/yiyangai/miniconda3/envs/chengxing_leo_try/lib/python3.9/site-packages/pointnet2-3.0.0-py3.9-linux-x86_64.egg/pointnet2/_ext.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZNK3c1017SymbolicShapeMeta18init_is_contiguousEv

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/yiyangai/Projects/chengxingxie/embodied-generalist/model/__init__.py", line 2, in <module>
    from .vision3d import OSE3D
  File "/home/yiyangai/Projects/chengxingxie/embodied-generalist/model/vision3d.py", line 7, in <module>
    from model.pcd_backbone import PointcloudBackbone
  File "/home/yiyangai/Projects/chengxingxie/embodied-generalist/model/pcd_backbone.py", line 9, in <module>
    from model.pointnetpp.pointnetpp import PointNetPP
  File "/home/yiyangai/Projects/chengxingxie/embodied-generalist/model/pointnetpp/pointnetpp.py", line 3, in <module>
    from model.pointnetpp.pointnet2_modules import PointnetSAModule
  File "/home/yiyangai/Projects/chengxingxie/embodied-generalist/model/pointnetpp/pointnet2_modules.py", line 21, in <module>
    import pointnet2_utils
  File "/home/yiyangai/Projects/chengxingxie/embodied-generalist/model/pointnetpp/pointnet2_utils.py", line 26, in <module>
    raise ImportError(
ImportError: Could not import _ext module.
Please see the setup instructions in the README: https://github.com/erikwijmans/Pointnet2_PyTorch/blob/master/README.rst

I finally found out how to solve the import error, I hope you can include this in the readme part:
You should install pointnet++ from the original repo:
git clone https://github.com/erikwijmans/Pointnet2_PyTorch.git
cd Pointnet2_PyTorch
pip install pointnet2_ops_lib/.

If the last step trigger another error like:
      nvcc fatal   : Unsupported gpu architecture 'compute_37'
      error: command '/usr/local/cuda/bin/nvcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pointnet2_ops
  Running setup.py clean for pointnet2_ops
  error: subprocess-exited-with-error

  × python setup.py clean did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-zi8vpr6o/pointnet2-ops_03476694dfae46a19d022fa9ff7ccadf/pointnet2_ops_lib/setup.py", line 17, in <module>
          exec(open(osp.join("pointnet2_ops", "_version.py")).read())
      FileNotFoundError: [Errno 2] No such file or directory: 'pointnet2_ops/_version.py'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed cleaning build dir for pointnet2_ops
Failed to build pointnet2_ops
ERROR: Could not build wheels for pointnet2_ops, which is required to install pyproject.toml-based projects

you should open setup.py file and change:
# os.environ["TORCH_CUDA_ARCH_LIST"] = "3.7+PTX;5.0;6.0;6.1;6.2;7.0;7.5"
os.environ["TORCH_CUDA_ARCH_LIST"] = "5.0;6.0;6.1;6.2;7.0;7.5"

Then you can run pip install pointnet2_ops_lib/..
After that, go back to the model/pointnetpp/pointnet2_utils.py
change the _ext line to import pointnet2_ops._ext as _ext, then you can run the command to check if you can import pointnetpp:
python -c 'from model.pointnetpp.pointnetpp import PointNetPP'

I don't know what happens to your environment. But there is nothing wrong with my environment. I simply create a new conda env and follow the readme to install pointnetpp. It still works and I can import pointnetpp.


Error invoking gcov

Error invoking gcov: /home/ssingasa/app/src/helloworld.c.gcno:version 'B23*', prefer 'B14*'
gcc --version
gcc (GCC) 12.3.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

My initial guess is that you've got a new version of gcc and/or gcov recently. Have you tried a clean build?

Yes

Not sure if this applies to you, but in the past I had problems with doing clean builds when I use ccache. It might not realize that there is a new version. If you're using ccache or something similar, can you clear its cache?

i have the same issue:
Error invoking gcov: c:\a\b\source.c.gcno:version 'B32*', prefer 'B33*'
c:\a\b\source.c.gcda:version 'B32*', prefer version 'B33*'
.\gcc.exe --version
gcc.exe (i686-posix-dwarf-rev1, Built by MinGW-Builds project) 13.2.0


[Gally]: master <- dev

Automatically created by Git-Ally

:tada: This PR is included in version 5.0.4 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


configure polling interval

Motivation
#290

Solution
Add methods to RpcClient and RootProvider to configure the polling interval.

PR Checklist

[ ] Added Tests
[ ] Added Documentation
[ ] Breaking changes

This wouldn't work. Making changes


Did ContentBehavior go away?

Watching this video: http://channel9.msdn.com/Shows/Web+Camps+TV/Create-your-own-Domain-Specific-Language-in-ASPNET-with-TagHelpers
I'm creating the same "EmailTagHelper" but the attribute ContentBehavior is nowhere to be found "ctrl + ." doesn't give any suggestion except generating a class.
What could be missing here? (The project is a "starter"-project.)
Without the attribute it compiles and runs, but without any "magic" happening to my "email"-tag.
I have the VS 14.0.22823.1 and project.json says
"Microsoft.AspNet.Mvc.TagHelpers": "6.0.0-beta4"
[ContentBehavior(ContentBehavior.Append)]
public class EmailTagHelper : TagHelper
{
    public static string EmailDomain = "mycompany.com";
    public override void Process(TagHelperContext context, TagHelperOutput output)
    {
        output.TagName = "a";
        output.Content.SetContent(output.Content + "@" + EmailDomain);
        output.Attributes["href"] = "mailto:" + output.Content;
    }
}

Ya we removed ContentBehavior in beta4, it's been designed away (https://github.com/aspnet/Razor/issues/221). Sorry for the confusion, i've reached out to get the comment updated on the video .
An equivalent EmailTagHelper now looks like:
public class EmailTagHelper : TagHelper
{
    public static string EmailDomain = "mycompany.com";
    public override void Process(TagHelperContext context, TagHelperOutput output)
    {
        output.TagName = "a";
        output.PostContent.SetContent(output.Content + "@" + EmailDomain);
        output.Attributes["href"] = "mailto:" + output.Content;
    }
}

Note the Content => PostContent change :smile:

Works better now =)
But getting the content seems to be not working. While in debug-mode the output.Content is empty while the markup looks like:
robert
and the output is:
<EMAIL_ADDRESS>Bug?

@roblom Ahh yes, so we transitioned "receiving" the content into a method call. You can do this as follows:
// Executes everything inside of your TagHelper and returns the rendered output
var childContent = await context.GetChildContentAsync();

Aka you can't directly retrieve what's inside of the TagHelper via the .Content property, you have to choose to invoke it.

All good!


Refactor multipart download to a more async model

The previous approach of kicking off the stream requests for all parts
of a file did not work well for very large files. For example, a 20GiB
file uploaded in 16MiB parts will consist of 1200+ parts. When we
attempted to initiate streaming for all parts concurrently, some parts
would hit a client timeout after 2 minutes without being able to get a
connection due to the other parts not having been completed in that time
frame. This refactoring adds yet another layer of indirection in order
to allow the code that is actually writing the destination file to
control the rate at which streams are started. This should allow for
downloading files consisting of arbitrarily many parts at any connection
speed.
This commit also wires in the download rate limiter so that the
indices.recovery.max_bytes_per_sec is properly honored.
This PR supersedes #10284
Check List

[x] New functionality includes testing.

[x] All tests pass

[x] New functionality has been documented.

[x] New functionality has javadoc added

[x] Commits are signed per the DCO using --signoff
[x] Commit changes are listed out in CHANGELOG.md file (See: Changelog)
[x] GitHub issue/PR created in OpenSearch documentation repo for the required public documentation changes (#[Issue/PR number])

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@andrross I think you need to rebase. There is a serialization version change which might be causing test failures.

#9115

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/26967/
CommitID: a9b8b63
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

I'm going to merge this to unblock subsequent PRs, but will follow up with @Bukhtawar and @gbbafna and address any comments or concerns.


choosing room page (welcome) needs to be styled

make sure to stick to the theme and use colors we have consistently used: White, yellow, green, pink.
So I guess the main theme color is white.
And the secondary colors are pastel.


Use Apptainer in the HTCondor tarball

Is your feature request related to a problem? Please describe.
The jobs which require Apptainer depend on a good installation on the EP and that is not always the case.
Describe the solution you'd like
GlideinWMS should use the Apptainer included in the HTCondor tarball
Describe alternatives you've considered
I've not thought about other solutions.
Info (please complete the following information):

Priority: High
Stakeholders: OSG
Components: glidein

Additional context
apptainer has been included in HTCondor tarballs since version 10.7.0. It is invoked just like any of the condor_* commands. It is in /bin, /usr/bin.

@mambelli you mentioned configuration that we could apply ahead of time (hopefully in the FE?) to institute this. Could you pass that along here?


Allow sugar cane to be placeable on sand

Modern Minecraft allows sugar cane to be growable on sand. I see no reason why we wouldn't also allow this as a config option in AnnoyanceFix

This will be a part of the FinalBeta mod: https://github.com/telvarost/FinalBeta-StationAPI


Issue 18 - Develop the backend of the Counter Application in the preproject branch

The backend will provide the following services, APIs:

[x]  1.  getCount(): Get the count of an object in the PostgreSQL database.
[x]  2.  incrementCount(): Increment the count of an object in the postgreSQL database.
[x]  3.  resetCount(): Reset the count of an object in the database to 0.
[x]  4.  getCurrentTime(): Get the current time and date.

Issue resolved
Commit hash: 1b75bb4


Investigate use of AstVisitorBase

We all traverse the AST with manual pattern matching (for implement interface/record stub/union cases generation), but since the tree is huge, we often forget or misconsider the cases (cf a recent issue opened by Vasily...)
I wonder if using the AstVisitorBase object would help us treat more cases by default?
This way we would homogenize our code base with something more "robust".
https://github.com/fsharp/FSharp.Compiler.Service/blob/f3e5a7cc181e921115e91944ad17d9cc54cefb6f/src/fsharp/vs/ServiceParseTreeWalk.fs#L47

I don't think we plan to do this anytime soon. Closing for now.


command for opening terminal in a container odo terminal

/kind user-story
User Story
As an odo user, I want to be able to open a terminal in the container with my application So that I can run commands in the container and debug application.
Acceptance Criteria

[ ] odo terminal command should automatically open bash/sh in the container where the application is currently running
[ ] odo terminal --container mycontainer should open terminal to a container named mycontainer
[ ] if there are multiple containers and the user doesn't specify one with --container flag odo should error out and list all container names and tell the user to specify one.

/remove-lifecycle stale
/priority Medium
Still relevant, and bumping priority to Medium because it came as a feature request following a recent talk at a local meetup.

Would really like this feature! commenting for visibility.


There is no view currently focused

Refs: https://github.com/microsoft/vscode/issues/91335
Move a view via a "Move a focused view" command
Once the view is moved notice how the focus is still on the view
Trigger the command again -> get the There is no view currently focused error.
Please try it a couple of times, I do not repro 100%

I hit this too, there is something wrong with the context key, need to investigate.

I can still reproduce this with VSCode 0eebaf9ad23748baca2afc0da425650922bc23de
Which seems to contain the fix. Thus reopening

@isidorn I can use the command, were you trying to do it from the command palette? If so, the command palette steals focus so it won't work there. The above commit will keep it from appearing there to avoid confusion.

You are correct. And yes it should not appear there
If I use the shortcut it works fine -> verified


taxonKey filter on checklists out of order?

taxonKey filter on checklists out of order?
We searched for taxonKey=1355948 and found it should be in 14 checklists. When clicking the link alle checklists are displayed and not only the 14 which contains taxon 1355948

fbitem-0059f96b33a12ae38a9a2449c0865b3a3cdfe76c
Reported by: @NHMD
System: Chrome 55.0.2883 / Mac OS X 10.10.2
Referer: https://demo.gbif.org/dataset/search?type=CHECKLIST&taxonKey=1355948
Window size: width 1835 - height 1065
API log
Site log

Thank you for reporting @NHMD
How did you end up with that url?
It is intended to be: https://demo.gbif.org/dataset/search?type=CHECKLIST&taxon_key=1355948
Using the interface I haven't been able to reproduce the error
I don't have any preference on underscore vs camel case, but at the time of implementation the general opinion at the office seemed to be underscores. It is a bit confusing though as the API documentation states camelCase (but supports both). The site of course do not have to align to the same parameter naming and it isn't at this point.

I ended up on the url by clicking the link to the 12 checklist datasets containing the species on this page: https://demo.gbif.org/species/1353364
Just under the text "Dataset coverage".

Thank you
Odd - i thought i just tried that, but I now see that it is wrong. I have released a new version. Notice that we still have issues with cache busting https://github.com/gbif/portal16/issues/27 so you might need to do a hard refresh to get the new version. closed by https://github.com/gbif/portal16/commit/3169e3ac7f4ddfb0ef3100f31b6359d22c5ff529


Use casbin as a storage backend

Hi mikespook, I'm Yang Luo, the author of casbin. It is an authorization library that supports models like MAC, RBAC, ABAC.
I'm lucky to find the gorbac project when searching authorization in Go:) Compared to gorbac, casbin is more low-level and doesn't provide a friendly RBAC interface as gorbac. But it provides permission storage via file, DB. And I found that currently gorbac's persistence is still not perfect. So what do you think of using casbin as a storage backend for gorbac? Of course it's better if you want to use it as the enforcement engine too. Looking forward to your reply.

Hi Yang,
Thank you for the suggestion. I did some rough research about casbin. It's a good idea to have some abstract descriptions file to model the privileges stuffs.
However, I think we have the different concept about the libraries. I've explained some ideas in the post you can check. Thus goRBAC won't have any persistence functions, unless the users have to implement their own one. On the other hand, casbin can be considered as a full-functional middleware, isn't it?
BTW, I saw you wrote the rbac manager in casbin. I think it has the similar scope to goRBAC. What if we replace the RBAC manager by using goRBAC? It seems a more practical way for supplying a user-friendly API.
Anyway, I'm open my mind to discuss further. Hoping we can make two projects working together.
Cheers,
Xing

However, I think we have the different concept about the libraries. I've explained some ideas in the post you can check. Thus goRBAC won't have any persistence functions, unless the users have to implement their own one. On the other hand, casbin can be considered as a full-functional middleware, isn't it?

Well, some users indeed don't want persistence but there are still someone do. However I understand that you want to keep it light-weight.

BTW, I saw you wrote the rbac manager in casbin. I think it has the similar scope to goRBAC. What if we replace the RBAC manager by using goRBAC? It seems a more practical way for supplying a user-friendly API.

Yeah. Because RBAC is one of casbin's supported models. goRBAC can detect some errors like InherCircle which is a good function and provides more friendly API, which makes it at a higher layer (closer to user) than casbin. However, I also can't give up the current primitive API because non-RBAC users will need to use it. Or it makes no difference with gorbac, except that it has a persistence way.