SHAP TreeExplainer is not compatible with XGBoost models that use Pandas categorical features

As of XGBoost 1.6.0, it is possible to use Pandas categorical dtypes as inputs to models, avoiding the need for encoding:
https://xgboost.readthedocs.io/en/latest/tutorials/categorical.html
However, if you do this, XGBoost only allows you to serialise the model using JSON / UBJSON. SHAP currently loads the trees in a model by reading a raw binary dump generated via the save_raw() method, so it can't load models that make use of this functionality. Attempting to do so results in the following XGBoost error:
  explainer = TreeExplainer(model)
  File "/usr/local/lib/python3.9/site-packages/shap/explainers/_tree.py", line 149, in __init__
    self.model = TreeEnsemble(model, self.data, self.data_missing, model_output)
  File "/usr/local/lib/python3.9/site-packages/shap/explainers/_tree.py", line 824, in __init__
    xgb_loader = XGBTreeModelLoader(self.original_model)
  File "/usr/local/lib/python3.9/site-packages/shap/explainers/_tree.py", line 1431, in __init__
    self.buf = xgb_model.save_raw()
  File "/usr/local/lib/python3.9/site-packages/xgboost/core.py", line 2216, in save_raw
    _check_call(
  File "/usr/local/lib/python3.9/site-packages/xgboost/core.py", line 203, in _check_call
    raise XGBoostError(py_str(_LIB.XGBGetLastError()))
xgboost.core.XGBoostError: [16:41:34] ../src/tree/tree_model.cc:871: Check failed: !HasCategoricalSplit(): Please use JSON/UBJSON for saving models with categorical splits.

Are there any plans to add support for this type of model, e.g. by using the JSON dump instead to load tree data if enable_categorical is true for the model?
Apologies if this is a known issue or there is a simple workaround that allows users to continue using this XGBoost functionality with SHAP - I searched briefly on the issues page but I couldn't find anything that seemed to match this. I got the error above when using the 0.41.0 release.

@dataversenomad thanks for the suggestions. Unfortunately, I need to be able to convert my model to PMML using sklearn2pmml, so I can't use xgb.train - only the scikit-learn interface will work for me. I am using tree_method = 'hist'.

@rwjmiller did you find a possible quick solution to this problem?
@dataversenomad I'm not sure that this is true using, tree_method="approx" or "hist" gives me the same error with xgb.train. It seems that you were getting no errors because you were using "exact", which actually doesn't use categorical features? (see this table)

Hi @rwjmiller, @dataversenomad and @YannDubs, I'm also facing the same problem. I don't think it's possible to get a quick fix and it also looks like support for JSON-serialized models won't be added to SHAP anytime soon.
Referencing this comment inside XGBTreeModelLoader, my guess is that the developers chose not to support the JSON/UBJSON serialization of XGBoost models because JSON doesn't specify a precision for floating point values. In edge cases where this precision becomes important to determine  the result of a node's split, such information may be lost when we save in the JSON format.
If we wanted to force XGBTreeModelLoader to accept JSON-serialized models regardless, we will need to rewrite the entire __init__() because it has been built around the default binary dump returned by xgboost.Booster.save_raw(). Until XGBoost supports serialization of categorical-enabled models in formats other than JSON/UBJSON, we might not be able to use TreeExplainer with such models.
Perhaps the most practical solution right now would be to use LightGBM's LGBMClassifier. LightGBM implements the same optimal partitioning technique to handle categorical data that is still experimental in XGBoost. I've tested LGBMClassifier and can confirm it works with TreeExplainer. It has a sklearn API too, so it might be able to fulfil your PMML needs.
Thanks!

UBJSON preserves the floating points exactly.

I can work on parsing the ubjson model.

Hi everybody, I'm facing the same issue. Any update on this?
Thanks!

I've tested LGBMClassifier and can confirm it works with TreeExplainer.

@meeps123 's comment is correct in that you are able to extract shap values from a lgbm model.
However the visualizations (e.g. beeswarm plot, force plots) are still broken for categorical variables (even with lgbm).

I also had this issue last week. The way I solved it is not using SHAP library.
https://xgboost.readthedocs.io/en/stable/python/python_api.html#xgboost.Booster.predict
If we set the pred_contribs == True, the we get the shap values as output from the predict method.
 shap_values = self.model.predict(xgb.DMatrix(self.data, enable_categorical=True), pred_contribs=True)
Also remove the last column because it contains the bias term.
shap_values[:, :-1]

@ajaychawda58 Thanks for this solution. A question: when using the shap library, there is also explainer.expected_value which is a single number. Do you know how we should get it when using the pred_contribs parameter as you explained?

@Hassan-Hafez The expected value is the last column of the shap_values array. It's a constant value, so you can just take the first row.
shap_values[0, -1]

shap_values = self.model.predict(xgb.DMatrix(self.data, enable_categorical=True), pred_contribs=True)

I got the shape values using your methods. Now, I want to find the most important contributing features for each label (contributing features that led the model to decide on that specific category). For example, I used only those rows that have category (label) X. It's shape is 1545 (rows) by 177 (features). The output of shap_values is (1545, 119, 178). Can you help me with that?

any update for shap to handle xgboost with categorical data, natively?
it's been a while,
thanks

to complete the very useful @ajaychawda58 response.
In case of a multilcass, we peak first class [0] for shap
x_dmatrix = xgboost.DMatrix(x, feature_types=feature_types, enable_categorical=True)
shap_values = learner_.predict(x_dmatrix, pred_contribs=True)
base_values = shap_values[0, :, -1]
base_values_0 = base_values[0]
shap_values_0 = shap_values[:, 0, :-1]
explanation = shap.Explanation( values=shap_values_0, base_values=base_values_0, feature_names=feature_names)
shap.plots.waterfall(explanation[0])

as stated above, a native shap integration would be better

Hi, I opened a PR for loading the UBJ/JSON format: https://github.com/shap/shap/pull/3462
When the tree explainer is using the SHAP value from XGBoost, categorical splits should work. However, if the SHAP implementation in the SHAP package is used instead, I don't know what will happen. Based on the code comments I found, it's not yet supported. Someone with familiarity with the package might shed some light. In addition, I can't be sure if the plotting is working correctly.
I have added logic to parse the JSON model with categorical splits, would be great if I could have some guidance on what the next step is to have the full support of categorical features.

When the perturbation is path dependent and prediction is directed to XGBoost, the SHAP values for categorical features can be calculated:
import shap
import xgboost as xgb
from shap import TreeExplainer
from xgboost.testing import make_categorical

def test_cat():
    X, y = make_categorical(512, 32, 9, False)
    reg = xgb.XGBRegressor(enable_categorical=True, n_estimators=10)
    reg.fit(X, y)
    print(reg.get_booster().feature_types)

    explainer = TreeExplainer(reg, feature_perturbation="tree_path_dependent")
    s = explainer(X)

Next would be working on interventional with a background dataset.


[Patch] Updated copyright and added default value

Adding to the review one more thing:

Remove every copyright appearance in the CDK docs, it must now use the preset value.


Is there any way to change the color of the spinner?

Is there any way to change the color of the spinner?

Hello!
Right now you can use something like this (if you don't changed default activity indicator):
(spinner.indicator.indicatorView as? CircleIndicator)?.lineColor = UIColor.blackColor()

I'll move color settings to SpinnerConfiguration in next release

Thanks :D.


fixed braziers for wintertodt + rs2grounditem.loot

fixed braziers for wintertodt, fixed bug in wintertodt where player would be stuck in WAITING state (i think i did, anyway), fixed rs2grounditem.loot for 50 range instead of 255. tutorialisland fix still.

I will close this one out as it no longer contains any relevant code to the original PR. I am keen to integrate your tutorial island changes to the script, if you want to PR those seperately


Pane resizing shortcut

Using hyper, there is one feature I'm missing and couldn't for the life of me find it -> resizing panes using only the keyboard. I'm only Ubuntu 20.4, using hyper 3.0.2.
I came across hyerinator and hyperlayour which would allow me to create layouts with asymmetrical pane sizes but what I want is something like terminator's ctrl+shift+arrow keys to be able to resize a pane.
Does this exist/any plans on making this?

It's really easy to be achieved by a plugin, but firstly, someone need to write this plugin😅

@bet4it yeah I assumed as much, guess it gives me a reason to learn some JS for a change, was just hoping maybe someone already came across it and made it


Gaze Direction

Hello,
Regarding the talk we had I attached 4 photos. It think there is sth wrong because I don't get even a single frame correct gaze direction.

and I have another question:
where can I find the FACE related prints on the left up corner, first line? I just want to simply change the 'FACE: FACING CAMERA' to sth els, e.g. FACE: FACING CENTER. just changing a word. I check index.js, node.js files but did not find where you print those words.
Thank you

Regarding the talk we had I attached 4 photos. It think there is sth wrong because I don't get even a single frame correct gaze direction.

Those are good examples - and yes, my simplified math is just too simple - comparing area sizes of irises is insufficient when you're facing camera, but looking away from camera
I'll update here when I have a new solution ready

where can I find the FACE related prints on the left up corner, first line? I just want to simply change the 'FACE: FACING CAMERA' to sth els, e.g. FACE: FACING CENTER. just changing a word. I check index.js, node.js files but did not find where you print those words.

inside demo/index.js:drawResults() it just calls built-in helper functions indside Human to draw
and the one you're looking for is src/draw/draw.ts:face()
but again, they just print what they get from human.detect() result object - words are defined in src/gesture/gesture.ts
so you can either:

implement a custom draw function instead of using helpers in human.draw
modify words in src/gesture/gesture.ts and rebuild human
or just edit the result before calling draw functions

for example, really silly but simple way of converting entire result to string, replacing strings and returning back as result
(better way would be to walk the object and replace values as needed)
let result = await human.detect(input);
const replace = JSON.stringify(result).replace('facing camera`, `facing center`);
result = JSON.parse(replace);

These are what I have in the Human folder.
I don't have demo and src folders.

is it ok to just download the missing folders and putting them into my Human folder?

this does not look like human npm package or git clone, this looks like someone manually copied demo folder and placed dist and models folders inside demo and that's what you're using.
where does this copy come from and how was it installed?

A friend of mine prepared it and gave it to me. I just got the file and extracted it.
how should I fix it or can I continue with it?

you can continue using it, but then you wont be able to update easily as i rollout any fixes or changes (your friend would have to prepare it again)
documentation you can see online here on github, so that's not an issue
but you're missing actual sources, you only see the demo (things in '/dist' are compiled and minimized, so not readable) - that's why you cannot find things when you're searching for strings
you could also download everything from here on github, but then you also wouldn't be able to update automatically
so it would be best to either use npm install ... to install NPM package or git clone ... to clone repository
anyhow, for purpose of this issue, to rename facing camera to facing center you don't need any of that, can still do that by doing either:

simple object -> string -> replace -> object conversion
walk the object and replace values as needed

ok, i've just made some changes to gestures:

face: 'facing center' (was 'facing camera')
iris: 'facing center' (was 'looking at camera', that is the old algorithm comparing left/right iris area size)
iris: 'looking center', 'looking left' and 'looking right' (new algorithm comparing distance of iris center to corner of eye)

i've tried also 'looking up' and 'looking down', but human eye just doesn't have height compared to iris height to be able to run math precise enough - it's ok for close zooms of eye, but otherwise not precise enough so i won't enable it.
new code is already on github and will be published on npmjs later this week (waiting for some fixes from tfjs team to be able to bundle new tfjs 3.4.0)

Hello Vladimir,

* iris: 'looking center', 'looking left' and 'looking right' (new algorithm comparing distance of iris center to corner of eye)

yes, this is exactly the thing I was looking for. thank you. Now,Looking left and right works very well since you get the corner points as reference. For Up and Down, it is possible to get the up and down extreme points of eyes as the reference. since eyes have several landmark points for each point, it is possible to get the upper most and lower most landmark points' ID/index and referring to it looking up and down should work. This is what I did in the algorithm ı was previously wrote.

I still need to find where you print those stuffs at the left up corner.
the only thing I found from the folder that I have is the below screenshot.

I don't know maybe js is completely different from what I know (python). I want to see the lines of code that you print those strings, Especially the first line (FACE: .....)

or Up and Down, it is possible to get the up and down extreme points of eyes as the reference. since each eye hase several landmark points, it is possible to get the upper most and lower most landmark points' ID/index and referring to it, looking up and down should work.

Yes, it can be done and I've tried it - the problem is reliability of results:

iris is partially obstructed by eyelid (especially when looking down),
so detection of iris is not that reliable
extreme points of eye on vertical side are much smaller than on horizontal side,
so again math results are not that reliable

and I can not find the source of last explanation you made (simple object -> string -> replace -> object conversion - walk the object and replace values as needed).

Those are two different methods, not two steps in one method:
simple object -> string -> replace -> object conversion:
let result = await human.detect(input);
if (result && result.gesture) {
  const replace = JSON.stringify(result).replace('facing center`, 'i want to say something different');
  result = JSON.parse(replace);
}

walk the object and replace values as needed):
const result = await human.detect(input);
if (result && result.gesture) {
  for (const gesture of result.gesture) {
    if (gesture.gesture === 'facing center') {
      gesture.gesture = 'i want to say something different';
    }
  }
}

or the same with a map and regex function:
const result = await human.detect(input);
if (result && result.gesture) {
  result.gesture = result.gesture.map((gesture) => {
    gesture.gesture = gesture.gesture.replace(/facing center/, 'i want to say something different')
    return gesture;
  });
}

ok, i've changed it to include up/down:

first check if 'facing center' by comparing iris area sizes
then check for both 'looking left/right' and 'looking up/down'
if if facing center and not looking in any direction, then 'looking center'

as suspected, precision is not perfect, but why not

Currently, I don't have but I will have as soon as I get able to install the library (Human). As you said it is better to have the right package but I have difficulty to install it. I clone the repository and do the npm i but I get this error. I tried some solutions but it didn't resolve the error.

Hope you can help :)

to install human, you don't need to clone it and then install it's dependencies
instead, just run npm install @vladmandic/human to install human package from npmjs and that's it
then to update human at any time, just run npm update to update to latest minor version or npm update --latest to update to latest major version
or alternatively copy human from git main branch using git clone https://github.com/vladmandic/human
in that case, to update to latest changes, you'd use git pull
but again, no need to run npm install inside human in either case
installing dependencies inside human is only needed if you plan to make changes to the library as it installs devDepenendencies
and in that case, best procedure would be to have a separate tree for human:

crate git fork of human
run `git clone
run npm install inside your cloned folder
make modififications as you want
rebuild library using npm run build
run lint checks using npm run lint
run automated tests using npm run test

and such local fork can be used in your actual project by installing it from local path

run npm install <path-to-your-copy>

i'm closing this issue as related to gaze detection.
if there are any other questions, let me know.

Hello,
I am able now to play with prints at the left corner, and changing them the way I want,
I need to find two more things:

There are several lines of print for FACE and IRIS, and the order of them are not fix, it is confusing.
I want to give them unique names. for example instead of two FACEs, I want to make the first one as FACE1: and second one FACE2:. as well for IRISs. Where can I edit them?
In the box created over the face I want to deactivate some modules or at least their prints, how to do it? when I comment out the line 165 of index.js, all the prints in the box disappear.

Thank you

best is to take a look at how drawing is done in /src/draw/draw.ts and copy those functions to your code and use them instead of built-in one. for example, in index.js, inside function drawResults, it calls human.draw.face() - remove that and replace with call to your function. and you can use /src/draw/draw.ts:face() as template to build that function. same for gestures.

Hello Vladimir,
I have tested the gaze direction in several different situations and the performance is not robust, as soon as I changed the position of the camera or light condition the accuracy drops because gaze direction function is a rule-based (mathematics based) and the ratios will change with both position of camera and the person.
I thought it will be a good idea to integrate a robust DL based gaze estimation model into Human. There is an open source python pre-trained model that I am using for my study.
I am supposed to change the algorithm to JS and then integrate it to Human, or is there any way to skip this conversion part?
What is your idea? any solution to make the gaze estimation part work more accurate?
Thx

I have tested the gaze direction in several different situations and the performance is not robust, as soon as I changed the position of the camera or light condition the accuracy drops. Actually I expected it because gaze direction function is a rule-based (mathematics based) and the ratios will change with both position of camera and the person. I thought it will be a good idea to integrate a robust DL based gaze estimation model into Human. There is an open source python pre-trained model that I am using for my study. I am supposed to change the algorithm to JS and then integrate it to Human, or is there any way to skip this conversion part?

Can you share the model so I can take a look?

The Iris distance is working reverse, When I get more close to the webcam it shows higher distance and we I get more far from the camera it shows lesser distance.

Iris was calculating iris size, not actual distance. I've corrected that, latest code is on git.
Values returned now are distance from camera in cm corrected for a typical webcam field of view of 88 degrees
(for example, when i'm sitting in front of notebook, iris distance will be ~30cm).
Note that there is no way to determine camera field of view programatically, so for more correct measurements user should adjust this value accordingly.
Btw, this iris distance was an actual issue - can you in the future open a separate issue for such items so it can be tracked and closed correctly?

What do you do exactly in the Human to find the gaze?
Is there any room to improve the gaze finding part? Is it possible to make it more robust?
I just shared the link via email.
Sure I will open separate issue for iris distance.
Thanks

No need to open separate issue for Iris distance anymore since it's already fixed in main branch as of few days ago
Just please do so for any new issues you find in the future
Anyhow, I've just created a new discussion item and tried to answer most of the questions there: https://github.com/vladmandic/human/discussions/124

Hi Vladimir,
I checked the iris detection of mediapipe and I faced problem regarding the directions correctly and while searching for similar issues I found this. It can give the iris position inside the eyes correctly when it is at right and left but for up and down it gives wrong estimation since the eyelid landmarks move as iris landmarks move and by the result the position of iris inside the sclera (eye) remains constant. see the video in the link I shared.
I think you have this problem for Human as well. Do you have any idea?
Thx

exactly the same problem and i don't see an easy way out. but its also a low priority for me given everything else.


Feature/sign-login

The functions you have programmed works as expected. However, you should redesign the styling of your forms to match the overall aesthetic of our website. I'm going to accept the PR because some teammates need this functionality to work on theirs, but fix it as soon as possible.


Cargo install's new updating feature doesn't pull from git repo.

Problem
Cargo's new updating feature doesn't pull from git repo.
Steps

cargo install --git https://github.com/nushell/nushell.git nu --features stable make it point at something before HEAD.
cargo install nu
cargo will pull from crates.io instead of the git repo.

Possible Solution(s)
I know that nu is on crates.io but as I'm actively working on it (and also I love it) I wanted to pull from the repo so as to get all the bugs as I'm using it. It could be that I'm just using this feature incorrectly.
Notes
Output of cargo version:
cargo 1.41.0 (626f0f40e 2019-12-03)

This is intentional.  You need to specify the source each time you run cargo install, otherwise it wouldn't know whether or not you wanted to switch sources.  That is, you need to specify --git each time you want to install from the git repo.

Do I have to include the url or just --git?

The command more or less needs to be the same (--git with url, the same features, etc.).

Oh, that's not terrible ergonomic. If someone could point me in the right direction, I could probably implement a flag or something to indicate where to update from.

It sounds like you're looking for a new command for managing installed packages.  I think something like that would require an RFC process to work on the design.  There are also third-party plugins like cargo-update, which I think might do something similar.

No, not necessarily a new command, but a flag for the existing install command makes sense to me. If that's out of scope or something, it's no big deal, I can just use the cargo-update plugin like you suggested.


React component in node_modules doesn't fast-refresh

Describe the bug
The case is I'd like to support fast-refresh for React components in node_modules.
For example a React component in package foo, I've configured as:
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
import { createRequire } from 'module'

const require = createRequire(import.meta.url);

const packages = ['foo']

// https://vitejs.dev/config/
export default defineConfig({
  server: {
    watch: {
      ignored: packages.map((p) => `!**/node_modules/${p}/**`)
    }
  },
  optimizeDeps: {
    entries: packages.map((p) => require.resolve(p)),
    exclude: packages
  },
  plugins: [react()],
})

From what I've tested so far:

On macOS it works as expected. When I modify the React component. The UI updates immediately.
However, on GitHub Codespaces, or WSL (Windows Subsystem for Linux), nothing happens after I modify it.
In @teambit, we have a self-maintained Cloud Workspace service, quite like GitHub Codespaces. It doesn't work on there neither.

Reproduction
https://github.com/Jinjiang/reproductions/tree/vite-watch-node-modules-20240913
Steps to reproduce
How to reproduce:
pnpm install
pnpm dev

then modify ./node_modules/foo/index.js:

on macOS the fast refresh works
on GitHub Codespace the UI doesn't update at all
on WSL the UI doesn't update at all (even with { usePolling: true })

System Info
System:
    OS: Linux 6.5 Ubuntu 20.04.6 LTS (Focal Fossa)
    CPU: (2) x64 AMD EPYC 7763 64-Core Processor
    Memory: 4.64 GB / 7.74 GB
    Container: Yes
    Shell: 5.0.17 - /bin/bash
  Binaries:
    Node: 20.16.0 - ~/nvm/current/bin/node
    Yarn: 1.22.22 - /usr/bin/yarn
    npm: 10.8.1 - ~/nvm/current/bin/npm
    pnpm: 9.7.1 - ~/nvm/current/bin/pnpm
  npmPackages:
    @vitejs/plugin-react: ^4.3.1 => 4.3.1
    vite: ^5.4.1 => 5.4.4

Used Package Manager
pnpm
Logs
No response
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to vuejs/core instead.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

I thought this is expected, but just confirmed it's not working on my Linux PC either.
Also, it even requires server restart + devtool disable cache to have a new code on browser after modifying ./node_modules/foo/index.js.

The file add event seems to be not called.
Duplicate of #8619


Implement $MUDROD_HOME/bin/mudrod script

Implement a $MUDROD_HOME/bin/mudrod script which will provide users with options to execute log ingestion, log deletion, etc. from the command line e.g. $MUDROD_HOME/bin/mudrod logimport /path/to/log, this will then run https://github.com/mudrod/mudrod/blob/war/core/src/main/java/esiptestbed/mudrod/weblog/pre/ImportLogFile.java with the relevant input parameter(s) in an ad-hoc fashion rather than EVERY time mud rod is started.

This issue is now addressed and you can run ./bin/mudrod to see a list of 'tools' we can invoke in a more friendly manner.


Store data in smaller steps with reseting emissions data

CodeCarbon version: 2.5.0
Python version: 3.12.4
Operating System: Windows 10

Description
I found some issues from last year, where some people asked for saving emissions of loop steps seperatly without using checkpoints like flush or making new instances of the Emission Tracker. I can't find a way to get the emissions of each step still. I am not sure, if I just don't understand how I can do it.
What I Did
Right now I achieve this by creating a new instance of the tracker in each loop step like that:
def run_method():
    for i in range(1, 5):
        tracker = EmissionsTracker()
        tracker.start()
        # do stuff ...
        emissions: float = tracker.stop()

So it kind of works, but it always prints this:
[codecarbon INFO @ 15:02:39] [setup] RAM Tracking...
[codecarbon INFO @ 15:02:39] [setup] GPU Tracking...
....
....

And if I do this:
def run_method():
    tracker = EmissionsTracker()
    for i in range(1, 5):
        tracker.start()
        # do stuff ...
        emissions: float = tracker.stop()

It always prints this:
[codecarbon WARNING @ 15:04:02] Tracker already stopped !
and add the "kWh of electricity used since the beginning." value and Energy consumed for ... values up after each step

From the documentation in quickstart you can use
task mode that will allow this behavior :
def run_method():
    tracker = EmissionsTracker()
    for i in range(1, 5):
        tracker.start_task(f"task_{i}")
        # do stuff ...
        emissions: float = tracker.stop_task()
    tracker.stop()

I already tried that, but it gives me this error:
Traceback (most recent call last):
  File "C:\Users\headscott\PycharmProjects\RobustBenchCarbon\run_models.py", line 37, in <module>
    run_attack()
  File "C:\Users\headscott\PycharmProjects\RobustBenchCarbon\run_models.py", line 19, in run_attack
    tracker.start_task(f"task_{i}")
  File "C:\Python312\Lib\site-packages\codecarbon\emissions_tracker.py", line 484, in start_task
    _ = self._prepare_emissions_data()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\codecarbon\emissions_tracker.py", line 627, in _prepare_emissions_data
    emissions_rate=emissions / duration.seconds,  # kg/s
                   ~~~~~~~~~~^~~~~~~~~~~~~~~~~~
ZeroDivisionError: float division by zero

Can you confirm it has been solved by https://github.com/mlco2/codecarbon/pull/589 ?

It fixed this exact error, I am not sure why but now there is another error. So this is my code:
def method():
tracker = EmissionsTracker()
emissions_list = []
try:
for i in range(1, 3):
tracker.start_task(f"task_{i}")
# do stuff
emissions: float = tracker.stop_task()
emissions_list.append(emissions)
finally:
tracker.stop()
mean_emissions = np.mean(emissions_list)
std_emissions = np.std(emissions_list)

print(f"Mean emissions: {mean_emissions} kgCO2 equivalent")
print(f"Standard deviation of emissions: {std_emissions} kgCO2 equivalent")

C:\Python312\Lib\site-packages\codecarbon\output_methods\file.py:79: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
df = pd.concat(
Traceback (most recent call last):
File "C:\Users\headscott\PycharmProjects\RobustBenchCarbon\run_models.py", line 39, in
method()
File "C:\Users\headscott\PycharmProjects\RobustBenchCarbon\run_models.py", line 29, in run_attack
mean_emissions = np.mean(emissions_list)
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python312\Lib\site-packages\numpy\core\fromnumeric.py", line 3504, in mean
return _methods._mean(a, axis=axis, dtype=dtype,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python312\Lib\site-packages\numpy\core_methods.py", line 118, in _mean
ret = umr_sum(arr, axis, dtype, out, keepdims, where=where)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: unsupported operand type(s) for +: 'EmissionsData' and 'EmissionsData'
Seems like the list is filles with elements of EmissionsData, even though stop_task gives float values. But I saw task_emission_data which is returned there, is actually of type EmissionsData

Yes, tracker.stop_task() does not return a float, but an object EmissionsData, you may do emissions: float = tracker.stop_task().emissions.
The documentation is wrong, sorry for that.

Okay thank you, this solved my problem. So I will close it thx


[WIP][BETA] Jasmine 2.0 Support

hacking in 2.0 support, pulling out things that are confusing, such as RequireJS support which I'm still not fully convinced ever worked.
you can now install this for testing through npm via:
npm install -g<EMAIL_ADDRESS>

I think that you should put a big fat warning that the timer mocking might not work as expected because it mock only {set,clear}{Interval,Timeout} functions, and Node has also other scheduling mechanism, process.nextTick.
Btw, there is also setImmediate, which can work AFAIK both in node and browsers... Maybe it's worth filing an issue for Jasmine itelf. :confused:

Looks like a vote to abandon Jasmine2.0 version (#437).

I hadn't seen that you added commits to this, was trying to clean up my github.com interface @brodybits. Can re-open the PR if you think it's going to be used, as I hadn't touched it since 2014 I thought it was long-gone.

I would personally rather keep this project in maintenance mode, no new features unless we can get someone else like yourself to help with support and maintenance. Keep in mind that the recommend solution is for users to migrate to jasmine or jasmine-npm.


Calendar Function Query

This PR contains the code for the query function to create a list of time ranges that fit the meetings request. Ideally, I would need to implement the optional attendees (and the test cases to go with it) in a different PR. I left some of my minor comments in the code for readability purposes, but I plan to only leave the more relevant ones when finally merged.

Sorry if I added any comments that Olivia already addressed yesterday, the github UI doesn't do a great job highlighting which lines of code have already been discussed.


[IC3 ACS Resource Provider - UI Library] SDK Review - Follow up

New SDK Review meeting has been requested.
Service Name: IC3 ACS Resource Provider - UI Library
Review Created By: Jorge Garcia Hirota
Review Date: 11/23/2022 2:00 PM PT
Hero Scenarios Link: Not Provided
Architecture Diagram link: Not Provided
Core Concepts Doc Link: Not Provided
APIView Links: iOS, Android,
Description: Review the new version of the UI Library for mobile platforms:
Enable the Mobile UI LIbrary to surface to users call information like the call id, so that it can be provided through support channels.
Detailed meeting information and documents provided can be accessed here

Cancelled by System Admin


Update train.py

Refactor and Optimize Model Training Script
This commit introduces significant improvements and optimizations to the model training script used in the Yugabyte database project. The training script has been enhanced for better code organization, readability, and configurability, providing users with a more intuitive experience.
Changes Made:
Argument Parsing Enhancement: The script now employs the argparse module to parse command-line arguments effectively. This change facilitates easy customization of the model, data, and training hyperparameters, allowing users to fine-tune the training process to suit their specific needs.
CustomTrainer Class: A new class named CustomTrainer has been added, extending the Trainer class from the Transformers library. This custom trainer class provides improved data loading mechanisms by implementing get_train_dataloader and get_eval_dataloader methods with distributed data loading support.
Model Configuration Flexibility: The script now offers more flexibility in configuring the model based on specific arguments. Users can choose to use LORA (Language Oriented Reliability Architecture) with various hyperparameters or opt for a standard model without LORA. The script automatically adjusts the model architecture and gradient checkpointing based on user preferences.
Code Comments and Clarity: Comprehensive and informative comments have been added throughout the script to guide users on where to make specific model configurations based on command-line arguments. These comments improve code readability and aid in understanding the underlying logic.
Code Optimization: Redundant and unnecessary statements have been removed, resulting in cleaner code and improved execution efficiency.
The enhancements introduced in this commit ensure that the model training script adheres to best practices and is well-optimized for both beginners and experienced users. By leveraging the power of the Transformers library and customizing the training process with ease, users can achieve more efficient and accurate model training for their specific tasks

Did you use ChatGPT to rewrite it?


Edge support data is invalid for dateStyle

Hi,
Check this site: https://caniuse.com/mdn-javascript_builtins_intl_datetimeformat_datetimeformat_datestyle
As Edge has the same Chromium engine as Chrome, I tried, and latest Edge support this feature.

As this is part of the MDN BCD data, you can update the data there or file an issue there.


chore: use Submodule.Quotient-specific induction lemma

Can you update the existing uses of Quotient.inductionOn specific to Submodule.Quotient?

Thanks for the PR! Can you update the existing uses of Quotient.inductionOn specific to Submodule.Quotient?

Done, thanks :)

Thanks!
bors merge

why is it failing I thought I passed all the tests :(

The "check the cache" step failed. Looks like a fluke

The "check the cache" step failed. Looks like a fluke

Anything I can do about it?

bors r+

bors r+


Feature request: Remove comment box, or allow adding rows

Would be very helpful to be able to add new rows to a set of boxed comments and have the extension detect the new rows and just add the delimiters to those rows, or at least remove the existing comment box and regenerate again to account for new rows being added. As of now you can pretty much only use it once, and most documentation grows over time

double wraps

Hello, and thanks for the feedback!
I've been looking into this and I think it's very useful, I'll try to add it soon.
There are a few things we can do:

remove a comment box selected by the user
add a new line in the middle of the comment box (assuming the user picked the same style)
detect that the current line is in the middle of a comment box and update it accordingly

IMO option 3 would be really nice, but it is really hard to implement and super easy to mess up, so I'll leave that for later. Meanwhile, I'll try to implement 1 and 2 and see how it works out.

I just released the new 2.1 update, with new commands to remove and update comment boxes, enjoy : )
For now you'll have to select all lines to update / remove a comment box - in the future I'll try to make it automatically detect where boxes start and end.


[Portuguese] Update tags in all terms

Signed-off-by: edsoncelio<EMAIL_ADDRESS>Describe your changes
Update tags in all terms localized in Portuguese (Brazil).
Related issue number or link (ex: resolves #issue-number)
https://github.com/cncf/glossary/pull/1583
Checklist before opening this PR (put x in the checkboxes)

[x] This PR does not contain plagiarism

don’t copy other people’s work unless you are quoting and contributing it to them.

[x] I have signed off on all commits

signing off (ex: git commit -s) is to affirm that commits comply DCO. If you are working locally, you could add an alias to your gitconfig by running git config --global alias.ci "commit -s".

PTAL @seokho-son @jihoon-seo

Closing due to the number of changes - I'll open a new one soon :D


mapFoldBack incomming list is not last parameter

In contrast to many other list, seq, array iteration functions, that always have the incoming collection as last parameter, allowing to pipe, List.mapFoldBack has acc as last parameter.
        let mapFold<'T,'State,'Result> (f:'State -> 'T -> 'Result * 'State) acc list =

        let mapFoldBack<'T,'State,'Result> (f:'T -> 'State -> 'Result * 'State) list acc =

I'd suggest to change mapFoldBack to
        let mapFoldBack<'T,'State,'Result> (f:'T -> 'State -> 'Result * 'State) acc list =

That would be a breaking change.

@vslynko Yes, this can't be changed. And it's the same as foldBack for example


cooperation

Here was the library where I started more than a year ago. But I realized today that it creates the same thing and has progressed a lot.
https://github.com/huntersofbook/naive-ui-nuxt
If you want to cooperate, I can deactivate my library or you can move to our side.
It's up to you. Having 2 identical libraries is not a good thing.

Thanks for this proposition.
The SSR support is already provided by naive-ui as explained in docs. This module provides extra features, you can check this issue (#9).
Thanks for your understanding.

I think you don't understand that there are 2 npm packages for the same topic. If you make a decision on what I wrote above, let's proceed accordingly.

These are 2 npm packages for this same topic with different goals and features. That's a good thing!
If you have ideas in mind to move this work forward, your contributions would be very appreciated!

No, it's not a good thing, people need to focus on one place.
I'm closing my project. I'll forward it to you.

I referred him to you.

people need to focus on one place

People choose the best for them, It's not up to us to decide. And please refer to this issue as well.


[BUG] Backup-target failing for the harvester-ci-testing bucket

Describe the bug
The backup-target in settings is failing when trying to use the aws account credentials provided for harvester-ci-testing bucket. It fails with error
backup-target Modified
Custom back target to store VM backups.
{
"type": "s3",
"endpoint": "",
"accessKeyId": "",
"secretAccessKey": "",
"bucketName": "harvester-ci-testing",
"bucketRegion": "us-east-2",
"cert": "",
"virtualHostedStyle": false
}
operation error S3: ListBuckets, https response error StatusCode: 403, RequestID: E0NEZH6VAXNEEWCT, HostID: lePh+/9xeaHXfS6WLYF6JHqfmk1NXfdd3BJDLZXLSBBxgqkkraaaYMOBwQF3SVXc7INtUJLY2BybQzLIj8bl+g==, api error AccessDenied: Access Denied
I performed the following steps on Longhorn UI and backup/restore seems to work there fine.

Created aws-secret on harvester-node-0 using kubectl command

Did settings on longhorn UI
Backup Target - s3://harvester-ci-testing@us-east-2/
Backup Target Credential Secret- aws-secret

Created a backup and restored and it is successful

To Reproduce
Steps to reproduce the behavior:
Try to modify the backup-target in settings with aws credentials of harvester-ci-testing bucket.
Expected behavior
backup-target should be created without any errors
Support bundle

Environment:

Harvester ISO version: master
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630): ipxe vagrant harvester setup

Additional context
Add any other context about the problem here.

It seems Harvester is performing a check that requires additional permission for S3.

The code
validateS3BackupTarget in backupTaget.go
tries to verify if the bucket is existing,
as per S3 API, HeadObject (longhorn uses) or HeadBucket is the proper API to use, it is high efficiency, and will not cause potential "AccessDenied",  harvester uses ListBucket.
HeadObject is a general API to check if a single object exists.
AWS has 3 APIs related here:
// * If you have the
// s3:ListBucket permission on the bucket, Amazon S3 returns an HTTP status code
// 404 ("no such key") error.
//
// * If you don’t have the s3:ListBucket permission,
// Amazon S3 returns an HTTP status code 403 ("access denied") error.

The HEAD action retrieves metadata from an object without returning the object itself. This action is useful if you're only interested in an object's metadata. To use HEAD, you must have READ access to the object.

A HEAD request has the same options as a GET action on an object. The response is identical to the GET response except that there is no response body. Because of this, if the HEAD request generates an error, it returns a generic 404 Not Found or 403 Forbidden code. It is not possible to retrieve the exact exception beyond these error codes.
func (c *Client) HeadObject(ctx context.Context, params *HeadObjectInput, optFns ...func(*Options)) (*HeadObjectOutput, error)

Returns a list of all buckets owned by the authenticated sender of the request.
func (c *Client) ListBuckets(ctx context.Context, params *ListBucketsInput, optFns ...func(*Options)) (*ListBucketsOutput, error)

// This action is useful to determine if a bucket exists and you have permission to
// access it. The action returns a 200 OK if the bucket exists and you have
// permission to access it. If the bucket does not exist or you do not have
// permission to access it, the HEAD request returns a generic 404 Not Found or 403
func (c *Client) HeadBucket(ctx context.Context, params *HeadBucketInput, optFns ...func(*Options)) (*HeadBucketOutput, error) {

Harvester usage:
pkg/controller/master/backup/backupTaget.go:293:        output, err := client.ListBuckets(h.ctx, &s3.ListBucketsInput{})

Longhorn usage:
vendor/github.com/longhorn/backupstore/s3/s3_service.go:102:func (s *Service) HeadObject(key string) (*s3.HeadObjectOutput, error) {
vendor/github.com/longhorn/backupstore/s3/s3_service.go:108:    params := &s3.HeadObjectInput{
vendor/github.com/longhorn/backupstore/s3/s3_service.go:112:    resp, err := svc.HeadObject(params)
vendor/github.com/longhorn/backupstore/s3/s3.go:150:    head, err := s.service.HeadObject(path)
vendor/github.com/longhorn/backupstore/s3/s3.go:159:    head, err := s.service.HeadObject(path)

Harvester current code:
func (h *TargetHandler) validateS3BackupTarget(target *settings.BackupTarget) error {
  ...
        // create a s3 service client
        client := s3.NewFromConfig(cfg)
        output, err := client.ListBuckets(h.ctx, &s3.ListBucketsInput{})
..
        for _, b := range output.Buckets {
                if *b.Name == target.BucketName {
                        return nil
                }
        }
..
}

test with minio, it said: failed to load custom CA bundle PEM file.   which step is missing ?
https://longhorn.io/docs/1.1.2/snapshots-and-backups/backup-and-restore/set-backup-target/

@w13915984028 The cert doesn't look like a PEM certificate file starting with -----BEGIN CERTIFICATE-----. Is it base64 encrypted?

With the used s3 harvester-ci-testing credentials, backup target setting is successful in my local harvester.

reproduced now
the "endpoint" field should be empty, then, the same error occurs:
operation error S3: ListBuckets, https response error StatusCode: 403, RequestID: MBWD5GVK5MS2NMY0, HostID: o7HZ6AXjotKv01pher5AByzwpwcpJP3Rpz8R46JQF56KR+WKoPrseJrYS5OykGSMETuqCXTBDiw=, api error AccessDenied: Access Denied

should be resolved in the latest v1.0.0-rc1 release of supporting both s3 and s3 compatible servers like MiniO.

After merge of #1447, both s3 and minio are working correctly.
#1447 refactor the validation of backup target, it is forwarded and delegated to longhorn and returns unified error message.

Verified it works as #1547 described.
As the Minio is compatible with AWS S3, it's testd on MInio environment rather than S3.
Test Information

Environment: qemu/KVM 3 nodes
Harvester Version: v1.0.0-rc1
ui-source Option: Auto

Verify Steps:

Install Harvester with any nodes
Setup Minio
a. then follow the instruction to create a single-bucket-user.
b. Create specific bucket for the user
c. Create other buckets
setup backup-target with the single-bucket-user permission
a. When assign the dedicated bucket (for the user), connection should success.
b. When assign other buckets, connection should failed with AccessDenied error message


Return an empty list when NetworkPolicyStats is disabled

Fixes #2214.  When NetworkPolicyStats is disabled an empty list is now returned rather than BadRequest.
In testing this change allowed a Sonobuoy test suite which used Flux to sync, rather than fail to sync.
Contrinuting step 5 says to update "Unreleased"; however, I did not find a section titled that in any of the changelog documents.
Signed-off-by: Peter Eltgroth<EMAIL_ADDRESS>

You may need to update pkg/apiserver/registry/stats/antreaclusternetworkpolicystats/rest.go and pkg/apiserver/registry/stats/antreanetworkpolicystats/rest.go as well, otherwise the other two APIs will still return the error if Flux discovers them.

I'll take a look today.

@PeterEltgroth thanks for the PR. It still missed "pkg/apiserver/registry/stats/antreaclusternetworkpolicystats/rest.go". They have to be 3 stats APIs because:

networkpolicystats is for K8s NetworkPolicy
antreanetworkpolicystats is for Antrea Namespace level NetworkPolicy
antreaclusternetworkpolicystats is for Antrea Cluster level NetworkPolicy

Codecov Report

Merging #2386 (7e9af0b) into main (c3b14ff) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #2386      +/-   ##
==========================================
- Coverage   42.03%   41.99%   -0.04%
==========================================
  Files         146      146
  Lines       18136    18127       -9
==========================================
- Hits         7623     7612      -11
- Misses       9827     9830       +3
+ Partials      686      685       -1

Flag
Coverage Δ

unit-tests
41.99% <100.00%> (-0.04%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...stry/stats/antreaclusternetworkpolicystats/rest.go
44.89% <100.00%> (ø)

...er/registry/stats/antreanetworkpolicystats/rest.go
45.28% <100.00%> (ø)

...piserver/registry/stats/networkpolicystats/rest.go
40.81% <100.00%> (ø)

pkg/agent/nodeportlocal/k8s/npl_controller.go
0.00% <0.00%> (ø)

...g/controller/networkpolicy/clusternetworkpolicy.go
60.91% <0.00%> (ø)

pkg/controller/grouping/group_entity_index.go
91.54% <0.00%> (+0.46%)
:arrow_up:

pkg/controller/traceflow/controller.go
66.26% <0.00%> (+1.20%)
:arrow_up:

pkg/apiserver/storage/ram/store.go
78.94% <0.00%> (+1.50%)
:arrow_up:

pkg/controller/grouping/controller.go
55.17% <0.00%> (+1.63%)
:arrow_up:

pkg/agent/nodeportlocal/npl_agent_init.go
70.83% <0.00%> (+10.11%)
:arrow_up:

/test-all

/skip-conformance
as https://jenkins.antrea-ci.rocks/job/antrea-conformance-for-pull-request/2411/console just failed to clean up after success validation


ROCm driver for Ubuntu 14.04.5 (Kernel 4.4) and Vega Frontier

Is the ROCm driver supported for Ubuntu 14.04.5 and Vega Frontier? We are trying to install in a university server and changing the operating system is tricky because the department won't support issues/problems for other OS. Is there any way to use the Vega Frontier GPU on such an operating system? We are trying to run compute workloads.

The guide says the GPU will be listed with the following two commands, but I get the following errors:
/opt/rocm/bin/rocminfo
/opt/rocm/bin/rocminfo: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version GLIBCXX_3.4.21' not found (required by /opt/rocm/bin/rocminfo) /opt/rocm/bin/rocminfo: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version GLIBCXX_3.4.20' not found (required by /opt/rocm/hsa/lib/libhsa-runtime64.so.1)
/opt/rocm/bin/rocminfo: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /opt/rocm/hsa/lib/libhsa-runtime64.so.1)
/opt/rocm/opencl/bin/x86_64/clinfo
terminate called after throwing an instance of 'cl::Error'
what():  clGetPlatformIDs
Aborted (core dumped)

The issue is that the runtime was compiled with a GCC of 3.4.20/21, and thus rocminfo can't work if there is a lower version of GCC installed (a GCC issue). Is it possible to upgrade the version of GCC on the system?

To be a bit more direct here: no, ROCm no longer supports Ubuntu 14.04. Our rock-dkms driver no longer supports kernel 4.4 and we do not plan to go back and have it do so. This kernel is also too old to support ROCm using the baseline kernel itself. As such, I do not expect ROCm to work on your Ubuntu 14.04 system.


Some issues about trying to send a basic shape with Marker

Hi , I have built the rviz in ros2 successfully .
But when i try to send a basic shape like what the tutorials do in ros1 , i have met some problems:

when i add a Grid , it shows a Status Error : Could not transform from [base_link] to [base_link]
(But i have set the Fixed Frame : base_link)

i write a node to publish Marker like ros1 , but the Marker shows a Status Error:
right : Topic : 23 messages received
right : Array Topic : OK
error : basic_shape/0
As i set my Marker like this :
marker = std::make_shared<visualization_msgs::msg::Marker>();

uint32_t shape = visualization_msgs::msg::Marker::CUBE;

marker->header.frame_id = "base_link";
//marker.header.stamp = rclcpp::Time::now();

marker->ns = "basic_shapes";
marker->id = 0;

marker->type = shape;

marker->action = visualization_msgs::msg::Marker::ADD;

marker->pose.position.x = 0;
marker->pose.position.y = 0;
marker->pose.position.z = 0;

marker->pose.orientation.x = 0.0;
marker->pose.orientation.y = 0.0;
marker->pose.orientation.z = 0.0;
marker->pose.orientation.w = 1.0;

marker->scale.x = 1.0;
marker->scale.y = 1.0;
marker->scale.z = 1.0;

marker->color.r = 0.0f;
marker->color.g = 1.0f;
marker->color.b = 0.0f;
marker->color.a = 1.0;

I guess it's something about marker->ns = "basic_shapes"; , but i dont't know how to set it
i'm wondering if there are some examples of rviz in ros2 publishing msgs that i can refer to
thanks !

Could you please publish a transform (any transform) from any frame to base_link and see whether everything clears up, e.g. in a terminal run ros2 run tf2_ros static_transform_publisher -- 0 0 0 0 0 0 map base_link?
There is a known issue with tf2 that it throws an error if it can't find a reference to a frame, even if no transform is necessary, see here: https://github.com/ros2/geometry2/issues/58

in a terminal run ros2 run tf2_ros static_transform_publisher -- 0 0 0 0 0 0 map base_link

This works !
the Marker is successfully shown now .
I'm trying to send image type then , thanks for your answer !


Using symlinks doesn't work on Windows and breaks puppet doc and geppetto

Windows doesn't support symlinks so any user trying to use rspec-puppet on Windows is simply out of luck.  Also since Java doesn't support symlinks either (mainly due to Windows lack of support for them) once you allow  rspec-puppet-init to create the symlinks from the fixtures directory to the top of the modules directory any user trying to work with Geppetto as their editor will be broken as well.  Especially if they keep their modules under source control which everyone should be doing.  Its no longer possible to use Geppeto for any git operations on my modules and its sees the symlinked directories as containing a separate copy of all of my manifests.
Also after using rspec-puppet-init to generate the layout in my module I'm no longer able to use puppet doc to generate rdoc documentation on any of my modules because it sees the classes defined multiple times.
$ puppet doc --outputdir /tmp/puppet/rdoc --mode rdoc --manifestdir ~/git/puppet/manifests --modulepath ~/git/puppet/modules

/usr/lib/ruby/1.8/rdoc/dot/dot.rb:28: warning: already initialized constant NODE_OPTS
/usr/lib/ruby/1.8/rdoc/dot/dot.rb:46: warning: already initialized constant EDGE_OPTS
/usr/lib/ruby/1.8/rdoc/dot/dot.rb:76: warning: already initialized constant GRAPH_OPTS
Could not generate documentation: Definition 'haproxy::proxy_farm' is already defined at /home/example/git/puppet/modules/haproxy/manifests/proxy_farm.pp:32; cannot be redefined at /home/example/git/puppet/modules/haproxy/spec/fixtures/modules/haproxy/manifests/proxy_farm.pp:32

I need to know what the best way to work around this bug is and another solution to the modulepath issue clearly needs to be found.  Why isn't it acceptable to include the parrent directory of the module in the module path?

Any work on this issue?  With cucumber puppet being dead and PuppetLabs giving more love to Windows lately, I think this should be bumped.
Are there any windows-friendly testing alternatives?

If you can't use symlinks, you could try copying everything into spec/fixtures/modules/#{modulename} in the 'setup' task instead of creating a symlink.

@Breeze773 @iljaweis We got around this in the puppetlabs_spec_helper by using the underlying puppet api to create the symlinks, it does however require puppet > 3.5.0
https://github.com/puppetlabs/puppetlabs_spec_helper/pull/87


DOCS-4623 proxysettings

Remerging to v1.3

@sundippatel - Somehow this got merged into v1.2 and not v1.3. Had to revert the previous change.


NETOBSERV-1214 & NETOBSERV-1399 Cluster name & zone features

Description

[x] Filter unecessary columns from operator
[x] Specify MultiClusterDeployment and AddZone states to frontend features array

Dependencies
https://github.com/netobserv/network-observability-console-plugin/pull/462
Checklist
If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

[X] Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
[ ] Does this PR require product documentation?

[ ] If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.

[ ] Does this PR require a product release notes entry?

[ ] If so, fill in "Release Note Text" in the JIRA.

[ ] Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.

[ ] If so, make sure it is described in the JIRA ticket.

QE requirements (check 1 from the list):

[X] Standard QE validation, with pre-merge tests unless stated otherwise.
[ ] Regression tests only (e.g. refactoring with no user-facing change).
[ ] No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

@jotak I rebased and addressed your comments. I'm updating the console plugin accordingly

/lgtm

@jpinsonneau I guess we can merge it?

@jpinsonneau I guess we can merge it?

I'm waiting for QE review before merging, just in case I need to update backports and so on.
Is it blocking us on something else ?

Resolved conflict https://github.com/netobserv/network-observability-operator/pull/546/commits/b1b8da0a309cb70360489a0918e410364280663f


Save settings on disk

When accessing the web ui from another server the settings (and columns) are not synchronized. It might be nice save them on disk so they are not tied to a browser.

Hey, I am also testing the use of logdy to replace an in-house unsupported service manager with json log reading that is used to run a bunch of microservices and view logs for them. The idea is to run microservices with regular cli tools instead and forward output to central logdy for viewing. The "big thing" to switch for is search and resizable columns, but that's beside the point here. For this to really be useful besides demoing, I'd need to be able to export the configuration and share it with another developer so they can also use it locally. The the JSON export/import from the UI would support this flow nicely as well as acting as a workaround for basically any other type of sharing.
As an after-thought, probably the easiest MVP to do for sharing from hosted (accessible by URL) logdy is to first allow choosing the server-side configuration from an exported JSON file when starting logdy and sending that to clients who connect. Then you can build on top of that with saving, loading, sending configurations on the fly as you wish and see need from other people. This of course means any changes by the user will be overwritten when they reconnect unless there's caching or versioning or they manually export and import on next load. But that's why I called it the easiest MVP 😄

@PeterOsinski Thank you for the excellent project. I came here to create an issue for this exact feature.
I wanted to share this with teammates but wanted them to avoid facing the trouble of setting up all the columns/facets. This should work.
Also, if logdy can support receiving this configuration via a CLI argument this can support more workflows.


Migrate to nodejs v18

Nodejs v16 active support ended a month ago and Backstage now requires v18, which the supported LTS now. We should migrate to ubi[8,9]/nodejs-18 based image.
This issue is now blocked (with potential workaround) by UBI based nodejs v18 images not being available without authentication. Nodejs v16 was available without auth, so this is a behavioral change on UBI side.
Related issue:
https://bugzilla.redhat.com/show_bug.cgi?id=2143885

BZ is resolved, this is unblocked now

Resolved via #9


publish to npm

This project already has a package.json file. Would it be possible to get this published to npm for easier integration with front-end build systems?

+1

I would also like to see this project published to npm.

Please, publish on npm!

I do not currently have time to maintain this project. I'd love to add some of you guys to the github repo so you can easily publish to npm if you will.
Anyone interested?
Sent from mobile

On Aug 26, 2016, at 15:52, Bruno Wego<EMAIL_ADDRESS>wrote:
Please, publish on npm!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@brunowego @pho3nixf1re @samstr @georgejenkins anyone interested?

I no longer use angular in any projects. Sorry :(


Isolate react-app package

This PR changes the package dependency hierarchy such that ui no longer depends on ui/react-app:
What was once this:
└── cmd/alertmanager/
    └── ui/
        ├── app
        └── react-app

Now becomes:
└── cmd/alertmanager/
    ├── ui/
    │   └── app
    └── react-app

i.e. the react-app endpoints are now registered separately.
This has a few benefits:

Isolates the problem described in https://github.com/prometheus/alertmanager/issues/3588. reactApp has custom build tooling, downstream projects of alertmanager can choose to independently omit this if they don't wish to run that toolchain to compile.
Due to Go's package tree-shaking, downstream projects can selectively choose which UI they want to include (or both, or neither). This makes the alertmanager dependency lighter.
Downstream projects like Mimir or Thanos which don't want to serve the react app quite yet can choose to register the routes independently.

There's one more thing I think we should do here - let's make sure we version control the react-generated, and we also need a CI step that aborts if there's a difference between what we have and what is generated.
Otherwise, we don't have visibility when changes are made.

This test has flaked out more than once: issue filed. https://github.com/prometheus/alertmanager/issues/3593
Another was CircleCI failing to talk to NPM to download a dependency. This one seemed like a one-off.

I don't mind the UI package not dragging along more dependencies than needed, but checking in the entire build output for the React app seems a bit unusual to me - it's almost like checking in a compiled binary, and now you'll have commit it again for every little change in the source. In Prometheus we circumvent this problem in a different way: the embed.go.tmpl file contains a Go build tag that only builds this file when the builtinassets Go build flag is set. This flag is set when building with promu (which we use for releases), but when someone just uses the package as a Go dependency and the flag is not set, it just pulls in an empty UI via https://github.com/prometheus/prometheus/blob/main/web/ui/ui.go (which is only built when that build tag is not set). I'd prefer if we could keep Prometheus and Alertmanager in sync in that respect.


fix(jira): change source to connection

Summary

For jira, changed all sourceId, jira_sources to connectionId, jira_connections
For other plugins, changed api path from /sources to /connections
Also changed source in front end

Key Points

[ ] This is a breaking change
[ ] New or existing documentation is updated

Description
Describe what this PR does, and aims to solve in a few sentences.
Does this close any open issues?
close #1780
Current Behavior
Describe the current behaviour of this issue, if relevant.
New Behavior
Describe the new behaviour updated in this issue, if relevant.
Screenshots
.

Other Information
Any other information that is important to this PR.

@e2corporation Hi, Julien, please note that source has been renamed to connection to avoid term-confusion. config-ui is also updated by this PR, you may want to take a look.


[llvm] libcxx, libcxxabi, libunwind features should not require [clang,tools] features

Is your feature request related to a problem? Please describe.
The C++ standard libraries and libunwind can be built independent of [clang,tools], unless a bootstrapping build is required. Also, building the latter takes much longer than building the C++ standard libraries, which is an unnecessary waste of time.
Proposed solution

Remove [clang,tools] feature dependencies from [libcxx,libcxxabi,libunwind] features
Add [libunwind] feature to [libcxx,libcxxabi]

Alternatively, provide additional [libcxx-bootstrap] and [libcxxabi-bootstrap] features which preserve current behaviour, and change the existing features to use the system/toolchain compiler.

Describe alternatives you've considered
No response
Additional context
Building LLVM: the default build mentions very clearly (emphasis mine):

The default way of building libc++, libc++abi and libunwind is to root the CMake invocation at <monorepo>/runtimes. While those projects are under the LLVM umbrella, they are different in nature from other build tools, so it makes sense to treat them as a separate set of entities. The default build can be achieved with the following CMake invocation:
$ git clone https://github.com/llvm/llvm-project.git
$ cd llvm-project
$ mkdir build
$ cmake -G Ninja -S runtimes -B build -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind" # Configure
$ ninja -C build cxx cxxabi unwind                         # Build
$ ninja -C build check-cxx check-cxxabi check-unwind       # Test
$ ninja -C build install-cxx install-cxxabi install-unwind # Install

Do not close.


fix: plugin page router

Description
fix unable to navigate to plugin page, introduced in #7330.
fixed by adding back route config in no skill case.

Task Item
refs #7330

Screenshots
when select on root bot

when select on skill bot

Hi @beyackle , can you help take a look on this ?

Coverage decreased (-0.001%) to 50.871% when pulling 1e1b89ede5bd209dc9c21afb5b512b34650092c5 on zhixzhan/navigation-fix into 3e019b185be00986ce2d936fb2b51d0059a929d1 on main.

I'm taking this to unblock, looks like it's the right fix.


Supported version of RHEL

According to Red Hat doc, RHEL supported version on Azure Stack is  "7.1 (and later)".
https://access.redhat.com/articles/3413531
The expression "7.x" gives the misconception that RHEL 7.0 is supported on Azure Stack.

@kojinak : Thanks for your contribution! The author, @sethmanheim, has been notified to review your proposed change.

@kojinak Thank you very much for catching that! We'll get the change merged right away.
#sign-off


LICENSE file still a template, doesn't contain copyright line

Source files have copyright IBM, but the LICENSE file at the root of the project has neither year nor copyright holder.
https://github.com/IBM-Swift/BlueSocket/blob/e2f6aee81b9e316f61e173e8350cb69d7714ead8/LICENSE#L189

The LICENSE file at the root of the project contains the Apache License...  Closing this issue as not valid.

@billabt Sorry if I failed to communicate the issue clearly. The issue is that the license at the root of the project contains the Apache license TEMPLATE. It is not complete. According to the Apache documentation, the line with "Copyright {yyyy} {name of copyright owner}" should be replaced with "Copyright 2016 IBM".
See the difference between what exists in the source files:
https://github.com/IBM-Swift/BlueSocket/blob/e2f6aee81b9e316f61e173e8350cb69d7714ead8/Sources/Socket/Socket.swift#L6
and what exists in the current template at the root of the project:
https://github.com/IBM-Swift/BlueSocket/blob/7061bd13ed7bf2292283e542287cdcb2cb71ad01/LICENSE#L189
A tangible benefit to this change is enabling use with tools such as https://github.com/mono0926/LicensePlist that integrate with Carthage/Cocoa pods to automatically include the LICENSE file as an acknowledgement in projects that use the library.

Fixed them ALL.  The other Blue* were missing it too.  Thanks.


[SPARK-42299] Assign name to _LEGACY_ERROR_TEMP_2206

What changes were proposed in this pull request?
The PR assigns a more descriptive name to the error class _LEGACY_ERROR_TEMP_2206.
Why are the changes needed?
This change improves the error framework by making the error name more descriptive.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Tested with existing test cases.

Can you trigger the error by not using internal API?

@asl3 There is the test in QueryExecutionErrorsSuite which uses a fake file system to trigger an error:
https://github.com/apache/spark/blob/2687d784fe4d20af321f11074139c0ce382bbaef/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryExecutionErrorsSuite.scala#L695-L700
Can you try to trigger the error from user space using a fake filesystem, please.

@MaxGekk
This is very Structured Streaming specific, so I'd guess it's not very straightforward to craft the test with fake filesystem. It requires understanding of how things are working. It's OK to file a new JIRA ticket for follow-up of this adding test. I (or my team) can take it over.

here is the JIRA ticket for creating a test: https://issues.apache.org/jira/browse/SPARK-43940

@asl3 Do you have an account at OSS JIRA? https://issues.apache.org/jira/browse/SPARK-42299
If not, please, submit a request, see
https://github.com/apache/spark/pull/41319#pullrequestreview-1445538528

+1, LGTM. Merging to master.
Thank you, @asl3 and @HeartSaVioR @allisonwang-db for review.

@asl3 Congratulations with your first contribution to Apache Spark!


Add package bson

Added package bson

My bad. I added two in a row and it ended up in the same PR. Will cancel and try again.


Allow worker pods to restart on failures

Restart failed worker Pods using the usual backoff mechanism.
/cc @yevgeny-shnaidman

/lgtm


Small panel refactors

Some small improvements following on from #31

Better ToolbarSection constructor
Turn commands into dataclasses

What does changing these commands into dataclasses do?

Dataclasses give us __eq__ for free. Normally, they also generate __init__ but there was an issue with the QUndoCommand base class.
But I figured out how to make it work (see latest commit), so now we don't have to write constructors for commands

Okay, is this fine for merging now?

Yes, should be good to go 👍


Feature request: Don't alert for scaled down nodes

I'm not sure if this already exists, but it would be nice if we didn't get "Host or monitoring unavailable" alerts for scaling down events.
An example alert:

But it was for a normal scaling down event:

The Operator already provides this functionality. If a node gets scaled down on purpose we send a mark for termination event to the Dynatrace cluster which prevents creating a problem for that specific host.

Ok, odd seems that didn't work in my case. I'll close this for now and open as a bug if the issue re-occurs.


Autorest Regen Preview to 3.0.0-alpha.20230607.5 by Chun Yu from refs/pull/3479/head

Triggered from https://github.com/Azure/autorest.csharp/pull/3479

API change check
API changes are not detected in this pull request.


C# examples are missing ConnectionStringSetting, thus are currently not working examples

C# examples on this page are missing the  `ConnectionStringSetting = <SIGNAL_R_CONNECTION_STRING> parameter that should be passed alongside HubName. Other language examples show that the connection string reference is required but C#'s examples do not.
For example, SendToGroup should be
[FunctionName("SendMessage")]
public static Task SendMessage(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post")]object message,
    [SignalR(ConnectionStringSetting = <SIGNAL_R_CONNECTION_STRING>, HubName = "chat")]IAsyncCollector<SignalRMessage> signalRMessages)
{
    return signalRMessages.AddAsync(
        new SignalRMessage
        {
            // the message will be sent to the group with this name
            GroupName = "myGroup",
            Target = "newMessage",
            Arguments = new [] { message }
        });
}

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b253c164-d46b-0193-b168-7eb024097bf6
Version Independent ID: 5858135e-528b-81ba-84a9-a89e8f165c36
Content: Azure Functions SignalR Service output binding
Content Source: articles/azure-functions/functions-bindings-signalr-service-output.md
Service: azure-functions
GitHub Login: @craigshoemaker
Microsoft Alias: cshoe

@alteredtastes Thank you for your feedback! We will review and update as appropriate.


npm not starting

It seems that is not starting:
npm ERR! Linux 4.10.0-38-generic
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "start"
npm ERR! node v4.2.6
npm ERR! npm  v3.5.2
npm ERR! code ELIFECYCLE
npm ERR<EMAIL_ADDRESS>start: "tsc && concurrently "tsc -w" "lite-server" "
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script 'tsc && concurrently "tsc -w" "lite-server" '.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the angular2-essential-training package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     tsc && concurrently "tsc -w" "lite-server"
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs angular2-essential-training
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls angular2-essential-training
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /home/mrzogs/3. Training/Angular2/angular2-essential-training/npm-debug.log

What  am I missing? by the way when I do : npm install it occurs me:
npm WARN optional Skipping failed optional dependency /chokidar/fsevents: npm WARN notsup Not compatible with your operating system or architecture<EMAIL_ADDRESS>npm WARN<EMAIL_ADDRESS>No license field.

Update node version


MediaRecorder demo is broken by string instead of dict options and mimetypes

The live version of the demo is currently broken on Firefox and Chrome I think there's a couple of things going on:

For the vp8 mime case a dict should be used here.
The mime types used should be similar to video/webm;codecs=vp8. The VP9 case is rejected on Chrome currently because a comma is used to separate the container from the codecs. The vp8 case should use the same style of string.

Thank you! I'll take a look.

Thanks! Related commits for posterity: 1, 2.

Ah — yes, thanks for adding links to the commits.


Explicitly embed CRDs for installation ONLY in the OKS

Embeds the CRDs for installation for the opentelemetry-kube-stack chart without handling upgrades for now. This brings us to parity with what the kube-prom-stack chart does. This also unblocks #1166. In the future we should investigate the approach @JaredTan95 describes here.

@TylerHelmuth I probably need to call this out explicitly, but this is for the world where the conversion webhook is fully unnecessary because we will only be applying v1beta1 CRs. We can do this because the chart is also creating the collectors.


Update OSA SHA to tip of stable/mitaka

This commit updates the OpenStack-Ansible submodule to pull in the
latest changes.
This update addresses two known issues:

the Percona signing key for Apt packages has changed.

Connected https://github.com/rcbops/rpc-openstack/issues/1515
(cherry picked from commit 35d069492eb1fcaa388d3167f89d7ffc00e1975e)

👍

👍


Django-shop elasticsearch bug "Unexpected queryset model"

Hi,
i am on Debian 10 buster, python 3.7
I installed django-shop 1.2.4 demo and it works quite well.
however a bug related to the elasticsearch_dsl library occurs when I use the site search module:
TypeError at /en/search/
Unexpected queryset model (should be: <class 'myshop.models.Product'>, got: <class 'myshop.models.Product'>)
Request Method:  GET
Request URL: http://www.cris-shiatsu.fr:9009/en/search/?q=test
Django Version: 3.0.14
Exception Tyoe: TypeError
Exception Value: Unexpected queryset model (should be: <class 'myshop.models.Product'>, got: <class 'myshop.models.Product'>)
Exception Location: /usr/local/lib/python3.7/site-packages/django_elasticsearch_dsl/search.py in filter_queryset, line 26
/usr/local/bin/uwsgi
Python version: 3.7.17
This bug refers to the following code in search.py of the django-elasticsearch-dsl library:
def filter_queryset(self, queryset, keep_search_order=True):
        """
        Filter an existing django queryset using the elasticsearch result.
        It costs a query to the sql db.
        """
        s = self
        if s._model is not queryset.model:
            raise TypeError(
                'Unexpected queryset model '
                '(should be: %s, got: %s)' % (s._model, queryset.model)
            )

For the moment I commented out the error throwing condition and it works but i would like to understand and eventually resolve the problem without hacking the library
Please would you have an idea ?

Without more context, I can't name any reason.
Did I write the code you have shown?

sorry... i probably should ask to the library writer

sorry... i probably should ask to the library writer

I had the same problem, and I sent the solution to the library uploader, you can see it here:
https://github.com/django-es/django-elasticsearch-dsl/pull/464


Feature Request: Global Hotkey

VSCode Version: 1.19.1
OS Version: 10.13.1

It would be nice to have the ability to toggle VSCode with a global system-wide hotkey.
Example:

I don't quite understand what you are asking for.
On macOS I can use Command-H to hide all VS Code windows.
Where does the screenshot from above originate from?

Ah yes, I'm currently doing that too. I use cmd + h to hide then cmd + tab to bring it back.
Though I still think it would be beneficial to toggle VSCode with a hotkey. So I can go between multiple apps and switch back to VSCode easily.
I use a couple of applications that allow for this option.
That screenshot above is from iTerm.
Todoist:

Bear App:

@epilande I am not sure what VS Code can do here, isn't that something that should be configured per OS using OS mechanisms?

@epilande for this kind of things I would rather configure something like phoenix.

@bpasero @fabiospampinato I've tried setting something up with MacOS but had no luck. I've resulted in using a third party application to achieve this, similar to Phoenix. Though it would be nice if I didn't have to open a different application and if VSCode can support this as a configurable option.
This is really just a workflow thing. Not everyone will use this option but this is something that helps boost my productivity and how I develop applications. My example workflow is, I work on my laptop a lot (small screen), so I like to have my applications take up the full screen (but not in fullscreen mode, the animation takes too long and lose context..😕). For example, my editor, chrome/firefox, slack, notes, etc. then toggle between applications by showing/hiding my editor. So having a global hotkey to show/hide VSCode would be extremely helpful.
The point of this is to remove friction from switching between apps. I know there's cmd + tab, but what happens when a user switches between 3 or 4 apps, they'll have to hit cmd + tab multiple times. Another example is using cmd + h to hide and using Spotlight or Alfred to reopen, this can be simplified with a global hotkey to toggle show/hide.
Nonetheless, great job with VSCode! I've completely switched from iterm + vim + tmux to VSCode. Thanks for a solid application 👌

@epilande I am trying find solution for this show/hide operation as well, so I am wondering what kinds of solution you end up to. Since there do not have native support from VSCode, a 3rd party application would also be appreciated.😀

@nylqd I'm currently using BetterTouchTool, it's a paid product but it's pretty powerful, it lets you remap your keys, mouse, touchbar, applications, etc..

@nylqd yep, that's a good approach for this feature.

@epilande Unfortunately this issue is closed, still,  thank you for ur recommendation, much appreciated.


support for React 18.2

Maybe it possible to fix this error:
npm WARN Could not resolve dependency:
npm WARN peer<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm WARN node_modules/@dantecoder/react-resizablebox
npm WARN<EMAIL_ADDRESS>from the root project

If you wish i can try to fix it by myself

@DanteCoder
If you wish i can try to fix it by myself

Yeah, please try updating the peer dependency of React. And thank you.

Plz give me access to push updated package.json & package-lock.json or give me your email (to send this files to you)

@g5-freemen I think you can fork this repository and then make a pull request to this repo.
https://github.com/gabrieldemarmiesse/getting_started_open_source

@DanteCoder made PR


Support replacement of commands for custom recipes

Currently custom recipes will not be able to take advantage of the automatic set up bob does for the library list, retrieving of the job log, etc. We should support being able to replace the default command with a custom command.
For reference: https://github.com/IBM/vscode-ibmi-projectexplorer/issues/583

I'm also very interested in this enhancement.


Add some example screenshots in the github readme

It would be nice if you added some pictures so that users can see how the module looks like before installing it.  In particular, it would be nice to see differences between certain modes, and how grayscale vision looks like.

yes please

I finally added some screenshots and improved the readme a lot in general.


Fix/improve Class Code handling in signup form

This does two things:

Fix a regression where signing up from /courses with a class code would not really join the class, because application.router.navigate doesn't work if the path is the same but the query parameter is different.
Autofill the class code in the signup form if you get there from a classroom join link (/courses?_cc=...)

👍


[zh-cn] Sync pod-lifecycle.md

/approve

Currently, there is an issue with the Netlify CI tool.
I would suggest using the following command to trigger a Netlify rebuild, which has the advantage of not losing the lgtm label and reducing the duplication of code views:
git reset
git commit --amend --no-edit
git push --force-with-lease

/lgtm


TO: Add 'Geolimit Redirect URL' parameter in Delivery service

Codes for #1518

Can you please fix the conflicts?

I ran the tests and it looks like a couple failed, can you please fix?

Test Summary Report
t/api/1.0/data.t                        (Wstat: 512 Tests: 88 Failed: 2)
Failed tests:  9, 11
Non-zero exit status: 2
t/deliveryservice.t                     (Wstat: 11264 Tests: 72 Failed: 44)
Failed tests:  8, 11, 14, 16, 18, 20, 22, 25-33, 35, 37
40-64, 67
Non-zero exit status: 44

When I tried to edit an existing DS I get this error

Invalid parameter 'ds.geolimit_redirect_url': cannot use 'geolimit_redirect_url' on a Schema::Result::Deliveryservice at template delivery_service/_form.html.ep line 209.

Can you run db/admin.pl reverse_schema from traffic_ops/app to update the schema file?   You will then need to add the schema file to your PR.

Hi David,
Sorry for the late updates.  I have fixed all the issues you mentioned above.   By the way, after I ran command: db/admin.pl reverse_schema ,  there were some modifications in schema file which was not related to 'geolimit_redirect_url' param, so I removed it manually.  Please let me know if there is anything else I need to update.
Thanks!
Leo Z

Great, thanks!  I will take a look when I get a chance.

A few small things:

Can you update the traffic ops documents to include the new field?  http://traffic-control-cdn.net/docs/latest/admin/traffic_ops_using.html#delivery-service
Can you update the API and docs to include the new field?  https://github.com/Comcast/traffic_control/blob/master/traffic_ops/app/lib/API/DeliveryService.pm
If I put an invalid URL into the field (for example:  thisisnotavalidurl), it accepts it.  Can you double check your validation?  https://github.com/Comcast/traffic_control/pull/1519/files#diff-dd2f89cbcb46d2fa1240f754fb07682bL523

Hi @dneuman64 ,
I have committed codes for the issues 1 & 2.     As for issue 3,  it should be legal to input that kind of string in the field, for more detailed information, please refer to: http://traffic-control-cdn.net/docs/latest/admin/traffic_router.html#geolimit-failure-redirect-feature

@xiaodzha thanks for making the changes.
👍


Changing the Queue Library to Laravel

Currently we use Zend queue for all of our queue related shenanigans. All it really supports is FIFO and a few adapters (that I don't thing we expose in any useful way).
I would like to propose moving to Laravel Queues. (depending on what PHP version we end up going with, either http://laravel.com/docs/4.2/queues or http://laravel.com/docs/5.0/queues).
There are several reasons for this,
In our current queue, if something fails, it fails permanently, and we don't tell anyone. Laravel allows jobs to retry, and if it goes past a set number of failures it goes into a failed_jobs table. This is especially important with emails.
Zend queue does support adapters, but Laravel supports more, and more is better... right?
I really want to be able to allow users to choose their adapters, like Amazon SQS. By default, it would default to a database queue like we have now.
I know that this may add some dependencies, but I think it would be worth it.
Once we have a stable queue system, I would propose moving certain things, like sending emails, or other things that could be long-running (like file operations).
Thoughts?

Can you list out the dependencies this adds and what we could get out of
them? Also how baked in is eloquent? Can we use this with doctrine
repositories?
On Mon, Nov 2, 2015 at 11:52 AM Michael Krasnow<EMAIL_ADDRESS>wrote:

Currently we use Zend queue for all of our queue related shenanigans. All
it really supports is FIFO and a few adapters (that I don't thing we expose
in any useful way).
I would like to propose moving to Laravel Queues. (depending on what PHP
version we end up going with, either http://laravel.com/docs/4.2/queues
or http://laravel.com/docs/5.0/queues).
There are several reasons for this,
In our current queue, if something fails, it fails permanently, and we
don't tell anyone. Laravel allows jobs to retry, and if it goes past a set
number of failures it goes into a failed_jobs table. This is especially
important with emails.
Zend queue does support adapters, but Laravel supports more, and more is
better... right?
I really want to be able to allow users to choose their adapters, like
Amazon SQS. By default, it would default to a database queue like we have
now.
I know that this may add some dependencies, but I think it would be worth
it.
Once we have a stable queue system, I would propose moving certain things,
like sending emails, or other things that could be long-running (like file
operations).
Thoughts?
—
Reply to this email directly or view it on GitHub
https://github.com/concrete5/concrete5/issues/3087.

Additional requirements:

Backwards compatibility with the current approach.
Complete compatibility with the current queue progress-bar UI (used in
the sitemap, jobs, etc...)
Complete compatibility with QueueableJob and the new Queued Processor
functionality (introduced in 5.7.5)

On Mon, Nov 2, 2015 at 11:54 AM, Korvin Szanto<EMAIL_ADDRESS>wrote:

Can you list out the dependencies this adds and what we could get out of
them? Also how baked in is eloquent? Can we use this with doctrine
repositories?
On Mon, Nov 2, 2015 at 11:52 AM Michael Krasnow<EMAIL_ADDRESS>wrote:

Currently we use Zend queue for all of our queue related shenanigans. All
it really supports is FIFO and a few adapters (that I don't thing we
expose
in any useful way).
I would like to propose moving to Laravel Queues. (depending on what PHP
version we end up going with, either http://laravel.com/docs/4.2/queues
or http://laravel.com/docs/5.0/queues).
There are several reasons for this,
In our current queue, if something fails, it fails permanently, and we
don't tell anyone. Laravel allows jobs to retry, and if it goes past a
set
number of failures it goes into a failed_jobs table. This is especially
important with emails.
Zend queue does support adapters, but Laravel supports more, and more is
better... right?
I really want to be able to allow users to choose their adapters, like
Amazon SQS. By default, it would default to a database queue like we have
now.
I know that this may add some dependencies, but I think it would be worth
it.
Once we have a stable queue system, I would propose moving certain
things,
like sending emails, or other things that could be long-running (like
file
operations).
Thoughts?
—
Reply to this email directly or view it on GitHub
https://github.com/concrete5/concrete5/issues/3087.

—
Reply to this email directly or view it on GitHub
https://github.com/concrete5/concrete5/issues/3087#issuecomment-153139212
.

Dependencies are actually pretty minimal, pulled from laravel 5.1:
        "php": ">=5.5.9",
        "illuminate/console": "5.2.*",
        "illuminate/contracts": "5.2.*",
        "illuminate/container": "5.2.*",
        "illuminate/http": "5.2.*",
        "illuminate/support": "5.2.*",
        "symfony/process": "2.8.*|3.0.*",
        "nesbot/carbon": "~1.20"

If we want things like beanstalkd or SQS then they have their own dependencies also.
Eloquent models are optional, they actually suggest passing in things like IDs and re-building models by the IDs on the fly which I think would be better with Doctrine entities than passing the actual magical doctrine entity. I did some searching, there isn't anything to directly support doctrine that I could find. We don't support this now do we?
Backwards compatibility is going to be a little hard because we type hinted everything with Zend, we could do something along the lines of if a Zend_Queue/Message is passed in that we would use the old zend stuff, do some fancy reflection and instanceof checks.
Most of it is just aliasing function names (atleast thats what it looks like).
As for the progress bar UI, doesn't that just need count(rows), batch_size, and well, if the batch executed? If so, that isn't hard to support.
For the 2 other classes, I don't see anything special besides serializing/unserializing objects, which laravel supports out of the box, but as I mentioned above about the doctrine stuff, they recommend passing IDs instead of full objects in most cases.

@Mnkras not sure about 2 (progess bar). It's easy if you just add some polling to determine the current progress, but you'd have to handle errors was well. The laravel queue is truly async and not just a "chunk processor" and thus won't tell you if something has failed, that you'd have to take care too.
Yes, you only push primitive data types into the queue and then do whatever you like, whether that's doctrine or eloquent or text file handling.. Serializing those humongous doctrine objects will cause problems, even happend to me with eloquent.

Good point @Remo, So I guess for the progress bars we just poll, and check the failed_jobs table at the end?

@Mnkras I'd check the failed jobs with every callback, if all fail we don't really have an end, but basically yes.. I just wonder if a truly async process works well there. Sending a "delete pages" job to the queue can take a while till it's executed. Under heavy load you'll have to wait a bit and there won't be any progress at the beginning. It will only happen to busy sites, but it would be confusing, wouldn't it? While I really like those laravel queues, when deleting a lot of objects in the UI (for example), I feel that a "chunk processing system" might be more appropriate.

We could always use both, we will have to keep Zend for backwards compatibility anyway...
And problems with not deprecating Zend and having both of them be available?

I thought about that too, but concrete5 is already pretty heavy.. Maybe we could create a package for that and simply override the mail helper? Thanks to the core bindings this should be farily straight forward. I assume that only very few concrete5 users are willing to install something like beanstalk and those who are, are most likely capable of installing a package.

That is very true, but I foresee an async queue being used for more than just email, as an example, moving files from one storage location to another en-masse is not possible right now, with our current chucked queue, if 1 file fails, the queue will not go further no matter how many times you run it.
I have 2 reasons why I was looking into having an async queue along with failure/retry tracking,

The email server stopped responding, which killed page-loading on any page that tried to send an email.
When c5 fails to send an email, it fails permanently with no way to resend/retry. (This bit me when an email acct password was changed, had a few days of emails that I couldn't re-send and had to go through c5's log table >.<)

We could make it as a package, but I feel the benefit of having it in the core would out-weigh the extra weight added. Also, having support for enterprisely features out of the box can be a nice incentive for c5.

That is very true, but I foresee an async queue being used for more than just email, as an example, moving files from one storage location to another en-masse is not possible right now, with our current chucked queue, if 1 file fails, the queue will not go further no matter how many times you run it.

Agreed

I have 2 reasons why I was looking into having an async queue along with failure/retry tracking,

The email server stopped responding, which killed page-loading on any page that tried to send an email.

That part could be handled by a package, no big downside here.

When c5 fails to send an email, it fails permanently with no way to resend/retry. (This bit me when an email acct password was changed, had a few days of emails that I couldn't re-send and had to go through c5's log table >.<)

That part could be handled by a package, no big downside here.

We could make it as a package, but I feel the benefit of having it in the core would out-weigh the extra weight added. Also, having support for enterprisely features out of the box can be a nice incentive for c5.

I agree, enterprise level sites will require such functionality pretty quickly. I'd say more than half of my laravel projects use beanstalk.
I guess someone will have to make a decision here and that's probably not me ;-)

@aembler I think this is something we should revisit.

Maybe not a straight replacement with laravel queue, but we're still on a forked (read unsupported) zend queue repo and we're slowly finding that zend queue might not have all the features we're looking for.
Lets target 8.1 with the ability to swap out the underlying queue system so that a user can easily swap it out on their own then replace the queue in concrete5 completely with a better more supported queue library in 9.0.

I don't think the Queue library needs to be swappable. I do think that that queue system should support adapters, like Zend Queue does, so that you could swap out a MySQL implementation of a queue with something else like Redis, SQS, etc... Maybe this is the same thing, but I wanted it to be clear.
I'm fine with switching away from Zend Queue, it does lack features, and a nicer syntax for easily adding somethign to the queue would be cool, a la Laravel. That being said, the most important part of this functionality is feature parity with what we have today, which lets us queue up lists of page deletions, page duplications, file rescans, etc... Without the ability to have this work from the UI this is a non-starter. So it'll have to have feature parity with what we hvae today.

@aembler I want it swappable so that we can have the feature sooner than later. Replacing the zend queue is probably a no-go in a minor release unless we keep the vendor around.

In version 9 we have moved the Bernard queue library with our own custom queuing layer on top. It supports batchable commands (which are the commands you currently see in the core queueing stuff, with the progress bar) as well as a lower-level queue command that can simply be offloaded into a queue and return instantly.