DataFrame indexing for multi-task problems

I'm guessing for a multi-task problem, BayBE concatenates the df such that some of the original indices are lost. See the reproducer below, which gives a recommended df with an index of 40, even though the highest index for lookup_test_task is 26. Note that SMOKE_TEST environment variable was set to true when I ran this. Is this intended behavior?
If I dig further into the stack trace of simulate_scenarios, I'm guessing I'd find some handling of that, but it wasn't immediately obvious to me.
import os

# import sys
# from pathlib import Path

import numpy as np
import pandas as pd

# import seaborn as sns
from botorch.test_functions.synthetic import Hartmann

from baybe import Campaign
from baybe.objectives import SingleTargetObjective
from baybe.parameters import NumericalDiscreteParameter, TaskParameter
from baybe.searchspace import SearchSpace
from baybe.targets import NumericalTarget
from baybe.utils.botorch_wrapper import botorch_function_wrapper

from baybe.utils.random import set_random_seed

set_random_seed(42)

# Configuration
SMOKE_TEST = "SMOKE_TEST" in os.environ
DIMENSION = 3
BATCH_SIZE = 1
N_MC_ITERATIONS = 2 if SMOKE_TEST else 50
N_DOE_ITERATIONS = 2 if SMOKE_TEST else 10
POINTS_PER_DIM = 3 if SMOKE_TEST else 5
NUM_INIT = 2 if SMOKE_TEST else 5

# Define objective and search space
objective = SingleTargetObjective(target=NumericalTarget(name="Target", mode="MIN"))
BOUNDS = Hartmann(dim=DIMENSION).bounds

discrete_params = [
    NumericalDiscreteParameter(
        name=f"x{d}",
        values=np.linspace(lower, upper, POINTS_PER_DIM),
    )
    for d, (lower, upper) in enumerate(BOUNDS.T)
]

task_param = TaskParameter(
    name="Function",
    values=["Test_Function", "Training_Function"],
    active_values=["Test_Function"],
)

parameters = [*discrete_params, task_param]
searchspace = SearchSpace.from_product(parameters=parameters)

# Define test functions
test_functions = {
    "Test_Function": botorch_function_wrapper(Hartmann(dim=DIMENSION)),
    "Training_Function": botorch_function_wrapper(
        Hartmann(dim=DIMENSION, negate=True, noise_std=0.15)
    ),
}

# Generate lookup tables
grid = np.meshgrid(*[p.values for p in discrete_params])
lookups = {}
for function_name, function in test_functions.items():
    lookup = pd.DataFrame({f"x{d}": grid_d.ravel() for d, grid_d in enumerate(grid)})
    lookup["Target"] = lookup.apply(function, axis=1)
    lookup["Function"] = function_name
    lookups[function_name] = lookup
lookup_training_task = lookups["Training_Function"]
lookup_test_task = lookups["Test_Function"]

# Perform the transfer learning campaign
campaign = Campaign(searchspace=searchspace, objective=objective)
initial_data = lookup_training_task.sample(n=NUM_INIT)

campaign.add_measurements(initial_data)

df = campaign.recommend(batch_size=BATCH_SIZE)

# print(df)
#         x0   x1   x2       Function
# index
# 40     1.0  0.0  1.0  Test_Function

# NOTE: Indices other than the ones from df are ignored
df["Target"] = lookup_test_task.iloc[df.index]["Target"]

campaign.add_measurements(df)

# NOTE: with SMOKE_TEST and 42 as seed:
# print(lookup_test_task)
#      x0   x1   x2    Target       Function
# 0   0.0  0.0  0.0 -0.067974  Test_Function
# 1   0.0  0.0  0.5 -0.136461  Test_Function
# 2   0.0  0.0  1.0 -0.091332  Test_Function
# 3   0.5  0.0  0.0 -0.097108  Test_Function
# 4   0.5  0.0  0.5 -0.185407  Test_Function
# 5   0.5  0.0  1.0 -0.090204  Test_Function
# 6   1.0  0.0  0.0 -0.030955  Test_Function
# 7   1.0  0.0  0.5 -0.072904  Test_Function
# 8   1.0  0.0  1.0 -0.084769  Test_Function
# 9   0.0  0.5  0.0 -0.018048  Test_Function
# 10  0.0  0.5  0.5 -0.839061  Test_Function
# 11  0.0  0.5  1.0 -1.994263  Test_Function
# 12  0.5  0.5  0.0 -0.025729  Test_Function
# 13  0.5  0.5  0.5 -0.628022  Test_Function
# 14  0.5  0.5  1.0 -1.957039  Test_Function
# 15  1.0  0.5  0.0 -0.008194  Test_Function
# 16  1.0  0.5  0.5 -0.225915  Test_Function
# 17  1.0  0.5  1.0 -1.826650  Test_Function
# 18  0.0  1.0  0.0 -0.000274  Test_Function
# 19  0.0  1.0  0.5 -2.262308  Test_Function
# 20  0.0  1.0  1.0 -0.334829  Test_Function
# 21  0.5  1.0  0.0 -0.000204  Test_Function
# 22  0.5  1.0  0.5 -1.485659  Test_Function
# 23  0.5  1.0  1.0 -0.325958  Test_Function
# 24  1.0  1.0  0.0 -0.000038  Test_Function
# 25  1.0  1.0  0.5 -0.224631  Test_Function
# 26  1.0  1.0  1.0 -0.300476  Test_Function

Exception has occurred: IndexError       (note: full exception trace is shown but execution is paused at: _run_module_as_main)
positional indexers are out-of-bounds
  File "C:\Users\sterg\miniforge3\envs\baybe\Lib\site-packages\pandas\core\indexing.py", line 1714, in _get_list_axis
    return self.obj._take_with_is_copy(key, axis=axis)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\sterg\miniforge3\envs\baybe\Lib\site-packages\pandas\core\generic.py", line 4153, in _take_with_is_copy
    result = self.take(indices=indices, axis=axis)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\sterg\miniforge3\envs\baybe\Lib\site-packages\pandas\core\generic.py", line 4133, in take
    new_data = self._mgr.take(
               ^^^^^^^^^^^^^^^
  File "C:\Users\sterg\miniforge3\envs\baybe\Lib\site-packages\pandas\core\internals\managers.py", line 891, in take
    indexer = maybe_convert_indices(indexer, n, verify=verify)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\sterg\miniforge3\envs\baybe\Lib\site-packages\pandas\core\indexers\utils.py", line 282, in maybe_convert_indices
    raise IndexError("indices are out-of-bounds")
IndexError: indices are out-of-bounds

The above exception was the direct cause of the following exception:

  File "C:\Users\sterg\miniforge3\envs\baybe\Lib\site-packages\pandas\core\indexing.py", line 1717, in _get_list_axis
    raise IndexError("positional indexers are out-of-bounds") from err
  File "C:\Users\sterg\miniforge3\envs\baybe\Lib\site-packages\pandas\core\indexing.py", line 1743, in _getitem_axis
    return self._get_list_axis(key, axis=axis)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\sterg\miniforge3\envs\baybe\Lib\site-packages\pandas\core\indexing.py", line 1191, in __getitem__
    return self._getitem_axis(maybe_callable, axis=axis)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\sterg\Documents\GitHub\sgbaird\baybe\examples\Transfer_Learning\index_reproducer.py", line 81, in <module>
    df["Target"] = lookup_test_task.iloc[df.index]["Target"]
                   ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "C:\Users\sterg\miniforge3\envs\baybe\Lib\runpy.py", line 88, in _run_code
    exec(code, run_globals)
  File "C:\Users\sterg\miniforge3\envs\baybe\Lib\runpy.py", line 198, in _run_module_as_main (Current frame)
    return _run_code(code, main_globals, None,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
IndexError: positional indexers are out-of-bounds

xref: https://github.com/emdgroup/baybe/discussions/257 and https://github.com/emdgroup/baybe/discussions/283
NOTE: baybe==0.9.1.post209, Windows 11

Here is my workaround:
parameter_names = [p.name for p in searchspace.parameters]
obj_name = "Target"

full_lookup = pd.concat([lookup_training_task, lookup_test_task], ignore_index=True)

df = campaign.recommend(batch_size=BATCH_SIZE)

# update the dataframe with the target value(s)
df = pd.merge(
    df, full_lookup[parameter_names + [obj_name]], on=parameter_names, how="left"
)

Hi @sgbaird, I think it could be that you are mixing up unrelated things here. Note: you lookup_test_task merely holds the lookup data to close the DOE loop – it exists completely independently of your campaign and does not enter it in any way. They are only connected in the sense that you can use it to look up values that your campaign recommends, but the campaign isn't even aware that this objects exists and could in fact do its work without it. Thus, it's impossible that the recommendations of the campaign can have anything in common with the lookup, e.g. there is no reason to even assume that they would share indices or similar.
The indices you see returned by the campaign refer to the dataframe that is internally created to represent the discrete search space of the problem. But that is a completely arbitrary choice. In fact, I would even argue that the indices could be ignored entirely. We simply used the search space indices because we have to use an index for pandas DataFrames, and this at least gives us a reference to which search space elements have been recommended (compared to the alternative were we would simply start enumerating from 1).
Does it answer your question?

Ah, got it. So my "workaround" above is actually the correct way to do it - look for a matching configuration.
I guess part of the confusion is that the lookup tables were created based on the allowed search space values, but I see that is an arbitrary choice for this example.
Let's take the case where the training data is sampled within a continuous search space, and there isn't a particular pattern to the parameter sets. As a concrete example for a 1D problem.
training_data = [{"x": 0.43, "y": 1.86}, ... {"x": 0.78, "y": 2.3}]

and we say that the test function can only be sampled at x = [0, 0.5, 1.0], what is the correct way to set this up with BayBE's API?

Setup
import pandas as pd
from pytest import Cache

from baybe.campaign import Campaign
from baybe.parameters import NumericalDiscreteParameter
from baybe.parameters.numerical import NumericalContinuousParameter
from baybe.recommenders.pure.bayesian.botorch import BotorchRecommender
from baybe.searchspace.core import SearchSpace
from baybe.targets.numerical import NumericalTarget

parameters = [
    NumericalDiscreteParameter("x", [0.0, 0.5, 1.0]),
    NumericalContinuousParameter("y", (0, 1)),
]
searchspace = SearchSpace.from_product(parameters)
objective = NumericalTarget("t", mode="MAX").to_objective()
recommender = BotorchRecommender()
measurements = pd.DataFrame.from_records(
    [
        {"x": 0.43, "y": 0.55, "t": 1.86},
        {"x": 0.78, "y": 0.98, "t": 2.3},
    ]
)

Using Recommender Directly
rec = recommender.recommend(5, searchspace, objective, measurements)
print(rec)

Via Campaign
campaign = Campaign(searchspace, objective, recommender)
campaign.add_measurements(measurements, numerical_measurements_must_be_within_tolerance=False)
rec = campaign.recommend(5)
print(rec)

Note: For the latter, you currently need to explicitly specify the numerical_measurements_must_be_within_tolerance flag, since your measurements strictly speaking lie outside the range of the parameter you specified. However, we are currently still working on this interface, though, exactly because the behavior is not yet perfectly consistent between the two approaches and because the "tolerance" logic needs to be revised in general. #workinprogress

Hi @sgbaird, note that I've just updated the imports in my code above (which were a bit messy in the original version). Other than that, everything remains the same. I'll close the issue now, but feel free to reopen if further discussion is necessary ✌🏼


request : add npc by NpcID

would be easier to add mobs from wowhead or MDT (and in french client some mobs have male/female names on the same ID)
anyway, thanks for your work, very nice addon !

The issue with NPC ID is they are only relevant to UnitGUID , which requires a target to have been seen by the player already. This makes the configuration panel harder to setup for users.
I do recognize an issue with being able to share profiles, so I will be working on an import/export function for targets to make them easier to share.
I'll also be adding buttons to quickly load important S4 and Dragonflight S1 targets in an upcoming build.

I learned how to do this recently and will include npc id in the next build.

awesome, ty !

the latest beta build has this fix. unfortunately i had some hardware issues over the weekend so I haven't been able to fully commit to pushing a Release build.
Type in an NPC ID and if the NPC exists, it should convert to name and add it to your filter.

thank you very much, will make things a lot easier for me

1.2.5


HTML Profile report keeps crashing browser and jupyter

Current Behaviour
I reduced the size as much as possible to only do 1000 rows and 118 columns. It still takes 500 seconds to generate a report and crashed the jupyter notebook.  I remember running much larger dataframes quite quickly. So i saved it as a html file.  Now the jupyter notebook tab is fine(as long as i don't open it as a notebook iframe) , but anytime I open the html report it crashes that tab.
Tested it on both  (Mac, Windows) on (Chrome, Vivaldi) browsers. Once as a generated file and once as a downloaded file to avoid any cache issues. The html file crashes on any browser or any device 5 seconds after opening.
Expected Behaviour
Report loads for following the data analysis and doesn't crash the browser with an "Aw, Snap ! Something went wrong while displaying the web page" error
Data Description
Dataset shape is (1000,118) with dtypes: float32(20), int16(44), int64(1), int8(51), object(2)). No missing values
Code that reproduces the bug
No response
pandas-profiling version
v3.2.0
Dependencies
pandas==1.3.4
numpy==1.20.3

OS
Windows 11 ,  Mac OS BigSur v11.6.7
Checklist

[X] There is not yet another bug report for this issue in the issue tracker
[X] The problem is reproducible from this bug report. This guide can help to craft a minimal bug report.
[X] The issue has not been resolved by the entries listed under Common Issues.

UPDATE:

I have used minimal on the complete data(980k,118) and that runs and generates a report that loads without crashing
I removed the samples, missing, duplicates etc., by setting them to None
I ran a correlation report on a larger sample (98k, 118)  and it generated a report just for those which doesn't crash the tab either
When i run the report only for interactions, even with smaller data (500,118) it crashes the report and tab
However, when i split the target columns and only provide smaller subset of features but with large data (98k,25) then it runs and loads.

So essentially the Interactions part was crashing the report html and the way to make it work is to reduce number of features you use for target interaction. I don't think this is a good solution because now we are left with combinations of feature sets we run and multiple reports to view

Hi @snknitin,
this is an expectable behavior - the time of computation is expected to increase quadratically with the number of columns available in your dataset, particularly considering correlation matrices and interactions.
In what concerns your browser crash, it is also a known and expected limitation. It has to do with the amount of information held by the HTML, that for 118 columns is quite a lot, so the only way to depict all the columns is by reducing the amount of information shown.
If this is a blocker for you, I'm happy to discuss more on our commercial offering, otherwise, if no other questions, I'll be closing the issue.


[CS2103T-T09-3] Trackermon

Trackermon helps users to track and remember what shows they have watched. This enables them to be able to easily look up the list of shows that they have watched if they need to.

Codecov Report

Merging #38 (482eb68) into master (4f7660a) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #38   +/-   ##
=========================================
  Coverage     72.15%   72.15%
  Complexity      399      399
=========================================
  Files            70       70
  Lines          1232     1232
  Branches        125      125
=========================================
  Hits            889      889
  Misses          311      311
  Partials         32       32

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4f7660a...482eb68. Read the comment docs.


Integrate a runner and reporter for testing types

Summary
Type definitions help ensure your code is correct, but it’s also very useful to ensure your types are correct, especially if you ship types with your library. Ensuring that you get type errors where you expect is also useful, so your types don’t regress in functionality by accident. Being able to test your types next to your logic, with much the same API, will make it easier for people to maintain high quality type definitions.
This will help APIs injected into a test by Jest are well typed.
Test Plan

This can be tackled by making a custom runner that will just extend SamVerschueren/tsd inside.
tsd essentially runs the typescript compiler and injects assertions, which can be added in a custom runner which will run part of Jest's test suite
The issue is targeted towards "write tests for Jest's own types" rather than "create a typings test runner"
Assert type tests for expect and different jest objects like jest.mock, etc
Assertions that invalid things are reported as type errors would be great.

Todos

[ ] Basic Scaffolding
[ ] Integrate TSD

There are techniques like const a: import("jest").Expect which would allow types to be passed around internally without too much churn

[ ] Write tests for this runner
[ ] Update the documentation (CHANGELOG.md)

References

Current test suite in DefinitelyTyped types: https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/jest/jest-tests.ts
TSD: https://github.com/SamVerschueren/tsd
Custom Runner: https://jestjs.io/docs/en/configuration#runner-string

Could probably draw inspiration from jest-runner-prettier

@sauravhiremath @jevakallio @smartclash I have updated the Summary and Test Plan. Waiting for feedback 👍

Since this issue is more of having to make a custom runner, we'll be making this as a separate project. Integrating it inside Jest itself could be a bad idea.
You can follow the development of this runner at https://github.com/MLH-Fellowship/jest-runner-tsd/

I agree

Sorry I never got back to this - the plan looks great!

The underlying library (TSD) that we are planning to import needs some stuffs to be fixed. So, a fork has been made at https://github.com/MLH-Fellowship/tsd. They'll be fixed and we will be opening a PR to the origin. After it is merged it'll be used in custom runner that we are making.

Final work plan.
We'll be having a separate config file for jest, something like jest.config.types.js which will use the custom runner we are making (jest-runner-tsd).
We'll also be adding a script inside package.json so that we can test types by running yarn test-types which would basically run jest with the jest.config.types.js config.
About where to place the type definition test files, I think it would be great to place them under __tests__ folder of each package and name the file as types.test.ts.
Or we can have a new folder inside __tests__ something like types and place multiple test files inside the types folder.
Let me know which one is the better approach or if you have something better.
No matter which approach, we have to configure testRegex option to make it correctly find the test files.

If the PRs take time landing in upstream tsd I vote for publishing a fork and using that in the runner to unblock work in landing this in Jest. Should be simple enough to switch back to mainline once tsd is updated 🙂


Requesting digitalOcean support

Hello Elie,
hope you are doing good, this is a nice work you have done. I would like to know if you could do the same for the DigitalOcean droplet for us, user of DigitalOcean.
Thank you.

Hey Joseph,
hank you for your positive feedback! I'm delighted to hear that you found the work useful. I'm currently working on a tutorial specifically focused on DigitalOcean droplets, and I'll be sure to cover the aspects you're interested in. Stay tuned for the next tutorial, I'll keep you posted on its progress. If you have any specific requests or preferences for the tutorial, please let me know. I look forward to sharing the DigitalOcean droplet guide with you!


[Bug] Drop temp tables when doing a full refresh on  Dynamic table by respecting qualified names

Is this a new bug in dbt-snowflake?

[X] I believe this is a new bug in dbt-snowflake
[X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior
While trying to do full refresh or recreate on dynamic table we are getting issue.
while droping it using double quotes with DB and Schema as "DB"."SCHEMA"."model__dbt_backup"
where as while alter table rename to backup its not using DB and schema (defaulted to profiles.yml schema not considering schema from model file). Also no double quotes (so snowflake creating as upper case and model__DBT_BACKUP) which causing issue when we do full refresh
Expected Behavior
I'm expecting to take schema and DB from model file in sql we have config also add double quotes when performing alter table rename to backup so drop and alter will be in sync
Steps To Reproduce
{{
config(
materialized='dynamic_table',
target_lag='30 minutes',
schema='fake_schema'
)
}}
select

from {{ ref('table') }}

Once you run this model in snowflake please do full refresh again to capture statements it will populate the queries
Relevant log output
No response
Environment
- OS:
- Python:
- dbt-core:
- dbt-snowflake:

Additional Context
Adding<EMAIL_ADDRESS>

We also met the same issue that the command alter view db.schema.xxxxx rename to model1 doesn't include snowflake db and schema before model1 when taking a look at the run SQL🙋

This is the same issue that was introduced by one of revisions in 1.7 version

Exactly, when it should be {{this.database}}.{{this.schema}}.model.
This is causing issues, because it is using the default database/schema instead and can't find objects to remove.

@amychen1776 I was about to open a bug but found this one which seems related.
I encounter a bug in dbt-snowflake around using dynamic tables with non-standard quoting policy.
I use the following configs:
dbt_project.yml:
quoting:
  schema: true

profiles.yml:
schema: TrYiNg-938

model file (from the dbt init project):
{{
    config(
        materialized='dynamic_table',
        target_lag='downstream',
        snowflake_warehouse='warehouse'
    )
}}

select *
from {{ ref('my_first_dbt_model') }}
where id = 1

Running dbt run once works and creates this dynamic table. However, the next invocations fail with error:
Database Error in model my_second_dbt_model (models/example/my_second_dbt_model.sql)
  001003 (42000): SQL compilation error:
  syntax error line 3 at position 37 unexpected '-'.

The actual SQL seen in Snowflake query history is:
show dynamic tables
            like 'MY_SECOND_DBT_MODEL'
            in schema DBT_TEST.TrYiNg-938
        ;

Which indeed shows that the schema remains unquoted. As far as I can tell, this command comes from dbt/include/snowflake/macros/relations/dynamic_table/describe.sql which indeed doesn't handle quoting.
Please let me know if more information is required.

Same issue when trying to replace a transient table with a dynamic table. While renaming the existing object the name is not fully qualified, so it fails at that step because there's no active USE SCHEMA context.
Executed SQL
drop table if exists "DEV__SOMEDB"."SOMESCHEMA"."MODEL_NAME__dbt_backup" cascade;

alter table "DEV__SOMEDB"."SOMESCHEMA"."MODEL_NAME" rename to MODEL_NAME__dbt_backup;

create dynamic table dev__somedb.someschema.model_name
target_lag = '30 minutes'
warehouse = SOME_WH
as (
...
)

Error returned when executing
05:31:14    Database Error in model model_name (models/somedb/someschema/model_name.sql)
  090106 (22000): Cannot perform CREATE TABLE. This session does not have a current schema. Call 'USE SCHEMA', or use a qualified name.
  compiled code at target/run/someproject/models/somedb/someschema/model_name.sql

Workaround
This works because the unqualified backup table creation SQL is never generated in the first place.

Drop the object(s) manually.
Run dbt again so that it creates the objects from scratch.

Fix?
I believe the issue is somewhere around here. It doesn't appear that the backup_relation.render() function is fully qualifying the backup relation's name when generating the alter sql using the make_backup macro.
dbt/include/snowflake/macros/relations/create_backup.sql


rename update

Part of #394.

Codecov Report

Merging #398 (503b313) into main (b5cc22e) will decrease coverage by 0.05%.
The diff coverage is 71.42%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@            Coverage Diff             @@
##             main     #398      +/-   ##
==========================================
- Coverage   86.58%   86.54%   -0.05%
==========================================
  Files          43       43
  Lines        1610     1605       -5
==========================================
- Hits         1394     1389       -5
  Misses        216      216

Impacted Files
Coverage Δ

src/StructuralAnalyses/StructuralAnalyses.jl
77.77% <0.00%> (ø)

src/StructuralSolvers/StructuralSolvers.jl
71.42% <80.00%> (-0.80%)
:arrow_down:

src/StructuralAnalyses/NonLinearStaticAnalyses.jl
86.84% <100.00%> (ø)

... and 2 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


O365 connectors within Teams will be disabled 2024-10-01

Problem
Microsoft announced July 3rd, 2024 that Office 365 (O365) connectors within Microsoft Teams would be retired in 3 months and replaced by Power Automate workflows, or just "Workflows" for short.
Quoting from the microsoft365dev blog:

We will gradually roll out this change in waves:

Wave 1 - effective August 15th, 2024: All new Connector creation will be
blocked within all clouds
Wave 2 - effective October 1st, 2024: All connectors within all clouds
will stop working

Preparation
As noted, Existing O365 connector webhook URLs should continue to work until 2024-10-01.
No immediate changes are needed to continue using this tool to submit messages to Microsoft Teams, but you may wish to go ahead and prepare for existing O365 webhook connector (URLs) to stop functioning on that date.

Select a long-lived (aka, "service") account
Create replacement Power Automate Workflow connector URLs
Continue to use existing O365 connector webhook URLs with this tool

Remediation
A new version of this tool is being prepared and should be available ahead of the 2024-10-01 end of life for O365 connectors.
Once available, please test using both the deprecated O365 connector webhook URLs and the replacement Power Automate Workflow connector URLs. You should have equivalent functionality from both.
See also:

https://github.com/atc0005/go-teams-notify/issues/262

I've been running the v0.13.0-alpha.1 and v0.13.0-alpha.2 releases together for about a week now without issue.
I've used both O365 connectors and Power Automate workflow connectors without issue.


Fix minor indent bug

Description
Fixed a minor indent bug I noticed in the --help output.

Thanks! :+1:


Investigate viz libraries for insights

Look into libraries for displaying charts/graphs/etc for displaying insights data.

D3
...

Currently using chart.js - https://www.chartjs.org/

I think we should investigate moving to visx - airbnb.io/visx

Requires a formal recommendation as an output


KEB jobs Dockerfile cleanup

Description
There is a newly introduced generic Dockerfile for KEB jobs - https://github.com/kyma-project/control-plane/pull/2475. This PR depends on https://github.com/kyma-project/test-infra/pull/6929 and removes no longer necessary Dockerfiles.

https://github.com/kyma-project/test-infra/pull/6929 merged, opening for reviews

https://github.com/kyma-project/test-infra/pull/7023#discussion_r1118393206
recent development in image-builder likely causing job builds to fail

/retest

/test pull-keb-cleanup-job-build


Make Ssclic mandatory if smclic is implemented and S mode is implemented.

Make Ssclic mandatory if smclic is implemented and S mode is implemented.

Hi All,
I have open this issue following our discussion but now I m not sure that it make sense.
Having just smclic and still having S mode is a valid option. We need to consider that the ssclic will bring the Mode bit inside clicinattr whcih will be potentially 1 bit per IRQ line if we are making mandatory the ssclic .
This could have a huge impact  and can be complex to justify in some situation.
I m happy to force that ssclic can be present only if smclic is , but I won t bring any other relation between ssclic and smclic,
Please comment. If you disagree.
Otherwise I will close the issue during the next meeting.

The ssclic extension depends on the smclic extension.
This sentence needs to be replhrased.


flat design

Development of flat design theme because it is the current standard on the web.

Not sure what you mean with flat.


BUG: TelepathyTransport delivers messages even when this.enabled == false across scene loads

Found this bug when joining a server using Unity 2018.3
TelepathyTransport delivers the first message which tells the client to change scenes and load the next scene.
However because the LateUpdate has already started, it doesn't check that this.enabled and processes the rest of the message queue during the scene transition.
Debugged and diagnosed with Paul to find this issue.

great find

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add AutoDLL webpack-plugin

AutoDLL webpack-plugin can boost the speed of webpack compilation.

This seems too good to be true! +2

Added


Feat/UI fixes n updates

Feature Enhancements and UI Improvements
This PR introduces several enhancements to improve user experience and functionality:
1. Enhanced New Project Dialog

Updated UI to accommodate more templates
Added search functionality with an empty state prompt for GitHub contributions
Improved template icons and descriptions

2. Optimized Terminal Resizing

Implemented text position adjustment during terminal resizing
Added 50ms debounce for performance optimization in live version

3. File Explorer Performance Boost

Introduced skeleton loading for a snappier user experience

4. Improved File Explorer Organization

Implemented sorting algorithm:

Folders displayed first
All items sorted in alphabetical order

Enhances ease of use and file navigation

Before
After

Please review and provide feedback. Thank you!

This is so good. You always make the most beautiful PRs.

This is so good. You always make the most beautiful PRs.

Thanks James


How webhook can be executed on system without the internet

How webhook can be executed on a system not accessible from the internet, as they are typically used to trigger events in response to certain actions. Is there a way to maybe configure a proxy for this, which would forward incoming webhook requests to a local system?

It would be possible to use something like ngrok or Expose to make the local system accessible from the outside. However, this is quite some configuration work. Hence, we are especially interested in a way to trigger the webhooks without exposing the local system to the outside. I can think of two ways:

A "Fetch Payment Update" button in the order view of Magento, which gets the latest updates from Wallee.
A possibility to copy a webhook as a curl request in the Wallee backend, which can then be executed locally.

Dear waleedbasit, sprankhub
We also would recommend you NGROK for local systems.
Else as recommended from @sprankhub you can update the concerned wallee objects (transactions, payment methods, etc.) over our SDK's.
You can extend our plugin for your local system purposes by yourself with our PHP SDK: https://github.com/wallee-payment/php-sdk/ (this SDK is already used within our wallee plugin)
For example, if you want to get the newest transaction status, you can do it like this with our SDK:
$apiClient = new ApiClient("your application user id as int", "your application user token as string");
$spaceId = "your space id as int";
$transaction = $apiClient->getTransactionService()->read($spaceId, "concerned transaction id as int");
echo $transaction->getState();

Please let me know, when you have further questions.
Best regards


Create executing_java_comments

Awesome, thanks for contributing! Can you move the files to a folder following the repository structure?

heyy i am a newbie & was unable to resolve this issue so i created a new pull request after following all your instructions!

Okay @Mitasha98 no problem, I will review your new Pull Request : )


Archisman Dawn | DP-LEETCODE , Trees Algorithm  | Added 0-1 knapsack …

…, coin change problem solution , egg drop , uniqueBST-II
Please review & approve @siddharth25pandey

@all-contributors please add @DawnArchi01 for his code


methodInterceptor  closure passed to Script invokeMethod has wrong signature?

https://github.com/lesfurets/JenkinsPipelineUnit/blob/4c86c064146e45185a1621062fd0257bc293b4bf/src/main/groovy/com/lesfurets/jenkins/unit/PipelineTestHelper.groovy#L130
I updated to JenkinsPipelineUnit v1.1 and found that my mocks are being called with single element arrays where I might expect say just a string (e.g args value of ["foo"] instead of "foo").
The invokeMethod signature for Script is defined here:
http://docs.groovy-lang.org/2.4.11/html/gapi/groovy/lang/Script.html
The expected signature is:
public Object invokeMethod(String name, Object args)
But the signature for what is used in the helper is passing args as an array:
public methodInterceptor = { String name, Object[] args ->
It affected my project here where I get args using Spock mocks and have to de-reference the array:
https://github.com/macg33zr/pipelineUnit/blob/52eae8392f4ddb1460988ea1b31ecb0d301a7fbb/pipelineTests/groovy/tests/job/JenkinsfileTestSpec.groovy#L30
It is a bit subtle and can be worked around (change all tests though!).
What do you think is this an issue or a feature?
Best regards,
Bill

Hi Bill,
I agree it is indeed a bit subtle. We had issues with this signature in the past, I am not sure which is the best solution, I’ll look and get back to you.
For now you can hold the migration to 1.1. There are other issues with it as well cf #60

I am facing the same issue, has anything changed since this issue was reported?


Makes default propagation for non-request spans B3 single

As described in the referenced issue. In-process messaging sometimes
uses propagation, and the default of B3 multi makes the overhead quite
bad. This switches to B3 single which significantly redudes that burden.
https://github.com/spring-cloud/spring-cloud-sleuth/pull/1607
^^ already changed this in sleuth. this is just defaulting here
We should note that B3 single is widely supported at this point, and all
our instrumentation extend Request when injecting.
See #1020 as this was the change that allowed requests to differentiate

It is a fair question. I guess it will be a sooner or later thing, and I don't expect us to do a lot of releases of 5.x... so if no-one addresses this they will pin themselves on something we aren't maintaining if we wait until 6.x...

My thought is that if someone is updating brave, and would update it to 5.12, then they likely update other things, and will be either using our instrumentation or other instrumentation that can parse single by now. It is hard to "deprecate" default configuration.. this is the dillemma of the issue. I suppose we can add a log-once to warn level now and defer it to another minor at the very least?

Yeah having a log to help transition at least seems nice. I'm less worried about Brave/Zipkin tooling, but I can imagine at least one case out there of user code itself reading the headers to do something with it that would stop seeing it

it is a good point. since this is default I have another thing in progress
RemoteGetter (including span kind) which can allow a more relevant means to
warn people. otherwise we will only get warned at startup vs when it is
read.
On Mon, May 11, 2020, 11:08 AM Anuraag Agrawal<EMAIL_ADDRESS>wrote:

Yeah having a log to help transition at least seems nice. I'm less worried
about Brave/Zipkin tooling, but I can imagine at least one case out there
of user code itself reading the headers to do something with it that would
stop seeing it
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/openzipkin/brave/pull/1204#issuecomment-626446406,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAPVVYRKQ4PRQ66NLT2SJDRQ5T3DANCNFSM4M5IUPMQ
.

I thought a lot about this and thing that having a WARN message, especially
as we have to use JUL, is a case where the cure is worse than the disease.
Just so we are clear, only B3 do we have the ability to separately control
local propagation format (at the moment), and all variants of B3 read all
header formats.
I'm inclined to just let this bomb on 6, I guess.. but we really must fix
this by then for sure. This is not just a high overhead problem, but also
leaking a hyphenated format exactly the place it is most likely to have
problems with it (messaging processors).
Maybe we can update Kafka Streams and the messaging README to warn people
about this configuration problem until we feel safe to fix it, in worst
case Brave 6 which might happen this year.

meanwhile in preparation I'll search all github for http
instrumentation not yet using our request types and try to help them
update. This will reduce the surface area ideally to the "really
local" so that when we go 6, this is less a concern and inventoried to
some extent.
thanks for sounding the alarm!

https://github.com/openzipkin/brave/issues/1211


KAFKA-4116: Handle <IP_ADDRESS> as a special case when using advertised.listeners

Issue: https://issues.apache.org/jira/browse/KAFKA-4116

@gwenshap Can you review this please?

@gwenshap, @harshach ping.

@guozhangwang can you review this?

+1

I think we'll go with https://github.com/apache/kafka/pull/3382 which disallows <IP_ADDRESS> for advertised.listeners. So maybe this PR can be closed.

Closing this since the JIRA was resolved by #3382.


Added a simple profiler for capturing load times

We'd like to be able to record the start and end times for each loading module.
This change provides two levels of profiling: a global profiler and a context level profiler. Some notes about these:

By default they are no-op
To set the instance used by GlobalReactLooselyLazyProfiler, call setGlobalReactLooselyLazyProfilerInstance from this package.
To set the instance used by ProfilerContext use the regular React context pattern// We can use more contextual data from this provider component in our profiling records.
const instance = {
  onLoadStart: ...
  onLoadComplete: ...
};

return (
  <ProfilerContext.Provider value={instance}>
    <MyLazyComponent />
  </ProfilerContext>
);

Lazy modules preloaded will use the global instance while instances loaded by <LazySuspense> will use the contextual profiler.
By default if you don't set a profiler context it will push through to the GlobalReactLooselyLazyProfiler instance.

NOTE: The two profilers receive different identifying data at the moment (href vs. moduleId). At present we're hoping to resolve this at processing time which avoids having to pass moduleId into insertLinkTag.

Thanks for the contribution! Hope I'm not misinterpreting the reasoning behind the PR, but I'd like to get a better understanding on the motivations.
First of all, in our product we achieve that on the consumer side by simply wrapping imports with a "tracking" function:
const LazyFoo = lazyForPaint(() => markBundleReady(import('./foo')));

Where markBundleReady can do all the things consumers want without the need of React being involved. It also has the benefit of working with React.lazy in the same way. I see the benefit of this being set-up once, trigger it from all the RLL components on the page. However, automatically adding the wrapper function is easy too, so wonder if this is really necessary given the above alternative.
Secondly, I worry about adding timings for preloads as those will not be reliable. Most of the times, they will be rendered by SSR so we won't be able to get right timing information. It is much more effective to use look for the entries in navigation timing, where browser will collect such information for us.

@albertogasparin I am not a fan of manually inserting markBundleReady, as consumers often miss adding them. You can automate the insertion, but in my opinion that has a bigger maintenance cost than supporting something directly in the library. For example:

You have to choose the right tool like babel, parcel, webpack, etc
The code generation should be optimal and work well with the bundler (however, this should be trivial if you just import the function from a common package)
The function becomes an implicit dependency, making it more difficult to understand how it works / where it gets generated

@nr-tw What is the use-case for the contextual profiler, and it needing to use the React APIs (context)?

We use context because we want to capture specifically which part of the application (effectively, applying labels to different subtrees) is triggering / consuming / waiting the LazyComponents. That allows us to attribute loading with the relevant component/team & sub-experience


Start the vsls-agent only when needed

VS has gotten really bloated with creating extra processes when all I do is start it up or load a solution. It is getting a little out of hand. I notice that when I start VS with Live Share installed it creates a vsls-agent process. It doesn't matter whether I ever actually use the extension (which I rarely do when not working remotely). Yet this process is running in the background.
Please don't start the vsls-agent process until I do something with the extension that would trigger it's need such as trying to start or join a session.

Adding @Chuxel, @lostintangent, and @jonwchu.

@CoolDadTx try to modify the package.json file in the root directory of the extension, they are located in %USERPROFILE%.vscode\extensions in Windows find the extension folder and try to play with "Activation Events", it's not recommended method so create a backup of the package.json!

@tsvetilian-ty , sorry I should have clarified. This is for VS 2017, not VS COde

Agree, this process is very unnecessary, just because you have installed the extension.
Furthermore, on a freshly installed Silicon chip Mac this is the only Intel process running, and it stands out.
Please make this configurable, and consider updating this process too.

Apologies for the delay here. For VS Code, vsls-agent has been removed. However, since the original post was for VS, we're keeping this open. We are currently working to improve the experience in this area for VS.

Thanks for taking the time to file this issue to submit this request. Unfortunately, we’re not able to prioritize this feature over the other current areas of focus on the team. As mentioned earlier, for VS Code, vsls-agent has been removed. We understand that this may be disappointing. We appreciate your input and comments on this issue till now.

VS has gotten really bloated with creating extra processes when all I do is start it up or load a solution. It is getting a little out of hand. I notice that when I start VS with Live Share installed it creates a vsls-agent process. It doesn't matter whether I ever actually use the extension (which I rarely do when not working remotely). Yet this process is running in the background.
Please don't start the vsls-agent process until I do something with the extension that would trigger it's need such as trying to start or join a session.

Despite the latest updates in Visual Studio 2022 and Live Share, this issue persists. I fail to comprehend why, upon installing the Live Share plugin, a 'vsls-agent' process is created in every project, even when not in use.


Meter numerator, tempo, and beat estimation in musicanalysis

Codecov Report
Base: 79.29% // Head: 79.63% // Increases project coverage by +0.34% :tada:

Coverage data is based on head (354319d) compared to base (eb0c917).
Patch coverage: 93.83% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #144      +/-   ##
===========================================
+ Coverage    79.29%   79.63%   +0.34%
===========================================
  Files           54       56       +2
  Lines         9380     9607     +227
===========================================
+ Hits          7438     7651     +213
- Misses        1942     1956      +14

Impacted Files
Coverage Δ

partitura/musicanalysis/meter.py
93.33% <93.33%> (ø)

partitura/musicanalysis/__init__.py
100.00% <100.00%> (ø)

tests/test_time_estimation.py
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


SLIM-125: log alarm notifications

Adds logging of push alarm notifications to the device log items of OSGP logging and the event log of OSGP core.

All is well.


Component manager deletes required dependencies (PACMAN-781)

The Component Manager version
1.4.1
ESP-IDF Version
5.1.1
python Version
3.11.4
Operating System
Windows 10
Browser (for https://components.espressif.com  Issues)
No response
Description
I think there is a bug in how download_project_dependencies works. https://github.com/espressif/idf-component-manager/blob/2711ccc08bfd0db60c4dbd15bbdd72a5e0f5b3fa/idf_component_manager/dependencies.py#L258.
It should scan project requirements (idf_component.yml), build the dependencies tree and add download both project requirements and their dependencies.
Problem is, that only packages that are NOT in project requirements (... noi in ... and ... not in ..., see below) are actually considered for download.
https://github.com/espressif/idf-component-manager/blob/2711ccc08bfd0db60c4dbd15bbdd72a5e0f5b3fa/idf_component_manager/dependencies.py#L312-L316
Result is that I specify two dependencies in my idf_component.yml file but they are removed from the managed component folder because considered "unused".
To Reproduce
Create a idf_component.yml file:

dependencies:
  esp-zboss-lib: "^1.0.7"
  esp-zigbee-lib: "^1.0.7"
  idf: ">=5.0"

Run make menuconfig (or other cmake commands), this is the wrong output:
Deleting 2 unused components

 espressif__esp-zboss-lib

 espressif__esp-zigbee-lib

Processing 1 dependencies:
[1/1] idf (5.1.1)
Content of dependencies.lock file is this one:
dependencies:
  espressif__esp-zboss-lib:
    component_hash: null
    source:
      path: C:\Users\semer\Documents\PlatformIO\Projects\nfc-alarm-panel\managed_components\espressif__esp-zboss-lib
      type: local
    version: 1.0.7
  espressif__esp-zigbee-lib:
    component_hash: null
    source:
      path: C:\Users\semer\Documents\PlatformIO\Projects\nfc-alarm-panel\managed_components\espressif__esp-zigbee-lib
      type: local
    version: 1.0.7
  idf:
    component_hash: null
    source:
      type: idf
    version: 5.1.1
manifest_hash: 89bfe204db7dbbb4b6e85994425215a0d70c45f775606ba52a1aea9f595c6bac
target: esp32c6
version: 1.0.0

### Expected behaviour

Project requirements should be downloaded and kept.

Fix: requirement_dependencies list should be added to the project_requirements_dependencies one.

### Additional info

_No response_

### I have checked existing issues and online Documentation

- [X] I confirm I have checked existing issues and online Documentation.

I just saw that this commit (https://github.com/espressif/idf-component-manager/commit/91058b8a36d11f42e9f68dacd1f2095e145e5c18) could have fix the issue, but it's not released yet, let's me try it from master

Nope...same issue

@Suxsem thanks for being patient, we were slow to respond during holidays.
I wonder why you dependencies.lock show them as local dependencies, I would expect something like:
dependencies:
  espressif/esp-zboss-lib:
    component_hash: 9dc1059b8a6695580ab0324d41f2f88e324c1486b196e20d087f3dcb78c1c654
    source:
      service_url: https://api.components.espressif.com/
      type: service
    version: 1.0.7
  espressif/esp-zigbee-lib:
    component_hash: 3171496afcbdf5c370a3f74f6bc6a7e3196227438b0405d8714796c9d6f68a36
    source:
      service_url: https://api.components.espressif.com/
      type: service
    version: 1.0.7

To help me to reproduce this, please share your project's and main component CMakeLists.txt files.
Do you have set(COMPONENTS main) in your CMakeLists.txt?

@Suxsem Did you manage to solve the issue? If not, could you please help us to reproduce it?

@Suxsem I believe this issue should be addressed in the component manager v1.5.1, so I'm closing this know. If you still experience the problem please don't hesitate reopening the issue.


Update bindgen 0.59 => 0.68

This fixes a few errors with some derive implementations.
It may need further testing. I tested host and device on rp2040, and it worked.

Cool thanks


Adding nyan

This probably not the best way to add new command, but at least I tried

Code refactored in latest commit


fixing:  Empty response for node-fetch http call

this PR closes; https://github.com/keploy/keploy/issues/376

@re-Tick  please review this PR

LGTM

Closing this PR, since the changes are merged for the issue in this.


Set missing display name property to Auth Initiation Data

Fixes the issue of not having the display name property in the auth initiation data during api based autehntication.
Related issue: https://github.com/wso2/product-is/issues/20219

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/9028232368

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/9028232368
Status: failure

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/9028339118

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/9028339118
Status: success


Improve library usage of the tool

What is the purpose of your issue?

[ ] Bug
[x] Feature Request
[ ] Question
[x] Other

Description

Currently, using the tool as a library can be unintuitive and feel kinda hackish. We need to work on improving this.
For example, calling downloader.Downloader.download_single would end up performing all possible operations on the track (download, convert and adding metadata). It would be a good idea to offer different methods for each purpose. There are many similar cases, for example we could add an attribute on downloader.Download class to get the filepath of the track being downloaded, etc.
If we really make it usable as a library, people would be able to write their own code, for example solving #417. This would be very neat.
Also see how the current library usage looks like - #467, #455, #359.

Major steps taken.
I know it's late, just clearing out old issues.


Minimize console output for folders that are nonexistent or with no new scripts

Since grate will suppress console output for skipped files by default, it would make sense to skip output of the extra dividers and whitespace to trim unnecessary content as well. This will help to eliminate noise when verifying that scripts are detected and run appropriately.
Perhaps showing something like:
Looking for scripts in xxxx
--no new scripts--

and/or checking for a directory before output to enable simplifying the message to
Skipping runAfterFirstUp, xxxx folder does not exist

Original comment posted by @docrinehart in https://github.com/erikbra/grate/issues/140#issuecomment-976111155

This is a good idea.
However, I have a question, do you have an idea to how we know if there are any new scripts, before we have tried running them?

@erikbra Might be overkill, but what if we create a hash of all the files in the folders and store that somewhere?
Or even just keep track of the amount of files?

+1

I've started looking into this, given the ease we can elevate the folder exists check here up a method and return early we can definitely quieten the output for non existent folders, which in my sample-size of one is far more common than fully created folder heirarchies that just don't have files in them.
My big question is what log level do people feel the "Folder doesn't exist" message should be logged at?  If we log at Information than it means grate self-describes the folder options available to you when you run the tool as a new user, which I like.
Logging at Debug or lower reduces your console output by another line.
Dealing with populated folders full of files but that don't have changes is TBD (and more difficult).

Great to see you have started thinking about this, @wokket !!
Could the following logic work?

For each folder we process:

In the beginning, check if the folder exists. If it doesn't, only output a very minimal message, à la:
"Skipping runAfterFirstUp, xxxx folder does not exist".
If it does exist, write the same as today, "Looking for {FolderName} scripts in \"{Path}\".{Message}" (maybe the "Message" should be of a lower verbosity(?) - as in your link above, @wokket
Make a boolean variable in GrateMigrator.Process, and update this on every run of a script, to see if any script was run
Loop through all the scripts, keep the logging in DbMigrator.RunSql unchanged
After the loop, back in GrateMigrator.Process, if no scripts were run, output a "--no new scripts--" message.

I would prefer to keep it at the same level as the "Looking for {FolderName} scripts in \"{Path}\"." message, i.e. Information.

@wokket / @erikbra - A bit late, but thanks for taking this on and pushing it through. Looking forward to updating my projects where I use it.


Cut release + publish -- v1.11.1

Description
Adding this as an issue so it's top of mind. @pitbulk mentioned that this will be taking place soon in our conversation in #548

Thank you for maintaining this library for the community!

@pitbulk @lawrencepit @olleolleolle
What's the status on this? Would love to get a new release cut and gem bump.

Sorry for the delay.
I have some PRs and code pendingbut hope to make the release soon.

@pitbulk is there anything we could do to help with this release? We're relying on the audience patch, pointing directly at the ref, so it's risky and we'd like to help with the release in any way possible.

@pitbulk is there anything we could do to help with this release? We're relying on the audience patch, pointing directly at the ref, so it's risky and we'd like to help with the release in any way possible.

Working on it right now, sorry again for the delay

Working on it right now, sorry again for the delay

Hey there ! 👋🏼 , similar situation here, @pitbulk do you have an estimation of a date for the release ? Just to keep in mind. Thanks for maintaining this project.

Hey there ! 👋🏼 , similar situation here, @pitbulk do you have an estimation of a date for the release ? Just to keep in mind. Thanks for maintaining this project.

I think the ETA is next week,
I still need to review & merge:

https://github.com/onelogin/ruby-saml/pull/558
https://github.com/onelogin/ruby-saml/pull/557
https://github.com/onelogin/ruby-saml/pull/545
https://github.com/onelogin/ruby-saml/pull/520

I'm working at the branch:
https://github.com/onelogin/ruby-saml/tree/1.12.0-dev

I think the ETA is next week,
I still need to review & merge:

https://github.com/onelogin/ruby-saml/pull/558
https://github.com/onelogin/ruby-saml/pull/557
https://github.com/onelogin/ruby-saml/pull/545
https://github.com/onelogin/ruby-saml/pull/520

I'm working at the branch:
https://github.com/onelogin/ruby-saml/tree/1.12.0-dev

@pitbulk Thanks for the update ! 👏🏼

@pitbulk Thanks for the update ! 👏🏼

@fede-moya @kurko @swordfish444  1.12.0 branch will be released tomorrow.
Let me know if you find any issue on it.

@pitbulk Thank you sr ! 👏🏼 👏🏼 👏🏼

1.12.0 released


[Flake] timeout in cmake-super build

One of the GCB builds timed out:
https://pantheon.corp.google.com/cloud-build/builds;region=global/0b9e85cc-2331-4136-98ae-7601d8b32519?project=cloud-cpp-testing-resources
Maybe the timeout is too optimistic?

1 hour on 32 cores should be enough. Not sure what happened here. I wish CMake could output periodic time stamps, that would help.
One other though is that it doesn't look like the super build is using ccache for the deps it compiles. We may want to change the CC and CXX vars to look like export CXX="ccache ${CXX}". That may help.

We have stopped using super builds, this is now obsolete.


Test/Check User and Role retrieval

Check/Verify how to retrieve Users and Roles from Abiquo.

Documentation how to get Abiquo Users and Roles can be found here.
Documentation how to retrieve Users from an entity can be found here
Roles documentation can be found here
Available methods (i.e. to get privileges from a role) are documented here

Example to test, the curl to get Users will be like:
> curl 'https://HOST/api/admin/enterprises/1/users?maxSize=5&maxSizeMini=1&currentPage=1&limit=25&startwith=0&totalSize=0&asc=true&connected=false&cacheStamp=25_1470749648130' -H 'Pragma: no-cache' -H 'Accept-Encoding: gzip, deflate, sdch, br' -H 'Accept: application/vnd.abiquo.users+json;version=3.8' -H 'Cookie: ABQSESSIONID=SESSIONID; auth=AUTH_TOKEN' -H 'Connection: keep-alive' -H 'Cache-Control: no-cache' --compressed --insecure

User calls done. Added script Get-Users.ps1 containing the commands to the newly created scriplets folder of biz.dfch.PS.Abiquo.Client.
Querying the users results in a list of user objects. A sample object is shown in the following code snippet.
<#
Sample result

links       : {@{title=Abiquo; rel=enterprise; type=application/vnd.abiquo.enterprise+json;
              href=https://HOST/api/admin/enterprises/1}, @{title=CLOUD_ADMIN; rel=role;
              type=application/vnd.abiquo.role+json; href=https://HOST/api/admin/roles/1},
              @{title=admin; rel=edit; type=application/vnd.abiquo.user+json;
              href=https://HOST/api/admin/enterprises/1/users/1}, @{title=virtual machines;
              rel=virtualmachines; type=application/vnd.abiquo.virtualmachines+json;
              href=https://HOST/api/admin/enterprises/1/users/1/action/virtualmachines}...}
id          : 1
name        : Cloud
nick        : admin
locale      : en_US
surname     : Administrator
active      : True
email       :
description : Main administrator
firstLogin  : False
locked      : False
authType    : ABIQUO
role        : @{links=System.Object[]; id=1; name=CLOUD_ADMIN; blocked=False; privileges=}
enterprise  : @{links=System.Object[]; ramSoftLimitInMb=0; ramHardLimitInMb=0; cpuCountSoftLimit=0;
              cpuCountHardLimit=0; diskSoftLimitInMb=0; diskHardLimitInMb=0; storageSoftInMb=0; storageHardInMb=0;
              vlansSoft=0; vlansHard=0; publicIpsSoft=0; publicIpsHard=0; repositorySoftInMb=0; repositoryHardInMb=0;
              id=1; name=Abiquo; isReservationRestricted=False; workflow=False; twoFactorAuthenticationMandatory=False}
#>

There is a role property. However this role property seems to only hold 1 role. This has to be checked. Furthermore no tenant information is present...

@dfch is enterprise the tenant information or is there really no tenant information available?

Mail sent to Daniel to clarify content of role property
Hi Daniel

Quick question. When retrieving users the answer contains a 'role' property. This property only contains one role. Is it possible for a user to be in more than one role? If yes, which role will be set in role property of a user if a user is in more than one role?

Regards,
Marc

Answer from Xavi
I respond to you

No, is not possible. The realation in 1 user 1 role

As shown in the first comment the privileges can be expanded from the role property of a user.

Retrieving role by Id (i.e. /admin/roles/{id} returns the desired result:
{
	"links": [{
		"title": "CLOUD_ADMIN",
		"rel": "edit",
		"type": "application/vnd.abiquo.role+json",
		"href": "https://HOST/api/admin/roles/1"
	},
	{
		"title": "privileges",
		"rel": "privileges",
		"type": "application/vnd.abiquo.privileges+json",
		"href": "https://HOST/api/admin/roles/1/action/privileges"
	},
	{
		"title": "Global scope",
		"rel": "scope",
		"type": "application/vnd.abiquo.scope+json",
		"href": "https://HOST/api/admin/scopes/1"
	}],
	"id": 1,
	"name": "CLOUD_ADMIN",
	"blocked": true
}

When trying to retrieve all roles (i.e. /admin/roles) I get the following result
{"links":[],"collection":[],"totalSize":0}

Role calls done. Added script Get-Role.ps1 containing the sample request to the newly created scriplets folder of biz.dfch.PS.Abiquo.Client.

I will close this issue as all questions related this issue are answered except the one concerning tenant information. I created a separate issue for that.

Answer concerning retrieving all roles from Xavi
When accessing the Abiquo API make sure you always set the Accept header to the right value. Some resources (the roles resource is one of them) support multiple representations and if you omit the Accept header (an Accept */* is sent in that case) Abiquo will pick one of the available representations and could return something you weren't expecting. If you inspect the response and look at the Content-type header, you'll see what Abiquo has returned. It will probably be a representation that is not the one you expected (and the returned list is probably empty because no roles were found that could be represented as such).
In this case, try sending the "Accept: application/vnd.abiquo.roles+json" (or xml, whatever you are using) header.
Hint: If you are navigating the API by following the links (you should do this instead of hardcoding URLs in your client), each link has a "type" attribute with the default value to be used for the Accept header when accessing the referenced resource.

UPDATE concerning tenant information. The equivalent to a Tenant in Abiquo is called Enterprise as described in the Abiquo Glossary
On our multi-tenant platform, an enterprise is a cloud tenant. So an enterprise could be a third-party company, development group, company department, etc. that can access the same virtual infrastructure.


FastForwarded still includes earlier RDATEs if fastForwardTo ≤ DTSTART

I believe there's a bug in the way FastForwarded handles RDATES. It could potentially be related to #67, but I'm not sure.
Steps to reproduce

Define FastForwarded Composite of

InstanceList of RDATEs, e.g. [2023-07-15, 2023-08-15]
RuleInstances of some RecurrenceRule, e.g. FREQ=MONTHLY;BYMONTHDAY=10;
fastForwardTo after an RDATE, e.g. 2023-08-01

Define RecurrenceSet of the above with first after or equal to fastForwardTo, e.g. 2023-08-01 or 2023-08-02

Observed behaviour
The RecurrenceSet iteration starts with the RDATE prior to fastForwardTo, then occurrences from the fast forwarded rule instances, i.e.

2023-07-15
2023-08-10
2023-08-15
2023-09-10
2023-10-10
...

Expected behaviour
The RecurrenceSet fast forwards past the RDATEs, and iteration starts with the first occurrence after fastForwardTo, i.e.

2023-08-10
2023-08-15
2023-09-10
2023-10-10
...

If instead fastForwardTo is strictly after first by one day, the expected behaviour is observed.
Notes

All the DateTimes I'm using are instantiated using the DateTime(int year, int month, int dayOfMonth) constructor (where month is 0-indexed).
I'm calling the lib from Clojure, but it should be functionally equivalent to something like this in Java:RecurrenceRule rrule = new RecurrenceRule(Freq.MONTHLY);
rrule.setByPart(Part.BYMONTHDAY, 10);

long[] rdates = new long[] {1689379200000,<PHONE_NUMBER>000}; // 2023-07-15, 2023-08-15

for (DateTime instance:
     new RecurrenceSet(new DateTime(2023, 7, 1), // 2023-08-01
                       new FastForwarded(new DateTime(2023, 7, 1), // 2023-08-01
                                         new Composite(new InstanceList(rdates),
                                                       new RuleInstances(rrule))))) {
    // first element is 2023-07-15
}

Thanks for the detailed report. Looks indeed like a bug. I'll take care of that.

fixed in 1.15.1


C14 : init

Motivation for this change
C14 is designed for data archiving & long-term backups and lets you store and retrieve large amounts of data.
Things done

[ ] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] OS X
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Fits CONTRIBUTING.md.

I would do the following changes:
diff --git a/pkgs/applications/networking/c14/default.nix b/pkgs/applications/networking/c14/default.nix
index a3d1c5f..3a7e1c7 100644
--- a/pkgs/applications/networking/c14/default.nix
+++ b/pkgs/applications/networking/c14/default.nix
@@ -1,16 +1,14 @@
-{ stdenv, buildGoPackage, fetchgit }:
+{ stdenv, buildGoPackage, fetchFromGitHub }:

 buildGoPackage rec {
-  name = "c14-cli-${version}";
-  version = "09092016-${stdenv.lib.strings.substring 0 7 rev}";
-  rev = "e7c7c3cb214fd06df63530a4e861210e7a0a1b6c";
-
+  name = "c14-cli-2016-09-09";
   goPackagePath = "github.com/online-net/c14-cli";
   subPackages = [ "cmd/c14" ];
-
-  src = fetchgit {
-    inherit rev;
-    url = "https://github.com/online-net/c14-cli";
+
+  src = fetchFromGitHub {
+    owner = "online-net";
+    repo = "c14-cli";
+    rev = "e7c7c3cb214fd06df63530a4e861210e7a0a1b6c";
     sha256 = "1k53lii2c04j8cy1bnyfvckl9fglprxp75cmbg15lrp9iic2w22a";
   };

See more on #19156


Access denied finding property "vendor.camera.aux.packagelist"

System information

OS Platform and Distribution (e.g., Linux Ubuntu 20.04, WSL2):  Windows10
Target (e.g. desktop cpu, android arm64): Android 9pkq1
Bazel version::3.7.2
Python version:3.9.7
GCC/G++ version:
Unity version:2020.3.8f1
Android NDK version (if building for Android):19.0.5232133

We just build in android: the permission pop up appears but after accept it the camera never work. We had looking in other webs but... nothing :( Any idea? thanks a lot!
LOGCAT:
2021/09/21 07:39:50.793 18707 18834 Info CameraManagerGlobal Connecting to camera service
2021/09/21 07:39:50.798 18707 18834 Error libc Access denied finding property "vendor.camera.aux.packagelist"
2021/09/21 07:39:50.798 18707 18834 Error libc Access denied finding property "vendor.camera.aux.packagelist.ext"
2021/09/21 07:39:50.798 18707 18834 Error libc Access denied finding property "vendor.camera.aux.packagelist"
2021/09/21 07:39:50.785 18707 18707 Warn UnityMain type=1400 audit(0.0:91664): avc: denied { read } for name="u:object_r:vendor_camera_prop:s0" dev="tmpfs" ino=11546 scontext=u:r:untrusted_app:s0:c249,c256,c512,c768 tcontext=u:object_r:vendor_camera_prop:s0 tclass=file permissive=0
2021/09/21 07:39:50.798 18707 18834 Error libc Access denied finding property "vendor.camera.aux.packagelist.ext"
2021/09/21 07:39:50.785 18707 18707 Warn UnityMain type=1400 audit(0.0:91665): avc: denied { read } for name="u:object_r:vendor_camera_prop:s0" dev="tmpfs" ino=11546 scontext=u:r:untrusted_app:s0:c249,c256,c512,c768 tcontext=u:object_r:vendor_camera_prop:s0 tclass=file permissive=0
2021/09/21 07:39:50.798 18707 18834 Error libc Access denied finding property "vendor.camera.aux.packagelist"
2021/09/21 07:39:50.785 18707 18707 Warn UnityMain type=1400 audit(0.0:91666): avc: denied { read } for name="u:object_r:vendor_camera_prop:s0" dev="tmpfs" ino=11546 scontext=u:r:untrusted_app:s0:c249,c256,c512,c768 tcontext=u:object_r:vendor_camera_prop:s0 tclass=file permissive=0
2021/09/21 07:39:50.798 18707 18834 Error libc Access denied finding property "vendor.camera.aux.packagelist.ext"
2021/09/21 07:39:50.798 18707 18834 Warn CameraManagerGlobal [soar.cts] ignore the status update of camera: 20
2021/09/21 07:39:50.799 18707 18834 Error libc Access denied finding property "vendor.camera.aux.packagelist"
2021/09/21 07:39:50.799 18707 18834 Error libc Access denied finding property "vendor.camera.aux.packagelist.ext"
2021/09/21 07:39:50.799 18707 18834 Warn CameraManagerGlobal [soar.cts] ignore the status update of camera: 21
2021/09/21 07:39:50.799 18707 18834 Error libc Access denied finding property "vendor.camera.aux.packagelist"
2021/09/21 07:39:50.799 18707 18834 Error libc Access denied finding property "vendor.camera.aux.packagelist.ext"
2021/09/21 07:39:50.799 18707 18834 Warn CameraManagerGlobal [soar.cts] ignore the status update of camera: 22
2021/09/21 07:39:50.799 18707 18834 Error libc Access denied finding property "vendor.camera.aux.packagelist"
2021/09/21 07:39:50.799 18707 18834 Error libc Access denied finding property "vendor.camera.aux.packagelist.ext"
2021/09/21 07:39:50.799 18707 18834 Warn CameraManagerGlobal [soar.cts] ignore the status update of camera: 61
2021/09/21 07:39:50.799 18707 18834 Error libc Access denied finding property "vendor.camera.aux.packagelist"
2021/09/21 07:39:50.799 18707 18834 Error libc Access denied finding property "vendor.camera.aux.packagelist.ext"
2021/09/21 07:39:50.799 18707 19584 Error libc Access denied finding property "vendor.camera.aux.packagelist"
2021/09/21 07:39:50.799 18707 19584 Error libc Access denied finding property "vendor.camera.aux.packagelist.ext"
2021/09/21 07:39:50.800 18707 19584 Error libc Access denied finding property "vendor.camera.aux.packagelist"
2021/09/21 07:39:50.800 18707 19584 Error libc Access denied finding property "vendor.camera.aux.packagelist.ext"
2021/09/21 07:39:50.800 18707 19584 Warn CameraManagerGlobal ignore the torch status update of camera: 21
2021/09/21 07:39:50.800 18707 19584 Error libc Access denied finding property "vendor.camera.aux.packagelist"
2021/09/21 07:39:50.800 18707 19584 Error libc Access denied finding property "vendor.camera.aux.packagelist.ext"
2021/09/21 07:39:50.800 18707 19584 Warn CameraManagerGlobal ignore the torch status update of camera: 22
2021/09/21 07:39:50.811 18707 18834 Debug StreamConfigurationMap setFlag Flag:1
2021/09/21 07:39:50.815 18707 18834 Debug StreamConfigurationMap setFlag Flag:0
2021/09/21 07:39:50.817 18707 18834 Info Unity WebCamDevice Changed: Camera 0
2021/09/21 07:39:50.817 18707 18834 Info Unity WebCamDeviceSelectorController:OnValueChanged(Dropdown)
2021/09/21 07:39:50.817 18707 18834 Info Unity <Start>d__2:MoveNext()
2021/09/21 07:39:50.817 18707 18834 Info Unity UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
2021/09/21 07:39:50.817 18707 18834 Info Unity
2021/09/21 07:39:50.825 18707 18834 Debug StreamConfigurationMap setFlag Flag:1
2021/09/21 07:39:50.830 18707 18834 Error libc Access denied finding property "persist.vendor.camera.privapp.list"
2021/09/21 07:39:50.856 18707 19762 Error libc Access denied finding property "vendor.camera.aux.packagelist"
2021/09/21 07:39:50.856 18707 19762 Error libc Access denied finding property "vendor.camera.aux.packagelist.ext"
2021/09/21 07:39:50.857 18707 19762 Error libc Access denied finding property "vendor.camera.aux.packagelist"
2021/09/21 07:39:50.857 18707 19762 Error libc Access denied finding property "vendor.camera.aux.packagelist.ext"
2021/09/21 07:39:51.735 18707 18834 Info IL2CPP Locale es-ES
2021/09/21 07:39:51.736 18707 18834 Info Unity WebCamTexture Graphics Format: 88
2021/09/21 07:39:51.736 18707 18834 Info Unity <ResetScreen>d__20:MoveNext()
2021/09/21 07:39:51.736 18707 18834 Info Unity UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
2021/09/21 07:39:51.736 18707 18834 Info Unity SceneDirector:ResetCamera(Nullable`1)
2021/09/21 07:39:51.736 18707 18834 Info Unity SceneDirector:ChangeWebCamDevice(Nullable`1)
2021/09/21 07:39:51.736 18707 18834 Info Unity WebCamDeviceSelectorController:OnValueChanged(Dropdown)
2021/09/21 07:39:51.736 18707 18834 Info Unity <Start>d__2:MoveNext()
2021/09/21 07:39:51.736 18707 18834 Info Unity UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
2021/09/21 07:39:51.736 18707 18834 Info Unity
2021/09/21 07:39:52.256 18707 18834 Error Unity Unable to open archive file: jar:file:///data/app/meifus.videomanual.com-xd_RrU7YK_5-svu7daiXSg==/base.apk!/assets/mediapipe
2021/09/21 07:39:52.256 18707 18834 Error Unity Mediapipe.AssetBundleManager:LoadAssetBundle()
2021/09/21 07:39:52.256 18707 18834 Error Unity Mediapipe.AssetBundleManager:PrepareAsset(String, String, Boolean)
2021/09/21 07:39:52.256 18707 18834 Error Unity HandTrackingGraph:PrepareDependentAssets()
2021/09/21 07:39:52.256 18707 18834 Error Unity DemoGraph:Initialize()
2021/09/21 07:39:52.256 18707 18834 Error Unity <RunGraph>d__24:MoveNext()
2021/09/21 07:39:52.256 18707 18834 Error Unity UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
2021/09/21 07:39:52.256 18707 18834 Error Unity
2021/09/21 07:39:52.256 18707 18834 Error Unity [ line<PHONE_NUMBER>]
2021/09/21 07:39:52.256 18707 18834 Error Unity
2021/09/21 07:39:52.257 18707 18834 Error Unity Failed to read data for the AssetBundle 'mediapipe'.
2021/09/21 07:39:52.257 18707 18834 Error Unity Mediapipe.AssetBundleManager:LoadAssetBundle()
2021/09/21 07:39:52.257 18707 18834 Error Unity Mediapipe.AssetBundleManager:PrepareAsset(String, String, Boolean)
2021/09/21 07:39:52.257 18707 18834 Error Unity HandTrackingGraph:PrepareDependentAssets()
2021/09/21 07:39:52.257 18707 18834 Error Unity DemoGraph:Initialize()
2021/09/21 07:39:52.257 18707 18834 Error Unity <RunGraph>d__24:MoveNext()
2021/09/21 07:39:52.257 18707 18834 Error Unity UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
2021/09/21 07:39:52.257 18707 18834 Error Unity
2021/09/21 07:39:52.257 18707 18834 Error Unity [ line<PHONE_NUMBER>]
2021/09/21 07:39:52.257 18707 18834 Error Unity
2021/09/21 07:39:52.325 18707 18834 Error Unity NullReferenceException: Object reference not set to an instance of an object.
2021/09/21 07:39:52.325 18707 18834 Error Unity   at Mediapipe.AssetBundleManager.PrepareAsset (System.String name, System.String uniqueKey, System.Boolean overwrite) [0x00000] in <00000000000000000000000000000000>:0
2021/09/21 07:39:52.325 18707 18834 Error Unity   at HandTrackingGraph.PrepareDependentAssets () [0x00000] in <00000000000000000000000000000000>:0
2021/09/21 07:39:52.325 18707 18834 Error Unity   at DemoGraph.Initialize () [0x00000] in <00000000000000000000000000000000>:0
2021/09/21 07:39:52.325 18707 18834 Error Unity   at SceneDirector+<RunGraph>d__24.MoveNext () [0x00000] in <00000000000000000000000000000000>:0
2021/09/21 07:39:52.325 18707 18834 Error Unity   at UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) [0x00000] in <00000000000000000000000000000000>:0

I think this is same as https://github.com/homuler/MediaPipeUnityPlugin/issues/119

Mmh. not pretty sure what I have to do. I tried removing the metas but is not working.

I tried removing the metas but is not working.

Not removing.
You need to run git checkout Packages/com.github.homuler.mediapipe/Runtime/Resources/ , restart UnityEditor, and build the app again.
If you build the app successfully, AssetBundle file will be built automatically, and Assets/StreamingAssets/mediapipe will be created.


issues in some docstrings for giddy

Hey, dynamic_lisa_composite_explore looks like it's not working? Does this build incorrectly for anyone else?

Also, line 111 renders as code rather than text, since it does not have a blank line above it.

dynamic_lisa_composite_explore() relies on Ipywidget functionality and opening the example in a notebook. This should be outlined in a note above the example. I added a # on the plt.show() call so it builds no image at all.
I also fixed all minor bugs in #25.

#25 was merged so this can be closed.


Implement GIT_DUMPER, GIT_LOADER and improve errors when loading files

Description
New PlatformType.GIT
This PRs implements a new PlatformType for git files (GIT_DUMPER and GIT_LOADER).
This new dumper is used to make sure we write readOnly fields in this dumper.
Changes in FileAttribMaker
Now we always get files from the external source, file fields are never assumed to contain the data.
We support now 3 ways if discovering the url to fetch the data:

If the file field has some value that will be used to compute the url
If we have a field referenced in x-filename or x-checksum use it to compute the url
else if we have a name, the url path will be name/fieldname
else if we have an id, the url path will be the id/fieldname
any other case we raise an error.

This requires changes in the controller spec API.
Checklist

[x] Bump .version and .appVersion in k8s/crd/Chart.yaml
[x] Bump .version and .appVersion in k8s/operator/Chart.yaml

I think we need to update the CRD schema so Kubernetes would accept the new special fields we added x-filename and x-checksum

Those are not saved in the entity yaml file, they are part of the yaml API specficication, the parser reads them and knows what fields are used to compute the path for the file url


Cannot send message to @KafkaListener from test unless test thread sleeps beforehand

I have the following test which simply submits a random UUID to an embedded kafka topic:
@SpringBootTest
@DirtiesContext
@EmbeddedKafka(partitions = 1, brokerProperties = { "listeners=PLAINTEXT://localhost:9099", "port=9099" })
public class KafkaConsumerTest {
    private static final Logger logger = LoggerFactory.getLogger(KafkaConsumerTest.class);

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    @Autowired
    private KafkaListenerEndpointRegistry registry;

    @Value("${kafka.fulfillment-topic}")
    private String topic;

    @Test
    public void messageListenerShouldReceiveMessage() throws InterruptedException, ExecutionException {
        var latch = new CountDownLatch(1);

//        Thread.sleep(5000);
        var uuid = UUID.randomUUID().toString();
        logger.info("## {}", uuid);

        var result = kafkaTemplate.send(topic, uuid);
        var fR = result.get().getRecordMetadata();
        logger.info("Sent {} to topic={}, partition={}", uuid, fR.topic(), fR.partition());

        Assertions.assertThat(latch.await(10, TimeUnit.MINUTES)).isTrue();
    }
}

I also have this listener setup that simply logs the message
@KafkaListener(id = "fulfillment-listener", topics = "${kafka.fulfillment-topic}", groupId = "fulfillment-group")
public void listenForFulfillment(ConsumerRecord<?, String> consumerRecord, Acknowledgment acknowledgment) {
    acknowledgment.acknowledge();
    logger.info("[topic={} partition={}] Got kafka message: {}", consumerRecord.topic(), consumerRecord.partition(), consumerRecord.value());
}

Without the Thread.sleep(5000); (or debugging with a breakpoint in the test and waiting for a few seconds) in the test, the UUID will never reach the kafka listener. This is happening for all my tests involving this kind of setup. I have tried debugging and looking into the internals but I simply can't work it out.

First of all, you're completely right I'll make no excuses, I didn't read the issue template and so please accept my apologies for that, I'll make sure to read it and adhere to it from now on.

What counts down the latch; seems to me the test will always fail.
This was intentional for now, since the message was never being received I was using it so simply wait a long period of time so I could watch the logs.

Adding the sleep avoids that, but change auto.offset.reset to earliest to receive any records sent before the consumer subscribes to the topic.
This was indeed the issue, thank you so much. I've been trying to figure this one out for over two day now, and again apologies for not reading the issue template.


Upload images from a local source to imgix

This is a potential feature for this library. If you want to see us build this, please indicate your support by reacting to this with a 👍
User Story
I have an image stored locally that I want to upload to image-manager and render.
Description
This would offload the processing of images stored on a developer's machine to imgix. This results in much lower build times and also allows the developer to take advantage of imgix's larger suite of transformations, better responsiveness, and best-in-class CDN.

Closing as there hasn't been much movement here but I have noted this feature in our backlog.


Interfaces

Here's an example of how we could (potentially) represent the validation schema outlined in the shared Google doc by way of TypeScript interfaces.
Open questions for discussion:

The ability to specify either a value (e.g. required: true) or an object (required: { value: true, failureMessage: "This is a custom error" }) is certainly convenient from a user perspective but it also complicates our interface definitions. I assume this is okay, but it seems worth confirming.
I collapsed the constraints field because it felt unnecessary/redundant. Is this okay?

The below interfaces describe a validation object like this:
var validationSchema = {
  fields: {
    username: {
      required: true,
      pattern: {
        value<EMAIL_ADDRESS>        failureMessage: "%s is not a valid email format"
    },
    username: {
      required: true,
      min: 6,
      pattern: /[0-9]/
    }
  }
};

I like it!
Feels nice and sleek. I'm not up to speed on typescript yet so I might be missing the point here, but I'm wondering a bit on how we should handle types.
So the scenario I'm thinking of is something like your example above:
var validationSchema = {
  username: { min: 6 }
}

Let's say we have an ordinary input for this, if the view adaptor is angular and we told it to be a text we get a string to validate against, but if its a `input type="number"` we get a number.

This can be fine, we effectively move the type transformation and therefore the validation of type to the view adaptor and what `min` actually validates is dependent on the view form. But it would mean that some validation isn't in the core validation function, but in the view which becomes a bit messy.

The other way to do it I guess would be to make type in the validation schema a none optional value (or optional with default) and let form-js be able to type coerce and check if needed.This on the other hand could end up doubling type coercion/transformation in some cases. (I do this in schema form and to properly support all type validations I had to validate the `ngModel.$viewValue` instead of the model)

Hey David! Thanks for the feedback. FWIW I'm working on an example validator function (for required) to illustrate how we could potentially consume the above interfaces.
As for your question... I'm still pretty on the fence about type validations, to be honest. (Coercing and) validating primitive types is easy. Objects is a lot more difficult. I'm half-tempted to say we consider only supporting primitive type validation. Either way- I had envisioned type validations being optional. (That's what the ? signifies).
I think one of our goals should be to keep all of the validation logic inside of the vanilla Forms JS library.

Cool, looking forward to it.
Yeah I think basic types only is a good idea, I don't think its necessary to support objects, arrays etc They are more suited to custom validation functions. But I do think we need to support integers and step values on numbers and date.
I'm not sure type can be optional, but I think that will clear itself up in the future.

Okay. I'll update the interface definitions shortly! And by "optional" I mean "defaults to string" :)

nice!

I may be misunderstanding what you mean when you say "validating objects" but if you're saying what I think you're saying then this would be an awesome place to bring in api-check (is there an echo in here?) where validation of objects is trivial:
apiCheck.shape({
  name: checkers.shape({
    first: checkers.string,
    last: checkers.string
  }),
  age: checkers.number,
  isOld: checkers.bool,
  walk: checkers.func,
  childrenNames: checkers.arrayOf(checkers.string)
})({
  name: {
    first: 'Matt',
    last: 'Meese'
  },
  age: 27,
  isOld: false,
  walk: function() {},
  childrenNames: []
}); // <-- pass

apiCheck.shape({
  mint: checkers.bool,
  chocolate: checkers.bool
})({mint: true}); // <-- fail

There's a lot more to the api that makes it very very flexible.

@davidlgj I've updated the interfaces and added an example Validator and RequiredValidator. (Baby steps for discussion.)
@kentcdodds Looks like using apiCheck for type-validation could potentially work. Two concerns though:

Using apiCheck would be adding a dependency on a small library. (No offense meant, just that the smaller the library, the greater the chance of future abandonment.) This concern is probably mitigated by the fact that you're the author though. :)
The way I am currently envisioning types- something that we should actually discuss- is not just validation, but validation and implicit coercion. For instance, if you tell Forms JS that field "foo" is an integer then we should validate that it only contains digits (0-9). I was also assuming we'd handle the case where a particular field-template didn't specify an input-type and the resulting value we got was a string that contained an integer.
I was also assuming we'd let users specify integer or float whereas I think apiCheck may only support number? (This could be changed, I'm sure.)

I like this architecture! I think we're on right track.
One minor reminder, the fields should be an array so we can have fixed order of fields

Thanks @mohsen1!
Actually a couple of us chatted last Thursday and I believe (@davidlgj, @mchapman, correct me if I'm wrong) we agreed that the benefits of making fields an object outweighed the benefits of ordering. The primary motivators for this decision were:

It's easier to override shared validation rules for specific forms.
It's easier/faster in general to lookup validation rules for a specific field if it's a map.

This changeset just illustrates the validation schema. What isn't (yet) represented here is what we started calling the view or layout schema - which controls the ordering of fields. I'm not sure if we settled on a proposal for how that ordering should be specified. Seems like something we should all chat about?

Good concerns @bvaughn. First, I don't know what to say about the future of any library. One thing that I can tell you is that we are using this library heavily at work and angular-formly is also using this library. So I'm pretty committed to its success.
It wont do anything for us with respect to coercion.
I'm not sure I understand what you mean when you say:

if we have to validate individual fields already

Could you explain what you mean?

Sure thing @kentcdodds. Please correct me if I make any wildly inaccurate statements or assumptions here.. :)
Forms JS supports multiple validation rules. Data type is only one of these conditions. In order to support other, non-type validations (min/max length/size, pattern, required, etc) we need to look at individual field values. (One way we could do this is shown in this pull request.) I believe apiCheck is concerned solely with data type so while it may be useful for validating the structure of an object- using it would not prevent us from having to iterate over individual fields of the other validate criteria. If we're already doing lower-level validations (e.g. min value) then it's easy to also check the type while we're working at the level of an individual field.
We also need to return validation errors at the field-level (not form-level) so we can show inline validation errors. I think apiCheck deals more at the form-data-level than the field level, so we'd have to merge the 2 sources of potential failure...

@bvaughn so you're planning to not use JSON Schema and use a secondary object to describe the schema. Those are the main reasons I moved away from angular-schema-form. For a dynamic app like Swagger Editor, this was a problem since I had to construct a "view" schema for each schema programmatically.
My suggestion is, if we want to use a JavaScript object for schema, we should use JSON Schema semantics and don't invent a new format.

I see what you're saying. Yes, built-in stuff with apiCheck right now mostly deals with types. However, adding custom types is quite easy (as you can see in the example with the ipAddress type).
In fact, I should probably add a matches for regular expressions (which is a fairly large use case for validating values). Then there could be some types added for min/max and length/size.
Required is also handled because when using the shape type, everything you specify (that uses a built-in checker) is automatically required unless you add .optional to the definition. Custom types could do whatever they like (return an error if it's not specified, or simply let it go).
So, I hadn't thought about this, but users could actually specify an apiCheck object (call it whatever you want) which we could use as the validation api. It could be used to validate the entire model (not just a single field). I think that the API is clean and battle tested (it's very similar to React's PropTypes used by thousands of React developers all over the world for years now).
Whether we do this with formly-js or not, I think I'm going to be implementing this is angular-formly for sure. So this conversation was productive :-)

@mohsen1, I think you're illustrating the very reason that accomplishing this forms-js library as the end all be all solution for forms in JavaScript will be very difficult/impossible. It's just not possible to have one solution to something with such a broad use case reach. I think that forms-js can cover most use cases, but if it tries to cover all use cases, the api will be too complicated for it to be very useful.

@mohsen1 Ah, I thought we had already discussed this in a previous issue and more or less reached agreement. In short- Forms JS validations are neither a subset nor a superset of JSON schema. There is some overlap, but supporting the full JSON schema would be too much work and likewise, extending JSON schema would (in my opinion) be just as much work because of the need to merge validation results from a JSON schema validator and Forms JS custom validations. I feel strongly that the right approach here is for us to write a library that consumes JSON schema and turns it into Forms JS validation. We can video chat about this more though if there's still concern? I think it's important that we reach a consensus here.
@kentcdodds Cool, apiCheck sounds pretty powerful. Maybe even something we should run on an incoming Forms JS validation schema? (I think you've proposed this in the past, and I'm totally game - provided it doesn't negatively impact performance.) I personally think it would be nice if our validation approach was consistent (principle of least surprise) rather than half apiCheck and half something else. So for that reason primarily, I'd initially be slightly opposed to using apiCheck to validate field-type.

@bvaughn, totally agree with the principle of least surprise. As far as performance goes, it definitely impacts performance, though I've never benchmarked it. However, just like with React's PropTypes, you generally want to disable it in prod (which is quite easy).

@kentcdodds Ah, right. I recall you suggesting as much before. In that case, I'm totally on-board. I'm kind of envisioning that we'll build a debug version of Forms JS and a production ready, minified/uglified version. In the debug version, using apiCheck to do run-time analysis of incoming form and layout schema objects sounds pretty useful. Let's do it. :)

With the power of webpack and uglify, we could even have it totally remove all the apiCheck code as well and save on some bytes :-) This is similar to what React does (except they don't use webpack, but some of their own home-grown stuff).

Here's a thought: one thing we could consider that would address my concern about additional complexity of using TypeScript's union types while simultaneously making our validation schema easier to validate using apiCheck.js...
Instead of a validation schema that looked like this:
var validationSchema = {
  fields: {
    simpleRequired: {
      required: true
    },
    advancedRequired: {
      required: {
        rule: true,
        failureMessage: "This is a custom required failure message"
      }
    }
  }
};

Maybe we could approach it like this:
var validationSchema = {
  fields: {
    aRequiredField: {
      required: true,
      requiredFailureMessage: "This is a optional, custom required failure message"
    }
  }
};

Then we could validate our schema with apiCheck more or less like this:
apiCheck.shape({
  required: apiCheck.boolean.optional,
  requiredFailureMessage: apiCheck.string.optional,
  // Other validation types here...
})(validationRules);

Just updated the pull request with an example of what I'm talking about above. (Can always revert if you guys dislike it!)

In light of our Hangout and email decisions to move forward with some prototypes, I'm going to merge this branch. If we decide to throw it out in favor of a different type of validation in the future- that's fine. At least it makes a starting point.
When our wrapper libraries (Angular, web component, etc.) detect a change in field-values, they can validate with ValidationService.validateField(fieldName, formData, validationSchema) and the resulting Promise will inform the of the validation status. I think this will be enough for now to get us moving forward with the adapters while I flesh out some more of the validators.


Add fbeta_metrics

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (d2af907) to head (4219f5a).

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #129   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           42        43    +1
  Lines          920       976   +56
  Branches       106       115    +9
=========================================
+ Hits           920       976   +56

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


accelerator again: parameter reset after release intended?

One more thing regarding the accelerator: Is it intended that after releasing it while continuing to hold a move button, the speed and acceleration values are reset (i.e. the cursor starts moving as if one released the move button as well and pressed it again)? Before submitting today's pull request, I was using my earlier implementation (temporary speed override while accelerating in the background) and now the sudden “brake”-effect after releasing the accelerator feels a bit unnatural.
I think it would be sensible as well to have the following behaviour:
When releasing the accelerator, the acceleration value is set to the value it would be if the accelerator had never been pressed and the currently reached cursor speed is kept.
I hope it is understandable what I mean, otherwise please let me know.

This is by design.
One could argue it is less natural in the sense that physical objects don't behave that way, but I would argue it is more useful given that the intended use is allowing the user to intermittently accelerate the pointer whilst doing some local manipulation.
It is better to think of the 'accelerator' as a speed toggle which happens to have an acceleration curve.


Add add_local_campaign example

Is this a draft or working example?

This example is working now -> if the CID is connected with a GMB

Is this example runnable more than once? Will there be any errors of the asset service?

This is a good question. Yes it can run multiple times, and there's no error from the AssetService when uploading the same image/video more than once with the same name.
Actually the image has been uploaded by other code examples such as add_smart_display_ad,
add_gmail_ad. The AssetService returns the same resource name if the image is uploaded already; but it fails if you try to specify a different asset name.
This is a lesson learnt for me when developing the FT application.

  # Optimization goal setting is mandatory for Local campaigns. This example
  # selects driving directions and call clicks as goals.


Examples + vike-vue: use composition API to support composition-only build of vue

Description
Something I stumbled upon some time ago:
The examples and vike-vue use the options API to define the root component & integrate it with routing: https://github.com/vikejs/vike/blob/main/examples/vue-full/renderer/app.ts
Because of this, the app breaks when you set _VUE_OPTIONS_API_: false, a compile-time flag to exclude code related to the options API and reduce bundle size. Using the composition API is just a tiny change (using a ref instead of data() for this.Page), happy to submit a PR :)

happy to submit a PR :)

I was actually about to ask whether that's something you'd be up for :)
Looking forward to it.

This repo uses composition API: https://github.com/royalswe/vike-fastify-boilerplate/blob/main/renderer/app.ts
Feel free to copy code if needed.


Test sdk integration

CI Vulkan-ExtensionLayer build queued with queue ID 57340.

CI Vulkan-ExtensionLayer build queued with queue ID 57340.


fixing windows file follower to include regex delimiting and adding a new trim flag

this also addresses https://github.com/gravwell/gravwell/issues/899