[Bug] Duplicate Rows in shopify_holistic_reporting__daily_customer_metrics

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the issue
In certain scenarios the join/coalesce logic in the combine_histories CTE is resulting in duplicated data. Consider the following (I have removed aggregate columns for brevity):
The result set from the klaviyo_daily CTE contains the following rows:
DATE_DAY   EMAIL                  LAST_TOUCH_CAMPAIGN_ID LAST_TOUCH_FLOW_ID CAMPAIGN_NAME FLOW_NAME               VARIATION_ID CAMPAIGN_SUBJECT_LINE CAMPAIGN_TYPE SOURCE_RELATION SUM_REVENUE_PLACED_ORDER COUNT_PLACED_ORDER COUNT_OPENED_EMAIL
---------- ---------------------- ---------------------- ------------------ ------------- ----------------------- ------------ --------------------- ------------- --------------- ------------------------ ------------------ ------------------
2023-04-12<EMAIL_ADDRESS>  (null)                 xyz123             (null)        Abandoned Checkout      (null)       (null)                (null)                        58                       1                  3
2023-04-12<EMAIL_ADDRESS>  (null)                 abc789             (null)        YEB - New Customer Flow (null)       (null)                (null)                        0                        0                  1

The result set from the shopify_daily CTE contains:
DATE_DAY   EMAIL                  LAST_TOUCH_CAMPAIGN_ID LAST_TOUCH_FLOW_ID LAST_TOUCH_CAMPAIGN_NAME LAST_TOUCH_FLOW_NAME TOTAL_ORDERS TOTAL_PRICE
---------- ---------------------- ---------------------- ------------------ ------------------------ -------------------- ------------ -----------
2023-04-12<EMAIL_ADDRESS>  (null)                 xyz123             (null)                   Abandoned Checkout   1            57.86

Because the shopify row has a flow id and the coalesce gives precedence to shopify we end up with two rows both with the shopify order related flow even though the other flow from klaviyo has no order:
DATE_DAY   EMAIL                  CAMPAIGN_ID FLOW_ID CAMPAIGN_NAME FLOW_NAME          shopify_TOTAL_ORDERS shopify_TOTAL_PRICE klaviyo_SUM_REVENUE_PLACED_ORDER klaviyo_COUNT_PLACED_ORDER klaviyo_COUNT_OPENED_EMAIL
---------- ---------------------- ----------- ------- ------------- ------------------ -------------------- ------------------- -------------------------------- -------------------------- --------------------------
2023-04-12<EMAIL_ADDRESS>  (null)      xyz123  (null)        Abandoned Checkout 1                    57.86               58                               1                          3
2023-04-12<EMAIL_ADDRESS>  (null)      xyz123  (null)        Abandoned Checkout 1                    57.86               0                                0                          1

As expected the dbt_utils_unique_combination_of_columns_shopify_holistic_reporting__daily_customer_metrics_date_day__email__klaviyo_source_relation__shopify_source_relation__campaign_id__flow_id__variation_id test fails

Relevant error log or model output
No response
Expected behavior
I would expect 2 rows but having different flows:
DATE_DAY   EMAIL                  CAMPAIGN_ID FLOW_ID CAMPAIGN_NAME FLOW_NAME                shopify_TOTAL_ORDERS shopify_TOTAL_PRICE klaviyo_SUM_REVENUE_PLACED_ORDER klaviyo_COUNT_PLACED_ORDER klaviyo_COUNT_OPENED_EMAIL
---------- ---------------------- ----------- ------- ------------- ------------------       -------------------- ------------------- -------------------------------- -------------------------- --------------------------
2023-04-12<EMAIL_ADDRESS>  (null)      xyz123  (null)        Abandoned Checkout       1                    57.86               58                               1                          3
2023-04-12<EMAIL_ADDRESS>  (null)      abc789  (null)        YEB - New Customer Flow  0                    0                   0                                0                          1

dbt Project configurations
name: 'chabi'
version: '1.0.0'
config-version: 2
profile: "{{ env_var('DBT_PROFILE', '') }}"
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]
target-path: "target"
clean-targets:
  - "target"
  - "dbt_packages"
models:
  klaviyo:
    +schema:
    intermediate:
      +schema: stg
  klaviyo_source:
    +schema: stg
  shopify:
    +schema:
  shopify_source:
    +schema: stg
  shopify_holistic_reporting:
    +schema:
    intermediate:
      +schema: stg

vars:
  klaviyo_schema: klaviyo
  shopify_schema: shopify

Package versions
packages:
  - package: dbt-labs/dbt_utils
    version: [">=1.0.0", "<2.0.0"]
  - package: dbt-labs/codegen
    version: 0.9.0
  - package: fivetran/klaviyo
    version: 0.5.0
  - package: fivetran/shopify
    version: 0.7.0
  - package: fivetran/shopify_holistic_reporting
    version: 0.3.0

What database are you using dbt with?
snowflake
dbt Version
Core:
  - installed: 1.4.1
  - latest:    1.4.5 - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - snowflake: 1.4.0 - Update available!

  At least one plugin is out of date or incompatible with dbt-core.
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Additional Context
No response
Are you willing to open a PR to help address this issue?

[X] Yes.
[ ] Yes, but I will need assistance and will schedule time during our office hours for guidance
[ ] No.

hey @jmussitsch 👋 thanks for explaining the issue so thoroughly! it's clear that we need to add some logic to de-dupe these instances
in your example, would you expect aggregate columns for flow xyz123 on this day to be summed up together across the two platforms? or would you elect to choose shopify vs klaviyo aggregates specifically?
i'm thinking we need to add a hefty and dynamic group by after joining the shopify and klaviyo daily CTEs and sum up the aggregate metrics

thanks for updating, that does make things clearer! still trying to recreate the issue in our environment and test this out, but i'm wondering if we just need to add the flows/campaigns/variations to the full outer join between the 2 CTEs, since they are part of the grain of this model
so like
    from shopify_daily
    full outer join klaviyo_daily
        on lower(shopify_daily.email) = lower(klaviyo_daily.email)
        and shopify_daily.date_day = klaviyo_daily.date_day
        and shopify_daily.last_touch_campaign_id = klaviyo_daily.last_touch_campaign_id
        and shopify_daily.last_touch_flow_id = klaviyo_daily.last_touch_flow_id
        and shopify_daily.last_touch_variation_id = klaviyo_daily.variation_id

@jmussitsch  if you have time would you mind trying out the following branch? it includes the above join logic
# packages.yml
packages:
  - git: https://github.com/fivetran/dbt_shopify_holistic_reporting.git
    revision: scope/issue-14

i have other ideas if this doesn't work... 🤔

@fivetran-jamie I tried your fix but I think the issue is that wont treat columns where both values are null as equal for the full outer join... since in most databases NULL = NULL is NULL not true. Because of this I get extra rows.
I am on Snowflake and don't author packages for use across databases so not sure how you guys typically handle null safe equality. For quick testing purposes I did this and got the results I would expect:
  and coalesce(shopify_daily.last_touch_campaign_id, '') = coalesce(klaviyo_daily.last_touch_campaign_id, '')
  and coalesce(shopify_daily.last_touch_flow_id, '') = coalesce(klaviyo_daily.last_touch_flow_id, '')
  and coalesce(shopify_daily.last_touch_variation_id, '') = coalesce(klaviyo_daily.variation_id, '')

ah yeah we definitely need those coalesces!! BigQuery really doesn't likenull = null either.
i just pushed your fix  to the working branch of the package if you have time to re-dbt deps and run again as a sanity check @jmussitsch

@fivetran-jamie I ran this against our data and did a few checks and it looks good

great to hear! we'll be pushing a release of the package with this fix in it next sprint (starts tomorrow). there are just some other package updates we should/need to batch with this one

hey @jmussitsch we've just released this in v0.4.0! this release also includes some upstream breaking changes from shopify v0.8.* FYI

@fivetran-jamie great thanks!


Trigger wallet sync after import or recovery

After importing a new wallet or recovering a wallet using a backup, automatically trigger a wallet sync.

Another related issue is that when a new wallet is created and control returns to the homepage, the imported wallet data is not synced and going into wallet settings shows information about an older wallet.
Test this by importing a wallet and then going to settings and checking the backup  - the seed ill be from another wallet.
To fix this the user has to swipe to the older wallet and swipe back, only then the new wallet data is properly loaded.
@mocodesmo

these are all fixed in the 0.1.6 release
@i5hi pls verify

You're right @mocodesmo
This is fixed! Cheers!


brief product description: idea, goals, expectations,

What is yy prime-exchange?
Action items:

[ ] idea
[ ] goals
[ ] expectations
[ ] who are the target audience
[ ] who are the competitors
[ ] what are the technical requirements
[ ] what type of an application
[ ] what are the application components (i.e. dashboard, social media integration, etc.)
[ ] delivery time

Resources

Convergence - Serum & Wormhole Hackathon

Problem statement:
"Serum trading strategies to provide liquidity w/in EMA-defined bands"

serum bots (aka agents) that implement a specific strategy
user deposited funds, user chosen parms (ema, depth etc)
Pyth ref (ema)
Crosschain Orderbook UI accessing Solana && Ethereum ie wormhole
Effects (ie "should have")
++ AMM (const product token management)
++ AMM rebalancing mechanism
++ pool creation/management
++ usd [other conversions?] pricing ~ pyth
++ sensible and reduced parms
++ dApp interface
Aesthetics ("nice to have")
+++ dashboard metrics / stats
+++ testnet simulator
+++


SW-5318 Add update functionality for ParticipantProjectSpecies

Add update function to ParticipantProjectSpeciesStore
Add controller endpoint for updating a participant project species

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#1935
#1934  👈
#1933
#1932
#1924
#1919
#1918
#1917
#1916
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @nickgraz and the rest of your teammates on  Graphite

Merge activity

May 8, 12:09 PM EDT: @nickgraz started a stack merge that includes this pull request via Graphite.


❓ [Question] My aten::chunk op converter does not work correctly

❓ Question
I want to use trtorch to compile shufflenet. However, aten::chunk op is not currently supported. I wrote a converter implementation referring to converters/impl/select.cpp. Unfortunately, it does not work.
Is there anything wrong?
The python code
model = torchvision.models.shufflenet_v2_x1_0(pretrained=True).cuda().eval()
input_data = torch.randn(1, 3, 224, 224).cuda()
scripted_model = torch.jit.script(model)
out = scripted_model(input_data)

compile_settings = {
    "input_shapes": [(1, 3, 224, 224)],
}

trt_ts_module = trtorch.compile(scripted_model, compile_settings)

The converter I wrote (compared with the converter of aten::select, I only changed numOuputs and sizes)
auto chunk_registrations TRTORCH_UNUSED =
    RegisterNodeConversionPatterns()
        .pattern({"aten::chunk(Tensor(a) self, int chunks, int dim=0) -> (Tensor[])",
                  [](ConversionCtx* ctx, const torch::jit::Node* n, args& args) -> bool {
                    auto in = args[0].ITensor();
                    auto numOutputs = args[1].unwrapToInt();
                    auto axis = args[2].unwrapToInt();
                    auto inDimSize = in->getDimensions().d[axis];
                    LOG_DEBUG("Number of chunk outputs: " << numOutputs);
                    std::vector<int64_t> sizes;

                    if (inDimSize % numOutputs == 0) {
                        for (int64_t i = 0; i < numOutputs; i++) {
                            sizes.push_back(inDimSize / numOutputs);
                        }
                    }
                    else {
                        for (int64_t i = 0; i < numOutputs - 1; i++) {
                            sizes.push_back(inDimSize / numOutputs + 1);
                        }
                        sizes.push_back(inDimSize - (inDimSize / numOutputs + 1) * (numOutputs - 1));
                    }

                    c10::ListTypePtr lt = n->output()->type()->expect<c10::ListType>();
                    c10::TypePtr elementType = lt->getElementType();
                    auto list = c10::impl::GenericList(elementType);
                    list.reserve(numOutputs);

                    int start_idx = 0;
                    for (int64_t i = 0; i < numOutputs; i++) {
                        at::Tensor indices = torch::arange(start_idx, start_idx + sizes[i], 1).to(torch::kI32);
                        auto indicesTensor = tensor_to_const(ctx, indices);

                        auto gather_layer = ctx->net->addGather(*in, *indicesTensor, axis);
                        auto gather_out = gather_layer->getOutput(0);

                        auto tensor_holder = TensorContainer();
                        tensor_holder.hold_tensor(gather_out);
                        auto ival = c10::IValue(std::move(c10::make_intrusive<TensorContainer>(tensor_holder)));
                        list.emplace_back(ival);

                        start_idx = start_idx + sizes[i];
                    }

                    auto chunk_output_ivalue = std::move(torch::jit::IValue(list));
                    ctx->AssociateValueAndIValue(n->outputs()[0], chunk_output_ivalue);

                    LOG_DEBUG("Converted chunk op into a list of IValues");
                    return true;
                  }});

DEBUG log
INFO: [TRTorch Conversion Context] - Adding Layer %78 : Tensor[] = aten::chunk(%input.152, %self.stage2.0.stride, %self.stage2.1.stride) # /opt/conda/lib/python3.8/site-packages/torchvision/models/shufflenetv2.py:89:21 (ctx.AddLayer)
DEBUG: [TRTorch Conversion Context] - Node input is an already converted tensor
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
DEBUG: [TRTorch] - Number of chunk outputs: 2
DEBUG: [TRTorch] - Weights: [58]
    Number of input maps: 58
    Number of output maps: 58
    Element shape: [1]
DEBUG: [TRTorch Conversion Context] - Freezing tensor 0x7fbb1c37c4e0 as an IConstantLayer
DEBUG: [TRTorch] - Weights: [58]
    Number of input maps: 58
    Number of output maps: 58
    Element shape: [1]
DEBUG: [TRTorch Conversion Context] - Freezing tensor 0x7fbb1be22d30 as an IConstantLayer
DEBUG: [TRTorch] - Converted chunk op into a list of IValues
DEBUG: [TRTorch Conversion Context] - Evaluating %x1.9 : Tensor, %x2.9 : Tensor = prim::ListUnpack(%78)
DEBUG: [TRTorch Conversion Context] - Found the evaluated value(s) to be True for node: %x1.9 : Tensor, %x2.9 : Tensor = prim::ListUnpack(%78)
DEBUG: [TRTorch Conversion Context] - Found the evaluated value(s) to be True for node: %x1.9 : Tensor, %x2.9 : Tensor = prim::ListUnpack(%78)
DEBUG: [TRTorch Conversion Context] - Evaluating %81 : Float(58, 58, 1, 1, strides=[58, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
DEBUG: [TRTorch Conversion Context] - Found the value to be a tensor (shape [58, 58, 1, 1])
INFO: [TRTorch Conversion Context] - Adding Layer %82 : Tensor = aten::_convolution(%x2.9, %81, %7, %9, %11, %9, %10, %11, %self.stage2.1.stride, %10, %10, %10, %10) (ctx.AddLayer)
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
DEBUG: [TRTorch Conversion Context] - Node input is a result of a previously evaluated value
Traceback (most recent call last):
  File "test_shuffle_net.py", line 27, in <module>
    trt_ts_module = trtorch.compile(scripted_model, compile_settings)
  File "/opt/conda/lib/python3.8/site-packages/trtorch/_compiler.py", line 73, in compile
    compiled_cpp_mod = trtorch._C.compile_graph(module._c, _parse_compile_spec(compile_spec))
RuntimeError: [Error thrown at core/conversion/var/Var.cpp:149] Expected isITensor() to be true but got false
Requested ITensor from Var, however Var type is c10::IValue

I think that all you need to do is replace auto in = args[0].ITensor(); with auto in = args[0].ITensorOrFreeze(). The difference is that ITensorOrFreeze can handle the case where the argument is an ITensor like args.ITensor() and the case where the arg is a PyTorch Tensor by creating a IConstantLayer and giving you the ITensor output back.

Thanks for your advice! I replaced ITensor() with ITensorOrFreeze(ctx) in the converter of aten::_convolution and it works.


Patch Requests

Does this library support PATCH?  router.patch() is undefined.

It does not.
Could you explain a bit more about how you see sw-toolbox interacting with PATCH requests? It doesn't seem like one of the runtime caching strategies would be particularly appropriate for use with a PATCH.

I'm building an app that needs to run offline and place requests into a queue while a connection is not available.  Some of these requests will be patch requests because the api is JSONAPI compliant.  The goal is to queue the requests into indexeddb and replay them once a connection is available.

sw-toolbox is good for implementing runtime caching strategies, but doesn't have any support for queueing and replaying failed requests.
There's nothing stopping you from mixing and matching your usage of sw-toolbox along with your own fetch handlers that implement custom behavior like what you describe. Just make sure that your fetch handler is added before sw-toolbox's routes have a chance to match a request. Inside your fetch handler, only call event.respondWith() if you're sure it's one of the requests you want to handle. So, something like:
self.addEventListener('fetch', event => {
  if (event.request.method === 'PATCH') {
    event.respondWith(
      // Your logic goes here.
    );
  }
});

// Use sw-toolbox to handle other runtime strategies here.

Thanks


Question about additional AI tweaks

Just a quick question, @Sh0wdown do you think that it would be possible, to edit the executable in way, that you could edit what kind of stuff an ai uses?
-What economy it builds, and how much of each resource it saves on its stockpile
-Which kind of units are recruited and how many used on a defensive point, on aggressive patrols, on defensive patrols and of course for attacks
Examples:
-Make the Sultan build up to 5 hunters.
-Let the rat build up to 7 apple farms, and add more defensive archers
-Make the Wolf build more iron mines
-Let the pig also recruit archers
-Change Frederick's swordsmen to pikemen
-Let the snake use assassins and very, very, very few arabic swordsmen
-Let the Wolf man his towers with more units, and enable arabic archers
-Let the Sultan recruit up to 5 defensive fire throwers
-Let the rat build supportive shields

It would be great if this was possible; certainly the stronger Arabian AIs in particular (Saladin, Emir, Nizar, Wazir and Caliph), because they have more expensive armies, could definitely do with the ability to produce a few more quarries or iron mines, as they often run out of gold far too quickly. And Phillip definitely could do with at least 1 quarry, at the moment he has none :(

I agree with trying to add some quality of life changes to the AIs. However for Sultan, you can add 5 hunter huts yourself in the AIV Editor.
Some other things I would suggest:
-Allow Marshal to build more quarries (1 isn't enough for him)
-Allow AIs in general to space out siege engine tents so that they do not block other engineers from entering tents
I like the idea of Snake using assassins. He's described as a sneaky, underhand opponent that prefers letting 'hired help' do his dirty work for him. It makes sense for him to use stronger Arabic units for offense while using weaker European units for castle defense.

Haven't looked into it yet, but I probably will eventually. Philipp & Richard with their puny amounts of archers on the towers could also be added to this list.

I agree with trying to add some quality of life changes to the AIs. However for Sultan, you can add 5 hunter huts yourself in the AIV Editor.

Yep it would good for AIs to have small economic boosts, another iron mine or two for all the AIs that currently make iron mines is a good option of this as they use less workers than stone quarries and also provide income for iron selling AIs (and for some of the Crusader lord AIs, could reduce costs of buying in weapons), but for AIs that use ale it must be made sure that any additional buildings do not push the population too high for ale buildings to still provide a +8 popularity boost.

I like the idea of Snake using assassins. He's described as a sneaky, underhand opponent that prefers letting 'hired help' do his dirty work for him. It makes sense for him to use stronger Arabic units for offense while using weaker European units for castle defense.

Yeah I suppose it would go with his personality, although I do like the fact Snake uses laddermen so without the spearmen attacking it would make them useless.
Other changes that would be good:
More monks in Abbot's raids.
More knights in Phillip's raids. He has enough stables to create at least 20 knights to be sent in a raid.
More pikemen sent in Richard's raids.
A proper siege force for King Phillip. Tunnelers would be a great option for him in sieges as he can create a breach in a castle wall, and once there is a breach in the castle wall, Phillip does often send any raiding knights he may have through to destroy castle buildings such as the granary.

More knights in Phillip's raids. He has enough stables to create at least 20 knights to be sent in a raid.
I disagree here. He is a weaker AI opponent, he should not raid as strongly as Marshall. I would prefer him rebuilding his defensive knights around his castle though.
Tunnelers do not sound like King Phillip at all. I would add it to the pig and the snake and / or the rat instead.

I disagree here. He is a weaker AI opponent, he should not raid as strongly as Marshall. I would prefer him rebuilding his defensive knights around his castle though.
Tunnelers do not sound like King Phillip at all. I would add it to the pig and the snake and / or the rat instead.

Disagree aswell, i think it would be cool if Phillip does actually use all his stables. But the Knights do not need to be in his raids. Like someone other mentioned, he could use Knight in his attacks.
Everyone has an other opinion for Tunnelers it seems, because i think they would fit for Phillip. The Rat is also an weaker AI opponent why should he use the Tunnelers? I would maximum give the Rat Laddermen, catapults and shields. (Like he used in Stronghold 1)
Ist not an must have but Tunnelers for the Pig sounds good for me.

I never actually see any of the AI using ladder men. Would be a nice addition!

Considering Phillip's primary use of Spearmen, I think Laddermen are pretty fitting for him. In his SHC description it says his "over-reliance on Knights as his main fighting force is most often if not his undoing". It makes sense to me that he would overly use Knights in his forces, whether be for defense, patrol, attacks and harassment.

Tunnelers it seems because I think they would fit for Phillip. The advice is therefore weaker AI opponent why should he use the tunnelers?

But with spearmen as the only units that can get onto an enemy keep, he will still be weak! I think tunnelers would be a better option than laddermen because Phillip's best strategy I can see for besieging a castle is managing to create a gap in the walls so he can send any raiding knights he has through to destroy the granary and castle buildings before his enemy can repair the gap in the walls!

Like someone else mentioned, he could use Knight in his attacks.

I mentioned it. 😃  It would certainly look amazing if Phillip gathered a group of knights, archers and spearmen (and one or two catapults) to attack an enemy! Yes it might not be the most effective way to use knights but it would fit the SHC description of Philip's 'reckless insistence of the use of knights as his main fighting force'.

Considering Phillip's primary use of Spearmen, I think Laddermen are pretty fitting for him. In his SHC description it says his "over-reliance on Knights as his main fighting force is most often if not his undoing". It makes sense to me that he would overly use Knights in his forces, whether be for defense, patrol, attacks and harassment.

I mentioned it. 😃 It would certainly look amazing if Phillip gathered a group of knights, archers and spearmen (and one or two catapults) to attack an enemy!

That sounds all good for me. The King of France definitely needs more Strength! He kinda feels so bugged for me as an AI Lord. Because he has so many weird things going on. (Not using quarries, dont man his Tower properly, mostly using only one Stable but Building so many, attacking only with spearmen,..)

That sounds all good for me. The King of France definitely needs more Strength! He kind of feels so bugged for me as an AI Lord. Because he has so many weird things going on. (Not using quarries, don't man his Tower properly, mostly using only one Stable but Building so many, attacking only with spearmen,..)

Attacking with spearmen sounds fine to me. I like that. I just feel like he should have support of siege engineers. Preferably battering rams.
He should use a few more knights, but neither for attacking nor to destroy enemy economy. I would prefer him to have defensive knights.
He would be too strong and too much like the Marshal...

Yep battering rams would be excellent for Phillip, about 3 or 4 in a siege and that would make his sieges unique for an AI. The rams could open up that all important gap in the castle defence so that his raiding knights could get in and cause havoc. He would benefit though from bringing some archers in his siege forces to try and protect the engineers from enemy arrow fire so they could make the rams.

Attacking with spearmen sounds fine to me.

Well now with the improvements in AI attack number increases over time in this Unofficial Patch, he seems to finally be able to send big numbers of spearmen eventually from what I have seen and also spearmen have been made a little more resistant to arrows in the Patch.

Philip's one problem is his castle.

By the way, it would be great if the wolf used knight.

Richard should use knights, as they are seen amongst his starting troops during "less gold, more troops" mode.

Richard should use knights, as they are seen amongst his starting troops during "less gold, more troops" mode.

Yes great point, I think he should recruit a few, but to make him different to Phillip and Marshall, he should just use them for defensive purposes, not raiding (and use bigger numbers of pikemen than he does currently for raiding).

I think that additionally to the patch there should be added a mini-mod to-mark option allowing one of the AI lords to do it's absolute best to the maximum of modding possibilities(being capable of doing all troops and using multiplayer-like gameplay economic behavior, additionably if possible: act like a smart online human-player).
Myself I've had such a thing in mind 2 years ago. That's when I've made a Wolf AIV manage to do a prospering 250 population economy consist of 54 workshops, 10 breweries, 10 Inns and both cathedrals. All this stuff mixed alltogether had allowed him to gain tons of gold quicky by running whole game on between -22 to -24 tax, selling also the over-produced weaponry and apple baskets - gathering a whole ton of these even on such a high population, by tricking their income and consuption through accurate regular purchase of cheese and bread.
PS: A wolf capable of doing bread would be even better and more fitting for most maps. Sadly his behavior doesn't allow him to build wheat farms. Even if I place these for him directly in the map editor he declines using the wheat, taking rid of by selling it immediatly as quickest it gets dropped on the stockpile, causing bakers doing nothing whole game.
At any case, he was capable or earning more gold than current top online players! when I was testing him play against them. Definately -24 tax whole game on 250 pop is more efficent than -24 tax on 460+ pop for a while period of time.
READ THIS:
I know how alot of players don't enjoy the hardcore-like stuff and multiplayer pro way of gameplay, but I think that modding shouldn't be limited only for singleplayer-fans. I don't insist on buff a "regular or Evrey's mod" Wolf to the maximum, but (I repeat), as a mini-mod to-mark on the installer option for the part of the community, which simply is being too good for singleplayer AI-fighting, as a great additional challenge to exercise on.
I'm aware that a challenge might aswell be achived by playing AI 1vs7, but having a strongest possible AI in a 1vs1 environment, that could work for itself on a better way than the 7 allied alltogether definately looks better and less messy.
Think about that, how great would it be to modify an AI to play better than even the greatest multiplayer legends to walk across this game from over a decade back, like: {{DHK}} HolyLord, Gx_S0ck3tz`, @rm@, XXoDDu$$, FireDragon, Bartholo, Luis7, Blacky... and way more... These do not play anymore, but remained in memory as icons of multiplayer passion and addiction to develop all possible aspects of gameplays.
Slowly more and more players are playing with each next year. Even if they do so, they're taught already differently, than they would be past on Gamespy Arcade, in the community of the upper mentioned.
Knowledge about the game is also dying. On 2007 probably 99% of possible in-game bugs were discovered. Now probably 1/3 of these got fogotten.
Creating a challenging AI opponent playing at the peek level of any of the old top players would surely bring interest to the game, and the mod. Similar situation as in Chess with Kasparov vs Deep Blue AI, where AI first time whooped a legend's butt :)

0_0 Wonderful idea !!!
P.S
If not difficult, please throw me your wolf AIV.

@Lolasik011 I won't share him, as now as I have just seen this Unofficial Patch going on, I'll be up to come back over my 2 years old work to provide him multiple variations of this one castle to be fitting on more maps. For now You can see my 2 years old Wolf over here dealing with the Rat on the only 400x400 map in the game I was working over him. His eco & stats. The biggest flaw of my Wolf AIV for the current moment is exacly that he isn't flexible to being capable of playing efficently on other maps.
So I'm going to emphasis my AIV editing over making him multiple-map compactible, by making more versions of his AIV files(Curretly he has still one) on my free time. Especially fit for the multiplayer maps such as EventMap07 or Aw-lub. Cheers.

Rakso69 Strange ... As you like.

Closing this, as it is now possible with AIC edits.


methods do not work on the returned objects from dataset.getTables

On calling dataset.getTables from the bigquery API, and then calling any method on the returned tables I get the following error:
Invalid table ID
Am I doing something wrong?

I was able to reproduce as well. Looks like table IDs are returned from the upstream APIs like this:
nth-circlet-705:gcloud_test_dataset_tempc24392d0_cbac_11e5_9063_0923b7e0f6f8.myKittens

I'm not sure if it's always been this way, or if we just missed it. But what we want instead is:
apiResponse.tableReference.tableId;

In the meantime, you can use something like this code to get around the bug:
dataset.getTables(function(err, tables) {
  tables = tables.map(function(table) {
    var fixedTable = dataset.table(table.metadata.tableReference.tableId);
    fixedTable.metadata = table.metadata;
    return fixedTable;
  });
});

Thanks for catching this and reporting it!

Thanks Stephen!

Thank you! @callmehiphop gave us the fix in #1116. We'll hopefully be releasing on Monday, keep an eye out!


Integration Test - Release - 2019/10/31 22:22:55

Auto create for integration test. IssueCommandFlow

#pri1

Just type '#pri3'

#pri2

#pri3

#assign PRMergerTest1

#assign NonAllowedUser

NonAllowedUser is not a valid GitHub ID, or is not a collaborator on this repo.

NonAllowedUser is not a valid GitHub ID, or is not a collaborator on this repo.

#reassign PRMergerTest2

GitHub user PRMergerTest1 has been removed from the MicrosoftDocs organization, so they were automatically removed as an assignee.

#reassign NonAllowedUser

NonAllowedUser is not a valid GitHub ID, or is not a collaborator on this repo.

NonAllowedUser is not a valid GitHub ID, or is not a collaborator on this repo.

#please-close


Do cluster slot renewal in a background thread

Instead of doing it as soon as a MOVED was received from redis.

Hmm I'm wondering if this makes sense.
The only scenario where I believe this could be useful is if you have little traffic to the cache or a specific node and the background renewal happens before the user gets a MOVED response. I still believe this scenario is pretty rare and I don't really see the benefit of doing this.
IMHO slot migration (because of failures or new node additions) shouldn't be something that happens often, so I don't think it's bad to lock if two queries get a MOVED at the same time and wait until the cluster heals. In addition to this, some questions came to my mind and I'm wondering if you considered them:

How often will you perform slot renewal by default?. Even though you can make this configurable, default should be good enough to fit "most" cases.

I guess you'll still be doing slot renewal on MOVED responses as well?. How would you coordinate the background renewal vs the one triggered by the MOVED response?

A hybrid idea that might work is to perform slot renewal in a separated thread only when receiving a MOVED response. As it is today, Jedis won't return a response to the user until the slow renewal has finished, maybe we can improve this and perform the slow renewal in a separate thread so we can let new queries to be processed.
@HeartSaVioR WDYT?

I think you got it wrong. It doesn't need to happen in background before a
MOVED to make sense. The whole idea is not to wait for the cache to catch
up when there is a MOVED, there is no need. We already know where to go,
why penalizing the command?
The idea is to continue and serve the command, and just update the cache in
the background. This also avoid unnecessary locks on the cache.
So it not only makes the command response latency smoother in general but
also probably make the code easier to follow.
On Mon, Jul 11, 2016, 08:28 Marcos Nils<EMAIL_ADDRESS>wrote:

Hmm I'm wondering if this makes sense.
The only scenario where I believe this could be useful is if you have
little traffic to the cache or a specific node and the background renewal
happens before the user gets a MOVED response. I still believe this
scenario is pretty rare and I don't really see the benefit of doing this.
IMHO slot migration (because of failures or new node additions) shouldn't
be something that happens often, so I don't think it's bad to lock if
two queries get a MOVED at the same time and wait until the cluster heals.
In addition to this, some questions came to my mind and I'm wondering if
you considered them:

How often will you perform slot renewal by default?. Even though you
can make this configurable, default should be good enough to fit "most"
cases.
I guess you'll still be doing slot renewal on MOVED responses as
well?. How would you coordinate the background renewal vs the one triggered
by the MOVED response?
A hybrid idea that might work is to perform slot renewal in a separated
thread only when receiving a MOVED response. As it is today, Jedis won't
return a response to the user until the slow renewal has finished, maybe we
can improve this and perform the slow renewal in a separate thread so we
can let new queries to be processed.
@HeartSaVioR https://github.com/HeartSaVioR WDYT?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/xetorthio/jedis/issues/1347#issuecomment-231709105,
or mute the thread
https://github.com/notifications/unsubscribe/AAIhp3hGAy-aQCSSe7usHbhZGK38vvnkks5qUijfgaJpZM4JID3f
.

The idea is to continue and serve the command, and just update the cache in
the background. This also avoid unnecessary locks on the cache.

Ok, I got it wrong then. This is something similar to what I thought about :smile:

The idea is to continue and serve the command, and just update the cache in the background. This also avoid unnecessary locks on the cache.

Please correct me if I'm wrong, but then slot will be shared variable between user threads and background thread. If we don't set a lock, while background thread reconstructs slots all operations can be probably failing. We should make operations waiting anyway while slot is reconstructing.

If we don't set a lock, while background thread reconstructs slots all operations can be probably failing

@HeartSaVioR how so?. Threads that don't see the change will still get the MOVED response while new threads will go to the correct node. The background thread can only process one slot update at a time.

We are "moving" the slot information, which means we "delete" the slot, and "add" the slot. It can't be atomic, and it means there's race condition between main thread and background thread.
But I like the idea @xetorthio said. Main thread doesn't need to wait for cache to be updated. If we handle the case what I'm saying properly, I'm OK to change.

Yeah, I think I did a mistake again. Slots are stored to a concurrent hashmap. I'm OK with this. Great idea.

Yeah, I think I did a mistake again. Slots are stored to a concurrent hashmap. I'm OK with this. Great idea.

I was about to say this :D

Also current implementation cleans all slots before filling them. We could avoid doing that. Just update the slots without doing the cleanup. This way we'll be making a better use of the ConcurrentHashMap

We could use a SingleThreadExecutor to make sure there in a single background thread and if it crashes for whatever reason, it will be re-launched.
The only thing missing is a signaling mechanism to tell the thread to "wake up" and renew slots when a MOVED was found.
Still thinking about hot to do it. Any ideas?

AFAIK, we can just create a new task which contains slot information from MOVED response to that ExecutorService whenever we found MOVED, and background thread runs every new task.

Ok. That means we never ask for slots again, right?
On Wed, Jul 13, 2016, 01:11 Jungtaek Lim<EMAIL_ADDRESS>wrote:

AFAIK, we can just create a new task which contains slot information from
MOVED response to that ExecutorService whenever we found MOVED, and
background thread runs every new task.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/xetorthio/jedis/issues/1347#issuecomment-232251942,
or mute the thread
https://github.com/notifications/unsubscribe/AAIhp_J5WMACDgqiIDwY8rrMHmcBTBeGks5qVGV4gaJpZM4JID3f
.

@HeartSaVioR but won't tasks queue if we do this?. The idea is that there can one be one task in the queue, it wouldn't make sense to queue slot renewal tasks.

I'm not sure I understood you comment @marcosnils

@HeartSaVioR suggest adding a new tasks to the ExecutorService each time a MOVED response is returned. The SingleThreadExecutor will execute one task at a time but every time you add a task the task will be queued for later execution. What I'm saying is that there shouldn't be any queuing in this process, only one task can be processed at a time and that's it.

But how your suggestion works? Imagine you get a MOVED at the same time from 2 different jedis instances. Both need to do something. What do they do? With what @HeartSaVioR suggest, both queue them in the executor. And eventually the background thread will refresh the cache.
What will your suggestion do?

@xetorthio @marcosnils
I'd like to clarify once more. There're two different strategy of updating slot information when we get MOVED.

update one slot which is based on MOVED response
Then I'd like to see how much time a thread is waiting for concurrent map to write. If it's fairly tiny we even don't need to make slot update fire and forget. Just update it.

update whole slots (say updating all 16384 slots)
We need to set strategy in this case.

What we're currently doing is in fact slowest but clearest to resolve this scenario. No race condition because it's guarded with RW lock.
Apart from race condition, there might be some threads throwing MOVED which reads slot information before write lock taking effect.
One sketched idea to resolve is comparing fetched timestamp (I mean the moment when the thread queries to the cache) and slot updated timestamp (in cache), and do update slots when only fetched timestamp is greater than slot updated timestamp. (which means that it queries to Redis Cluster with latest update of slot information but Redis responds MOVED.)
If we don't want to maintain RW lock but want to update whole slots, sketched idea could be extended to have timestamp for each slot. It's just a sketched idea so we could have better alternatives or this idea could be improved.
tl.dr; Summary of my sketched idea is: if borrowed thread got MOVED response, check that it refers latest information of that slot. If it does, we need to update slot(s). If not, we don't need to update slot(s).

I think that updating only one slot has it's benefits, but it could be also
bad. This depends on the scenario and what exactly happens is the cluster.
I guess that a MOVED shouldn't be something that happens very frequently,
so updating all slots when it happens sounds pretty good.
Now this doesn't need to be too strict. Because if cache is not up to date,
everything still works, only that we'll have hops.
So probably the best solution is the one that it' simplest to understand,
code and maintain.
A background thread that does the update sounds good. What triggers the
update is still something I'm not sure of.
On Wed, Jul 13, 2016, 11:23 Jungtaek Lim<EMAIL_ADDRESS>wrote:

@xetorthio https://github.com/xetorthio @marcosnils
https://github.com/marcosnils
I'd like to clarify once more. There're two different strategy of updating
slot information when we get MOVED.

update one slot which is based on MOVED response
Then I'd like to see how much time a thread is waiting for concurrent
map to write. If it's fairly tiny we even don't need to make slot update
fire and forget. Just update it.
2.
update whole slots (say updating all 16384 slots)
We need to set strategy in this case.
What we're currently doing is in fact slowest but clearest to resolve this
scenario. No race condition because it's guarded with RW lock.
Apart from race condition, there might be some threads throwing MOVED
which reads slot information before write lock taking effect.
One sketched idea to resolve is comparing fetched timestamp (I mean the
moment when the thread queries to the cache) and slot updated timestamp (in
cache), and do update slots when only fetched timestamp is greater than
slot updated timestamp. (which means that it queries to Redis Cluster with
latest update of slot information but Redis responds MOVED.)
If we don't want to maintain RW lock but want to update whole slots,
sketched idea could be extended to have timestamp for each slot. It's just
a sketched idea so we could have better alternatives or this idea could be
improved.
tl.dr; Summary of my sketched idea is: if borrowed thread got MOVED
response, check that it refers latest information of that slot. If it does,
we need to update slot(s). If not, we don't need to update slot(s).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/xetorthio/jedis/issues/1347#issuecomment-232370935,
or mute the thread
https://github.com/notifications/unsubscribe/AAIhpw5MTLQy-adY_ezNT0cFJK0XR8I7ks5qVPT6gaJpZM4JID3f
.

IMHO updating slot-wise on each MOVED redirection isn't necessarily efficient. lettuce follows the concept of having multiple strategies

Regular refresh(polling in time intervals)
Adaptive refresh

The adaptive topology refresh listens to persistent disconnects, MOVED and ASK redirections (configurable). If the cluster connection runs into a -MOVED error, then this is the signal to trigger a topology update (in lettuce topology updates are async anyway).
Another point to consider is a rate-limiting. Busy applications can run into 1000's of events per second, you don't want to trigger that many updates.
Integrating Threads into a framework always pulls in more stuff than expected. You don't want to launch a Thread per connection as that's expensive and with sharing Thread s you quickly end up adding configuration facilities. Wanted just mention that point to you . So sharing an ExecutorService with single/pooled implementation would be good when you want to perform updates using worker threads.

Thanks for your input @mp911de
Checking the ruby and nodejs client, they both do what we where thinking at the beginning of this thread. They handle the MOVED and serve the command, and then trigger the refresh. There is no regular refresh.
Now one thing that I like about ioredis, is that it does 2 things when it gets a MOVED. It changes the slot and then triggers the refresh (async of course). The good thing about this, is that any other command that happen to ask for the same slot, which the background thread refreshes the cache, will go to the right place. So it is a nice approach we should consider.

IMHO when client receives a MOVED command, we should not deem it as an exception ( JedisMovedDataException), it's quite normal during migration. We can just continue to serve MOVE command and update the slot cache in a background thread. Since redis is not a CP model in CAP, we can sacrifice consistency for liveness.
Actually, we don't even have to do it each time we see a MOVED response, when there is a brain-split each clients will frequently update the slot cache in vain, when the notorious "too many redirects" error occurs, our clients will be spending most of IO resources to update the slot cache, and update it wrongly.

This thread is very old, but I don't think the problem as been solved, or it has not been solved consistently.  I'm not overly familiar with all of the different cluster, connection pool, configuration options, but I do have a requirement to hold a consistent connection for a particular hash slot for a series of operations. Here is the basic code I worked out, after ruling out JedisCluster:
try (Connection conn = connectionProvider.getConnectionFromSlot(JedisClusterCRC16.getSlot(aStringKey))) {
    final Jedis client = new Jedis(conn);
    final String result = client.get(aStringKey); // ?
}

If the client sees a MOVED error that might work, I did not try it. But if the client gets a connection exception because the host has gone offline it never seems to recover from this, even after the cluster has performed a failover and recovered itself. It will be persistently stuck until you take care of the problem.
One solution is something like this - similar to Lettuce's adaptive refresh:
try (Connection conn = connectionProvider.getConnectionFromSlot(JedisClusterCRC16.getSlot(aStringKey))) {
    try {
        final Jedis client = new Jedis(conn);
        final String result = client.get(aStringKey); // ?
    } catch (JedisConnectionException ex) { // Only catches connection errors, will likely miss `MOVED` or others
        connectionProvider.renewSlotCache();
        // enter a retry loop here
    }
}

Another solution, or a companion solution in case you need both is to do the periodic refresh that was suggested above. Lettuce has this option and it has become absolutely necessary for stability in production.
// Somewhere in your application setup...
final ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor();
final ClusterConnectionProvider connectionProvider = new ClusterConnectionProvider(hosts, jedisClientConfig);
scheduledExecutorService.scheduleAtFixedRate(connectionProvider::renewSlotCache, 0, 15, TimeUnit.SECONDS);

If the client sees a MOVED error that might work, I did not try it. But if the client gets a connection exception because the host has gone offline it never seems to recover from this, even after the cluster has performed a failover and recovered itself. It will be persistently stuck until you take care of the problem.

What exception did you see? JedisMovedDataException or something else?

I did test this with JedisCluster and it works very well on failovers. I think the implementations are different when you take direct control over a Connection. Perhaps I can work around that requirement to hold a specific connection by using a pipeline or some other transaction mechanism.
I was migrating the test from Lettuce where the equivalent of JedisCluster::waitReplicas does not accept a key and thus cannot be routed properly by the cluster management. I originally missed that in Jedis that is not the case and it simplifies the implementation greatly! 😄


RequestExceptionAction is triggered for each type of exception in the exception inheritance hierarchy

I have the UnhandledExceptionLoggingAction, registred as open generic, that should execute for every unhanded exception
    internal class UnhandledExceptionLoggingAction<TRequest, TException> : IRequestExceptionAction<TRequest, TException>
        where TRequest : notnull
        where TException : Exception
    {
        public Task Execute(TRequest request,
                            TException exception,
                            CancellationToken cancellationToken)
        {
            var requestName = typeof(TRequest).FullName;
            Log.Error(exception, "Request: Unhandled Exception for Request {Name}\n{@Request}", requestName, request);
            return Task.CompletedTask;
        }
    }

the problem is that for a one trowed exception, the Action is triggered multiple times for each type of exception in the exception inheritance hierarchy
Di registration: services.AddMediatR(typeof(MediatrStartupModule).Assembly);

Call Stack:
I'm throw a NotImplementedException
>	PlatformCore.Behaviors.UnhandledExceptionLoggingAction<PlatformCore.Test.Application.MediatrTest.Query, System.NotImplementedException>.Execute(PlatformCore.Test.Application.MediatrTest.Query request = {PlatformCore.Test.Application.MediatrTest.Query}, System.NotImplementedException exception = {"The method or operation is not implemented."}, System.Threading.CancellationToken cancellationToken = IsCancellationRequested = false) Line 20	C#	Symbols loaded.

>	PlatformCore.Behaviors.UnhandledExceptionLoggingAction<PlatformCore.Test.Application.MediatrTest.Query, System.SystemException>.Execute(PlatformCore.Test.Application.MediatrTest.Query request = {PlatformCore.Test.Application.MediatrTest.Query}, System.SystemException exception = {"The method or operation is not implemented."}, System.Threading.CancellationToken cancellationToken = IsCancellationRequested = false) Line 20	C#	Symbols loaded.

>	PlatformCore.Behaviors.UnhandledExceptionLoggingAction<PlatformCore.Test.Application.MediatrTest.Query, System.Exception>.Execute(PlatformCore.Test.Application.MediatrTest.Query request = {PlatformCore.Test.Application.MediatrTest.Query}, System.Exception exception = {"The method or operation is not implemented."}, System.Threading.CancellationToken cancellationToken = IsCancellationRequested = false) Line 20	C#	Symbols loaded.

I got the same issue today, created PR #767 to fix this.


ISSUE-623 : Execute specified previous steps while retrying a given step( for scenarios where current step is dependent on previous steps, such as auth)

PR Branch
#ADD LINK TO THE PR BRANCH
fixes #623
Motivation and Context
The idea here is that instead of maintaining a List for ScenarioExecutionState, we will maintain a LInkedHashMap of StepExecutionState. This will make it easier to mutate ScenarioExecutionState in complex retry scenarios where previously successfully executed steps need to be re-executed (such as in the withSteps feature.
Checklist:

[ ] Unit tests added

[ ] Integration tests added

[ ] Test names are meaningful

[ ] Feature manually tested

[ ] Branch build passed

[ ] No 'package.*' in the imports

[ ] Relevant Wiki page updated with clear instruction for the end user

[ ] Not applicable. This was only a refactor change, no functional or behaviour changes were introduced

[ ] Http test added to http-testing module(if applicable) ?

[ ] Not applicable. The changes did not affect HTTP automation flow

[ ] Kafka test added to kafka-testing module(if applicable) ?

[ ] Not applicable. The changes did not affect Kafka automation flow

@authorjapps  can you review this? There are a few changes to some old constructs


Tyntec WhatsApp (typo fix)

Please check the following conditions for your PR.

[x] apiDefinition.swagger.json is validated using paconn validate command.
[x] apiProperties.json has a valid brand color. Invalid brand colors are #007ee5 and #ffffff.

@dougelliott A merge Conflict was resolved 👍🏻


mini.starter(NIGHTLY NVIM): Doesn't properly update when downsizing

Contributing guidelines

[X] I have read CONTRIBUTING.md
[X] I have read CODE_OF_CONDUCT.md
[X] I have updated 'mini.nvim' to latest version

Module(s)
mini.starter
Description
Looks like it's not refreshing properly when resizing the window down.
Neovim version
NVIM v0.10.0-dev-1603+gac1113ded
Steps to reproduce
Just setup mini.starter with default config and scale the window up and then down.
Expected behavior

The screen should update properly

Actual behavior

It does not:

Thanks for the issue.
I can reproduce. This doesn't happen on Neovim 0.9.4, though. And judging by the effects, it seems to be some issue with Nightly. I'll look into it.

Perfect, thanks!

I believe the root cause here is neovim/neovim#26139. There is a fix here, but it would lead to flickering on resize. As it is not a frequent event, it is acceptable, but we'll wait a bit for that to resolve.

As upstream issue is fixed, I can no longer reproduce it on latest Nightly and it seems to be fixed. Apart from query message disappearing (if was shown) after resize, but that is more or less aligned with not showing message on startup. Note for future: if want to fix that, wrap refresh  call here invim.schedule. Right now this seems like an unnecessary change.


Make keymgmt export functions leaner

They were made too strict, based on a misunderstanding on how selection works.

My tests come a lot further with this change.

Whoooo!  With this change, the only failing CI test is with test/rsa_genpkey_x509_cmp.sh, which we know is worked on separately (openssl/openssl#16050)

Whoooo! With this change, the only failing CI test is with test/rsa_genpkey_x509_cmp.sh, which we know is worked on separately (openssl/openssl#16050)

Are you sure this is related?
I cannot find any rsa_genpkey_x509_cmp.sh.

@DDvO I am (almost) sure it is related, because it works when I remove or revert the condition in https://github.com/openssl/openssl/commit/66be663b64e05fb5524edd051b85fb919f59e054.
https://github.com/tpm2-software/tpm2-openssl/blob/master/test/rsa_genpkey_x509_cmp.sh

Ah, I confused this with the OpenSSL repo. Sorry for the noise.


Switch to new Design System link styles

What
Set $govuk-new-link-styles: true in application.scss to enable the new link styles from the Design System.
Why
The new DS link styles are being rolled out across GOVUK.
We've already applied the new styles to the individual components in the component gem. The next step is to enable them on the frontend applications themselves.
This switches service-manual-frontend over to the new styles by setting the govuk-new-link-styles flag to true in the
application.scss file.
The types of pages rendered by this application are primarily content heavy pages. There are no app-level components to speak of, which makes transitioning to the new link styles a fairly straightforward process. I've tested this on the Example pages rendered by service-manual-frontend from the dev docs.

⚠ This repo is Continuously Deployed: make sure you follow the guidance ⚠

We seem to have a few instanced of app-link--inverse that could be replaced with govuk-link--inverse but not a blocker

Great catch @alex-ju, have updated 👏🏼


Convert Result Local OBX to NTE

Backlog Task
So that UCSD is able to file result ORUs that contain local codes from CDPH, OBXs need to be converted into NTE segments.
Completion Criteria
** More refinement of this completion criteria is likely needed **

[ ] For CA ORUs:

[ ] If OBX-3.3 is null and OBX-3.6 has the value "L", then the OBX segment from the incoming message is converted to an NTE segment.
[ ] For the outgoing NTE segment:

[ ] OBX-3.5 is moved to NTE-3
[ ] OBX-5 is also moved to NTE-3 (depending on OBX-2 value)
[ ] OBX-11 is also moved to NTE-3
[ ] Possibly other OBX values?

Example:
Adrenoleukodystrophy deficiency newborn screening comments-discussion|Negative|
Tasks

[ ] What tasks need to be done to fulfill the completion criteria?

Other Notes

Any other notes to help clarify this task for the team

On hold pending communication with CA

Still on hold; it's not clear if this approach will be able to be leveraged. Pushing back further into the backlog...

Likely to be canceled

Canceled in favor of https://github.com/CDCgov/trusted-intermediary/issues/1277


Embeding Direct Line channel to a website

Hi Team,
I am using direct line channel to connect with my QnA maker bot. i wanted to embed this channel to a website. I have used the below code on my website landing page to embed.

BotChat.App({
directLine: { secret: "XXXX" },
user: { id: 'userid' },
bot: { id: 'botid' },
resize: 'detect',
}, document.getElementById("bot"));
This is taking whole of my website UI space. I tried applying CSS but didnt work. I just want that to be embeded in a right corner. How can we do that ? . I was able to achieve that in Webchat channel by providing iframe style attributes. Can i have some sample working code for this please as i have some urgent deliverables.

Hi @Jagan-rajan,
Try the below bit of CSS. I have used this previously to manage the UI of the displayed bot. I had it placed in the html document as it was a small test project, so take that into consideration. Adjust the styling, as needed.
Closing as I believe this should resolve your issue. Please reopen if you find you have further related questions.
Hope of help!
Steve.
#bot {
      border: 1px solid #333;
      float: left;
      height: 600px;
      position: relative;
      width: 460px;
}

Hi Stevkan,
Thanks for answering.
I have a question regarding webchat channel since direct line is not exactly needed.
My req. is to pass user name and user id to the bot code not just the channel. Any ways of getting that done. If yes, can you please share the sample code.

Hi @Jagan-rajan,
The Microsoft Bot Framework team prefers that how to questions be submitted on Stack Overflow. The official Bot Framework Github repo is the preferred platform for submitting bug fixes and feature requests.


PublishCodeCoverageResults Fails to Generate HTML Report

Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Bug
Enter Task Name: PublishCodeCoverageResultsV1
list here (V# not needed):
https://github.com/microsoft/azure-pipelines-tasks/tree/master/Tasks/PublishCodeCoverageResultsV1
Environment

Server - Azure Pipelines or TFS on-premises?
Azure Pipelines

Agent - Hosted or Private:
Private Ubuntu 16.04

Issue Description
The PublishCodeCoverageResults task in our build has recently been unable to generate the HTML report, which I believe is what is causing us to be unable to view the CodeCoverage tab we are used to. It seems to be an issue with finding ReportGenerator, but I've included the task logs as I'm unclear what specifically to be looking for.
It seems like it could be introduced in the newest version of the task, as it was working on version 1.175.0, but is now failing on 1.175.2 (I don't have any runs on patch version 1). Nothing else has changed on our pipeline between the two, however it's possible some assumption of ours on how to use the task is no longer valid.
I've attached full debug logs for the failed run. Please let me know if I can provide any further information. Thanks in advance.
Task logs
PublishCodeCoverageResults_failed.zip
Error logs
/usr/bin/dotnet "/agent/_work/_tasks/PublishCodeCoverageResults_2a7ebc54-c13e-490e-81a5-d7561ab7cd97/1.175.2/netcoreapp2.0/ReportGenerator.dll" -reports:"/agent/_work/4/s/CodeCoverage/Cobertura.xml" -targetdir:"/agent/_work/_temp/cchtml" -reporttypes:HtmlInline_AzurePipelines
Could not execute because the specified command or file was not found.
Possible reasons for this include:
  * You misspelled a built-in dotnet command.
  * You intended to execute a .NET Core program, but dotnet-"/agent/_work/_tasks/PublishCodeCoverageResults_2a7ebc54-c13e-490e-81a5-d7561ab7cd97/1.175.2/netcoreapp2.0/ReportGenerator.dll" does not exist.
  * You intended to run a global tool, but a dotnet-prefixed executable with this name could not be found on the PATH.
##[warning]Failed to generate Html report. Error: 1

This is happening to me too. I confirm, it is also happening with the 1.175.2 version. I have a previous run on version 1.175.1 that worked, so something broke in 1.175.2.
2020-11-19T18:06:45.7947136Z ##[section]Starting: PublishCodeCoverageResults
2020-11-19T18:06:45.7960861Z ==============================================================================
2020-11-19T18:06:45.7961182Z Task         : Publish code coverage results
2020-11-19T18:06:45.7961460Z Description  : Publish Cobertura or JaCoCo code coverage results from a build
2020-11-19T18:06:45.7961731Z Version      : 1.175.2
2020-11-19T18:06:45.7961923Z Author       : Microsoft Corporation
2020-11-19T18:06:45.7962259Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/test/publish-code-coverage-results
2020-11-19T18:06:45.7962957Z ==============================================================================
2020-11-19T18:06:46.2875879Z [command]/usr/bin/dotnet "/home/vsts/work/_tasks/PublishCodeCoverageResults_2a7ebc54-c13e-490e-81a5-d7561ab7cd97/1.175.2/netcoreapp2.0/ReportGenerator.dll" -reports:"/home/vsts/work/1/s/coverage/cobertura-coverage.xml" -targetdir:"/home/vsts/work/_temp/cchtml" -reporttypes:HtmlInline_AzurePipelines
2020-11-19T18:06:46.3910421Z
2020-11-19T18:06:46.3977683Z Welcome to .NET Core 3.1!
2020-11-19T18:06:46.3978839Z ---------------------
2020-11-19T18:06:46.3979176Z SDK Version: 3.1.404
2020-11-19T18:06:46.3979312Z
2020-11-19T18:06:46.3979436Z Telemetry
2020-11-19T18:06:46.3979733Z ---------
2020-11-19T18:06:46.3980683Z The .NET Core tools collect usage data in order to help us improve your experience. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.
2020-11-19T18:06:46.3981229Z
2020-11-19T18:06:46.3981825Z Read more about .NET Core CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry
2020-11-19T18:06:46.3982079Z
2020-11-19T18:06:46.3982324Z ----------------
2020-11-19T18:06:46.3982686Z Explore documentation: https://aka.ms/dotnet-docs
2020-11-19T18:06:46.3983022Z Report issues and find source on GitHub: https://github.com/dotnet/core
2020-11-19T18:06:46.3983502Z Find out what's new: https://aka.ms/dotnet-whats-new
2020-11-19T18:06:46.3984010Z Learn about the installed HTTPS developer cert: https://aka.ms/aspnet-core-https
2020-11-19T18:06:46.3984586Z Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli-docs
2020-11-19T18:06:46.3985111Z Write your first app: https://aka.ms/first-net-core-app
2020-11-19T18:06:46.3985612Z --------------------------------------------------------------------------------------
2020-11-19T18:06:46.5702865Z Could not execute because the specified command or file was not found.
2020-11-19T18:06:46.5703268Z Possible reasons for this include:
2020-11-19T18:06:46.5704049Z   * You misspelled a built-in dotnet command.
2020-11-19T18:06:46.5705187Z   * You intended to execute a .NET Core program, but dotnet-"/home/vsts/work/_tasks/PublishCodeCoverageResults_2a7ebc54-c13e-490e-81a5-d7561ab7cd97/1.175.2/netcoreapp2.0/ReportGenerator.dll" does not exist.
2020-11-19T18:06:46.5706019Z   * You intended to run a global tool, but a dotnet-prefixed executable with this name could not be found on the PATH.
2020-11-19T18:06:46.5857030Z ##[warning]Failed to generate Html report. Error: 1
2020-11-19T18:06:46.5900205Z Reading code coverage summary from '/home/vsts/work/1/s/coverage/cobertura-coverage.xml'
2020-11-19T18:06:46.6990361Z ##[section]Async Command Start: Publish code coverage
2020-11-19T18:06:46.6990675Z Publishing coverage summary data to TFS server.
2020-11-19T18:06:46.6991076Z  Lines- 438 of 1231 covered.
2020-11-19T18:06:46.6991281Z  Branches- 162 of 654 covered.
2020-11-19T18:06:47.1937909Z Modifying Cobertura Index file
2020-11-19T18:06:47.1938306Z Publishing code coverage files to TFS server.
2020-11-19T18:06:47.1938485Z Uploading 1 files
2020-11-19T18:06:47.1938636Z File upload succeed.
2020-11-19T18:06:47.3472690Z Published '/tmp/Code Coverage Report_816' as artifact 'Code Coverage Report_816'
2020-11-19T18:06:47.3473077Z ##[section]Async Command End: Publish code coverage
2020-11-19T18:06:47.3474335Z ##[section]Finishing: PublishCodeCoverageResults

Can confirm this issue with version 1.175.2. Earlier pipeline runs with version 1.175.0 worked fin, now I'm running into the same error messages.

I can confirm this as well. I've managed to work around the problem by explicitly pinning the task to an older version, like this:
<EMAIL_ADDRESS>
I can also confirm that both 1.175.0 and 1.175.1 work correctly, but 1.175.2 doesn't. The strange thing about it is that from looking at the commit history, it looks like the only thing that changed between 1.175.1 and 1.175.2 was the version number being bumped, so I don't quite understand how 1.175.1 is working. Maybe I'm just not understanding how the build / publishing process for these tasks works.
Here's the two PRs corresponding to those new version:

1.175.1: https://github.com/microsoft/azure-pipelines-tasks/pull/13814
1.175.2: https://github.com/microsoft/azure-pipelines-tasks/pull/13862

And here's the issue relating to the changes: https://github.com/microsoft/azure-pipelines-tasks/issues/13062. It seems to have been trying to solve a problem where the task would fail if there were spaces in the path to the report generator dll by quoting the path. My guess is that the quoting is fine on Windows but causes issues on Linux.
This corresponds with what I can see from the build logs. Here's an example of the command run by 1.175.0:
/usr/bin/dotnet /home/vsts/work/_tasks/PublishCodeCoverageResults_2a7ebc54-c13e-490e-81a5-d7561ab7cd97/1.175.1/netcoreapp2.0/ReportGenerator.dll -reports:"/home/vsts/work/_temp/*/coverage.cobertura.xml" -targetdir:"/home/vsts/work/_temp/cchtml" -reporttypes:HtmlInline_AzurePipelines

And here's an example of the command run by 1.175.2:
/usr/bin/dotnet "/home/vsts/work/_tasks/PublishCodeCoverageResults_2a7ebc54-c13e-490e-81a5-d7561ab7cd97/1.175.2/netcoreapp2.0/ReportGenerator.dll" -reports:"/home/vsts/work/_temp/*/coverage.cobertura.xml" -targetdir:"/home/vsts/work/_temp/cchtml" -reporttypes:HtmlInline_AzurePipelines

I'll add a comment to the issue that this change was introduced for asking for it to be reverted until it can be fixed properly.

+1 one!
I'm hitting this as well - on several of my projects...
Pinning of minor version wasn't something I was aware of..
So a good workaround for now..
Looking forward to a bugfix on this..

We will try and patch this immediately.

I have observed the same bug. Most of my pipelines are using vmIImage: ubuntu-latest, and they all have the same error.
I have just seen a pipeline using vmImage: windows-latest work correctly with the exact same PublicCodeCoverageResults@1 (v1.175.2) task.
It successfully located the file D:\a_tasks\PublishCodeCoverageResults_2a7ebc54-c13e-490e-81a5-d7561ab7cd97\1.175.2\netcoreapp2.0\ReportGenerator.dll in the run log.
The failing ubuntu runs are trying to locate the file /home/vsts/work/_tasks/PublishCodeCoverageResults_2a7ebc54-c13e-490e-81a5-d7561ab7cd97/1.175.2/netcoreapp2.0/ReportGenerator.dll,

@dkstringer @adamconnelly @knom @MatthiasNistl @aressler38 @trangevi
Hi everyone!
Thank you for the information you've provided!
The fix was applied. Could you please confirm that it works fine for you at the moment?
The fixed version is 1.178.0.
Thank you!

Thanks @egor-bryzgalov. I can see it working now in 1.178.0.

@aressler38
Thank you!
Closed this issue as resolved. Please feel free to reopen if you have any issues.

I confirmed it's working for me now as well. Thank you for the quick turn around!

Me too - thanks for getting it sorted so quickly!


actually read file when creating secret file

Checklist

[ ] I've added an entry to CHANGELOG.md if necessary. You can comment this pull request with /changelog-entry [breaking-change|new-feature|bug-fix|chore] [message] and CHANGELOG.md will be updated automatically.

Why
In https://github.com/expo/eas-cli/pull/1354 there were a lot of comments that I had to address. It looks like this line was lost when I was refactoring it back and forth.
How
Actually read file.
Test Plan
Tested manually.

/changelog-entry bug-fix Fix creating EAS secret from file.


Add Role Cert Naming Format

Description
We have added environment variables to make the output directory and file names for the Role Cert more flexible:

ROLE_CERT_NAMING_FORMAT
ROLE_CERT_KEY_NAMING_FORMAT

Assignees

[x] Assignees is set

Type of changes

[x] Apply one or more labels of the following that fits:

bug: Bug fix
dependencies: Dependency upgrades
documentation: Documentation changes
enhancement: New Feature
good first issue: First contribution
logging: Log changes
refactor: Refactoring (no functional changes, no api changes)

Flags
- [ ] Breaks backward compatibility
- [ ] Requires a documentation update
- [ ] Has untestable code

Checklist
- [ ] Followed the guidelines in the CONTRIBUTING document
- [ ] Added prefix `[skip ci]`/`[ci skip]`/`[no ci]`/`[skip actions]`/`[actions skip]` in the PR title if necessary
- [ ] Tested and linted the code
- [ ] Commented the code
- [ ] Made corresponding changes to the documentation

Checklist for maintainer
- [ ] Use `Squash and merge`
- [ ] Double-confirm the merge message has prefix `[skip ci]`/`[ci skip]`/`[no ci]`/`[skip actions]`/`[actions skip]`
- [ ] Delete the branch after merge

moved to https://github.com/AthenZ/k8s-athenz-sia/pull/149


Deleting scenes, editions, projects

Deleting projects, editions, scenes involves deleting records from MongoDB and their related files and directories from the data file system.
We need to take extra care when records are linked to other records.
Records can be linked to other records in two ways:

master-detail: example: project-edition, edition-scene
cross-table: example: project-projectUser-user

Detail records have a link to their master record, not vice versa.
We can delete detail records, but we need a policy when deleting master records:

cascade: delete the master record and all its details recursively
prevent: let the user delete detail records before a master record can be deleted.

We choose the prevent strategy.
For cross-table situations: we delete the cross record if we want to unlink two linked records.
When we delete one of a pair of linked records, we automatically delete the cross-record as well.
When we place delete buttons next to projects, editions, scenes, we suppress it if there are detail records.
Instead, we place a count of the types of detail records.
All this is already in place.
What needs to be done is to also delete the relevant entities on the file system.
And we must enforce that a model cannot be deleted if there are still scenes in the same edition.
Also, we may not upload a model under a different file name, because that breaks the link.

We have changed our thinking: there is no longer a table with scenes. Every edition has a single scene with the predictable name voyager.svx.json, and there is no need to store scene properties in an extra MongoDb record.

Done


s3 private information in repo

i sent an email privately about this but haven't heard back, so i figure i'll report it here
the travis.yml file seems to have private info in it - https://github.com/fpco/wai-middleware-crowd/blob/master/.travis.yml
it should probably be removed and new keys generated, etc.

It contains the access key, which is not sensitive information. The secret key is encrypted.

okay.


Add Hirak Crypto to Cryptocurrency

[x] My submission is formatted according to the guidelines in the contributing guide
[x] My addition is ordered alphabetically
[x] My submission has a useful description
[x] The description does not have more than 100 characters
[x] The description does not end with punctuation
[x] Each table column is padded with one space on either side
[x] I have searched the repository for any relevant issues or pull requests
[x] Any category I am creating has the minimum requirement of 3 items
[x] All changes have been squashed into a single commit

Thanks

Hi @FayeNichols
Thanks for your contribution ;)
We have some problems with Public APIs, so we won't be able to review/add this improvement right now.
Read this issue for more information, please: https://github.com/public-apis/public-apis/issues/3104

This resource doesn't seem to be working anymore.


feat(semantic-release-config): add CI utility output file

Adding an exec step on success to write a version.txt file that can be read by CI utilities or other tools. This is a more generic version of the custom script that was used in settings-frontend.
We are using a file as this can often be passed between different CI jobs far more easily than an environment variable or similar. Also as we rely 100% on tags and not the package.json version for our versioning we can't just re-read the package.json file again.

📦 Published PR as canary version: Canary Versions

:sparkles: Test out this PR locally via:
npm install<EMAIL_ADDRESS># or
yarn add<EMAIL_ADDRESS>

:rocket: PR was released in<EMAIL_ADDRESS>:rocket:


Variables registry

All models in the Virtual Rainforest interact through variables stored in the data object. Dependencies of one model on another model essentially boil down to needing certain variables to exist in the data object, i.e. initialized by the model in question. We’ve slightly formalised this by introducing a configuration system to order init and update methods across modules, but in the main it remains quite loosely defined.
What would be useful would be a system that registers all variables, along with information about which module initialises them, which modules use them, and which modules updates them. This could then be used to constrain what sets of models can be run together and provide information of what variables need to be added to get a set of models to play nicely with each other. This becomes even more relevant when we have different versions of the same components, for example a simple and a mechanistic version of the abiotic model. This should also help automate the  documentation of variables used in the models, which currently documented manually.
As a sketch:

Variable descriptions are created from a new ABC that defines a unique variable name along with properties like dimensionality, a description and the expected units.
Individual models might then specify required_init_vars, updated_vars and required_update_vars. A particular set of models then defines which variables need to be provided in the input data, and potentially defines solutions for the model init  and update order.
Creating a new Variable instance automatically adds that variable to a registry, which checks for pre-existing usage of a name.
It might be better to have a separate variables module, so that models then import the variable definitions that they need. That makes it easier to avoid clashes.  However, in the long-term, we want users to be able to add their own models and so these probably should be able to declare their own variables. Maybe we just need to define Variable.__eq__ so that we can check if a variable declaration is identical?

Closed by #388


assemble in PPC

Inline editing, assembling, and right-click patch NOP are not implemented in PPC.

The nop and assemble are implemented in 766b3584e028fe007d483157050e863d48d53b8e.
Note that PPC assembler dialogue obeys the LLVM syntax which differs from what you see in disassembly. For example the disassembly lis r8, 0x1000 requires you to enter lis 8, 1000 (isolated numbers indicate registers).

LLVM won't accept a numerical displacement for branch, like b -0x20 but Ben Rodes found that if you make a little expression out of it, like b -1*0x20 it's accepted. Make sure the result would be on 4-byte boundary.

Invert branch and skip-and-return-value are implemented in<PHONE_NUMBER>ce7d2d317e2d146c4f71f21bd557d3, closing! Hope it builds!


Update to byteorder 1.0

Thanks! I'm currently updating the library, hoping to finish the webp decoder and fix other minor things, and bumping dependencies versions will be one of them. I'll merge this but I don't think I'm going to release the updated version right now.


Fix: incline would not hide on cursorline without tabs.

Resolves https://github.com/b0o/incline.nvim/issues/34.
I don't know the Vim API surface very well so this is cobbled together.
If something like has_tabline_showing exists that would be much better.

Closed in favour of https://github.com/b0o/incline.nvim/pull/36.


returnNew support

In the doc, https://docs.arangodb.com/3.0/Manual/DataModeling/Documents/DocumentMethods.html#insert, we can see that document.insert can return the newly created document.
In arangojs, collection.insert does not exist, only collection.save. Is this method can support returnNew ?
Regards,
Sylvain

The methods insert and save are identical. I'm not sure why the save method is not in the official documentation as the examples use it almost exclusively.
We can add support for returnNew as the HTTP API also supports this option but for the time being you can simply Object.assign the document to the return value (which only contains the _id, _key and _rev attributes):
const doc = {some: 'data'};
myCollection.save(doc).then((meta) => Object.assign(meta, doc)).then((savedDoc) => ...);


Wrong link to the advanced docs

https://docs.spring.io/spring-cloud-contract/docs/3.0.0/reference/html/documentation-overview.html#advanced-topics points to advanced.html.html
https://docs.spring.io/spring-cloud-contract/docs/3.0.0/reference/html/project-features.html#features-whats-next doesn't point to advanced.html
the documentation code can be found under docs/src/main/asciidoc

Hey @marcingrzejszczak! I would like to work on this issue. Can you please assign this to me?

Hey @marcingrzejszczak! I would like to work on this issue. Can you please assign this to me?

 I'm working on that.

Thanks, @marcingrzejszczak! I'm working on that.


ARROW-13246: [C++] Set partial to remainder of buffer

When rows are skipped but there are still more rows to skip set partial to the remainder of buffer.

LGTM.  I think the new streaming reader won't quite work with this fix since it infers schema on the first block of data. However, it shouldn't be too much work to change that to "first block that has actual data". I'd be happy to do that (probably in a follow-up PR).

That is an unfortunate limitation of the current streaming reader too. I was thinking it would be nice to have this work for all the readers but haven't created a ticket. If you don't mind doing that after your current PR is in that would be great but if you don't have time I could take a swing at it.


Cannot run Graph API query after logging in

Hello
I have setup authentication for users to login to my app, and this is working.  Users can login are are generated a token.  Decoding a token for my user session on https://jwt.ms/ shows me that the token has my user account details and scopes as:
"scp": "Directory.Read.All email openid profile User.Read User.Read.All",

The app setup in Azure AD has been configured by my IT colleagues as having these permissions "Delegated".  Additionally, Directory.Read.All and User.Read.All are grated as "Application".  Admin authority has been given.
However, when I then try to make any requests to the Graph API I get the following HTTP 401:
Access token validation failure. Invalid audience.

My code looks like this:
// This is the token that was stored as part of the authentication flow and then stored on the session
// $accessToken = $this->fetchAccessToken($client);
// $request->getSession()->set('azure.access.token', $accessToken);
$token = $request->getSession()->get('azure.access.token', '');

$graph = new Graph();
$graph->setAccessToken($token);
$result = $graph
                ->createRequest('GET', '/me')
                ->execute();

I am unsure what more I need to do to get this to work, I feel I may be missing a stage in the process flow but any help appreciated!

In your JWT, what is the claim's 'aud' value ?

@decomplexity
"aud": "https://graph.windows.net/",

Does this help?

And yes... yes it does help!
It made me look at why you might ask that, and whether that is wrong.
I identified that I need to exchange my "login" token for a "graph" token:
// Swap our login token for a graph token
/** @var Azure $provider */
$provider = $client->getOAuth2Provider();
$provider->urlAPI = 'https://graph.microsoft.com';
$provider->resource = 'https://graph.microsoft.com/';
$graphToken = $provider->getAccessToken('refresh_token', [
    'refresh_token' => $accessToken->getRefreshToken(),
    'resource' => 'https://graph.microsoft.com'
]);

I then use the graph token on the graph request - and boom! Got the data I need.
Thank you @decomplexity for the prompt.

Hmm.... graph.windows.net is the older AAD Graph library. You could try using the MSAL library endpoint graph.microsoft.com. The Access token AUD has to be identical to the resource API's address or the API will bounce authorization (unless MSFT has somehow aliased the two).
The AUD need not 'look' identical since it may be in the form of a client ID, URI or a 128-bit GUID (the V1 AAD could also be a resource ID) but all refer to the same thing


letsencrypt problem after upgrade to 0.9

After upgrading from 0.8 to 0.9 on a DigitalOcean installation, the nginx-container restarts exactly after 60 seconds. I can reach the login page, but here is "no response" to the request.
Log of nginx-container indicates a problem with letsencrypt. I have followed https://docs.getodk.org/central-upgrade/ including firewall disabling. All other containers run ok.
nginx: [emerg] BIO_new_file("/etc/selfsign|letsencrypt|customssl/live/local|your.domain.com/fullchain.pem") failed (SSL: error:02001002:system library:fopen:No such file or
 directory:fopen('/etc/selfsign|letsencrypt|customssl/live/local|your.domain.com/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
writing a new nginx configuration file..
starting nginx without certbot..
2020/07/27 09:23:49 [emerg] 10#10: BIO_new_file("/etc/selfsign|letsencrypt|customssl/live/local|your.domain.com/fullchain.pem") failed (SSL: error:02001002:system library:f
open:No such file or directory:fopen('/etc/selfsign|letsencrypt|customssl/live/local|your.domain.com/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such fi
le)
nginx: [emerg] BIO_new_file("/etc/selfsign|letsencrypt|customssl/live/local|your.domain.com/fullchain.pem") failed (SSL: error:02001002:system library:fopen:No such file or
 directory:fopen('/etc/selfsign|letsencrypt|customssl/live/local|your.domain.com/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

git did an overwrite of the .env file without complaining, therefore it reads "your.domain.com" in the error message. Restored previous .env.

Great to hear it’s working for you now! We’d appreciate it if you could write up issues you experience on the community forum as a support request. First, it greatly increases the visibility of your post. In this case, I’m pretty sure others will have a similar experience and having the question and answer in the forum makes it more readily available to others to learn from. Second, it’s sometimes a bit hard to know what repository the issue belongs in before some troubleshooting. It’s simpler for you to post in one place and then we can all work to triage together as we identify the action to take.
Really appreciate all the clear feedback you’ve been giving us! 😊 Hopefully this information makes it even easier to feed your findings back into the community so everyone benefits.


Add license package metadata

Hi, we have a small automated tool for verifying installed dependencies licenses (to make sure we can use them in our project) and Levenshtein_search does not provide license metadata (official docs), meaning we have to utilize GitHub API to retrieve it.
Could you please specify the branch to which we should open a PR with the change (as there are currently two in this project with develop being fresher)?
Thank you for your time.

Hi, I merged the branches, so Master is the only one remaining


JupyterCAD icon should be highlighted in the file browser when the file is selected

I presume this would be achieved by adding the right CSS classes to the SVG files.
Screenshots below show the change os style of the selected file with a JCAD file vs a notebook.

Related to https://github.com/jupytercad/JupyterCAD-FreeCAD/issues/14.

We may not need to take any action on this if we change the design of the logo and the new logo fits nicer with the blue background.
Let's see if we need to take any action once that new logo is settled.

We may not need to take any action on this if we change the design of the logo and the new logo fits nicer with the blue background.

It makes sense to wait for the new logo. Although even with the new logo, we should probably apply the selected style like for all other core Jupyter file types.


Update to include Report List Hierarchy and viewer for website

Great project.
I needed to expand it for a client who wanted to see a hierarchy of reports (included nestable jquery project to help with that) and display the report beside it. Thought you might be interested to add my changes it to your repo. Here are screen shots of the added "report list" link. First with a root level report and then with a secondary level report selected. Trick was that I needed to add a controller method to return the iframe result so that javascript could add it to the html dynamically rather than via the static Razor syntax.
Let me know if you have any questions.
Selected Report1 from root

Selected Child report Report2

Hi,
Thank you for your help with the project. I appreciate it! Unfortunately I am not going to merge your changes to the root repository. The goal of MvcReportVIewer helper is to have a simple wrapper for ReportViewer control and do not enhance it too much.
I really like your solution. I am going to add a link to your repository and this description to project's wiki, so other people will be able to use it.
Sincerely,
Ilya


rename 'master' branch to 'main'

Both Conservancy and the Git project are aware that the initial branch name, ‘master’, is offensive to some people and we empathize with those hurt by the use of that term.

https://sfconservancy.org/news/2020/jun/23/gitbranchname/
https://github.com/github/renaming

f you


Add support for automatically updating October

This should make it less fiddly to get the latest fixes
See: https://github.com/marcus-crane/october/issues/65

Doing experimentation over at https://github.com/marcus-crane/wails-autoupdater since I need to create releases to test this stuff out

I got a bit stuck on how I might implement Windows updating but I believe I can just unzip the installer and quietly run through it with stock settings (there aren't any installer options anyway)

I've been keeping an eye out for how things are done on Windows and I've seen a couple implementations that just close the application and pop open the installer, leaving the user to run through it so realistically, that could be a reasonable way to go.
There might not be much value in having the user run through it unless there are actually some options to toggle (and there aren't)


 An ability to manually upload an image would be good

duplicate of #12


Use Datakit as CI for Moby

Current status:

Builds alpine/kernel OK
Builds lots of packages
Fails cloning docker repository

Cloning into 'docker.git'...
error: Unable to create /database/branch/initrd-of-5da84d3c468a6ee03d550e74b58bba9439e362a8/transactions/log/rw/alpine/packages/docker/docker.git/.git/HEAD

The immediate issue is that we don't support rename in transactions. However, running Git inside a transaction is probably a bad idea anyway (will need cross-directory rename sooner or later, which 9p doesn't support).
Ideally, I think we want to handle cloning of repositories inside Datakit itself (which would be more efficient than downloading the whole thing on every build). However, a short-term fix would be to clone the repository into /tmp or similar.

Closing this one as I believe it now works properly.


chore: remove benchmark-like test for faster build time

On a macbook air, 1m+ build time -> 20s build time

Thanks!! release via https://github.com/paisano-nix/tui/releases/tag/v0.4.1


why Hub routes doesn't support Attribute routes on the Hub class itself, this is will be more easier than configure it in the stratup.cs

If you believe you have an issue that affects the security of the platform please do NOT create an issue and instead email your issue details to<EMAIL_ADDRESS>Your report may be eligible for our bug bounty.
Please consider the following before filing an issue

If you are using NuGet Packages that start with Microsoft.AspNet.SignalR, use the issue tracker at https://github.com/SignalR/SignalR to report the issue. This
repository is for packages that start with Microsoft.AspNetCore.SignalR (and NPM packages that start with @aspnet/signalr)

Please include as much of the following as you can in your bug report

Versions of Server-Side NuGet Packages:
Versions of Client-Side NuGet/NPM Packages:
Are you using the C# client or the JavaScript client:
The Server you are using (Kestrel/HttpSysServer/IIS/IIS Express/Azure Web App/etc.):
The Operating System on the Server (Windows/Linux/macOS):
The Operating System on the Client (Windows/Linux/macOS):
The Browser on the client, if using the JavaScript client (IE/Chrome/Edge/Firefox/etc.):
Please use the Diagnostics Guide at https://github.com/aspnet/SignalR/wiki/Diagnostics-Guide to collect server-side and client-side logs and network traces if possible. Please do not put the traces and logs in the Issue text as they are quite large. The Diagnostics Guide describes how to attach them to the issue.

When in doubt, feel free to file the issue, we're happy to help answer questions. We also suggest using the asp.net-core-signalr tag on StackOverflow to ask questions.

We'll likely be looking at this in detail when we look at integrating with Endpoint Routing in ASP.NET Core 3.0. I filed https://github.com/aspnet/SignalR/issues/3224 to track that work. Closing this as a routing attribute will be part of that work.


Unexpected char '99'

I have code:
url = 'http://path:80/v1/chain/get_table_rows'

headers = {"content-type": "application/x-www-form-urlencoded; charset=UTF-8"}

def get_table_data():

    payload = {
        "code": "string",
        "table": "contribs"
    }

    resp = requests.request("POST",url, data=payload, headers=headers)
    return resp

error:
b'{"code":500,"message":"Internal Service Error","error":{"code":4,"name":"parse_error_exception","what":"Parse Error","details":[{"message":"Unexpected char '99' in \"code\"","file":"json.cpp","line_number":437,"method":"variant_from_stream"},{"message":"","file":"json.cpp","line_number":460,"method":"from_string"}]}}'

need to add
payload = json.dumps(payload)


Bug Fix: Empty responses fails when sending messages

Sending Messages with empty responses for a SuggestedResponseKeyboard fails.
Only adds a keyboard if their are responses for the keyboard.

@Eagerod how about this

Seems like this is more intended behaviour then anything. If someone made suggested responses with an empty list or None they made a mistake,


How to let the braces line feed for functions

VSCode Version:
OS Version:

Steps to Reproduce:

function test ()
{
}
2.code format(alt+shift+f)
function test () {
}
How to let the braces line feed for functions

It depends on the formatting extension and the language you are coding. Just install one of them and edit the configuration, example: https://github.com/prettier/prettier for Javascript

Try the setting:   "javascript.format.placeOpenBraceOnNewLineForFunctions": true

Thank you all.
Is there a support brace on newline formatting extension for JAVA?


DeviantArt rss feeds disabled depending on server?

Hi, I've been using BazQux rss reader to view the deviations on dA from people and they all of a sudden stopped. According to the person that manages the reader the German and US servers were blocked from receiving the rss feed data.
He told me that: "They could use: dig bots.bazqux.com to find all the IPs reader can make requests from."
If it's possible I'd love to be able to see this unblocked. Or at least an explanation as to why it's happening. Thanks

Apologies, some of our rules to block traffic due to DOS attacks needed to be adjusted. Please let us know if things are fixed now, thanks!

Apologies, some of our rules to block traffic due to DOS attacks needed to be adjusted. Please let us know if things are fixed now, thanks!

That seems to have fixed it! Thanks so much! I appreciate everything you guys manage in the background


Add simple script for linting pan code

Mostly coding style conventions right now.

Nice script - I've wanted this for ages! :-) I don't think "stupid" is an accurate description though so can you please amend the commit log? Also how about a few tests too? Not expecting 100% coverage but a skeleton to make sure the code structure is testable so more can be added in the future.

@ned21 @jrha maybe we should start a python-quattor? we have a bunch of python code, including converting pan to python and writing it out again, and script to cleanup most of these linting errors.
we could contribute it (and a separate repo could help with the unittesting).
maybe open an issue and flag it "discuss at workshop"?

@jrha good initiative! As @ned21 I don't think it deserves the adjective stupid... I tried it quickly on a couple of templates from the template library. I see one potential problem: you enforce the usual "identation as a multiple of four spaces" but in the template library we standardized year ago on using a multiple of two spaces... That means that the whole template library has to be updated... Would it be acceptable to use "a multiple of 2 spaces" that should also work if you use 4...

@stdweird do you mean a new repo when you say python-quattor? If yes, I am not sure this is a good idea to structure our repo by languages... There are already quite a number of Python script in other repos and I had prefer to keep the current structure of repos based on "components". That said, if you have scripts related to pan that you think may be useful to share, why not to add them to this repo as the same place as panlint?

I agree with @jouvin, repos should group our tools by subject, not the language they happen to be written in, however we should still look at some static analysis for out python code.

I agree about the static analysis of our Python code, is not pylint the appropriate solution?

Yep, but probably wrapped in something like codeclimate (which I've just added to this repo to try).

Changes Unknown when pulling fc4f053b4fec86bfff0843ed402816f4d578538d on jrha:panlint into ** on quattor:master**.

LGTM. Just need to get the CodeClimate test to pass and then we can merge

Hmpf, looks like Jenkins broke itself.

test this please

Woohoo! :green_heart:

@ned21, @stdweird all tests are passing now, so could this be merged?

@jrha just a bunch of minor issues
LGTM otehrwise

Fixed.

@jrha thanks a lot!


[4.0] Encode Decimal as a numeric value in JSON

What's in this pull request?
Cherry-picks #10539 to swift-4.0-branch.
Explanation: Allow JSONEncoder/JSONDecoder to intercept Decimal values so they get a numeric representation in JSON (instead of their default keyed implementation).
Scope: Affects those who want to read large decimal values representable by Decimal natively from JSON.
Radar: rdar://problem/32926804
Risk: Low
Testing: Adds unit tests to confirm expected behavior.

@swift-ci Please test macOS

@swift-ci Please smoke test Linux

@swift-ci Please test macOS

@swift-ci Please test Linux

@tkremenek Thanks!