WIP: n-density dry model

I have an implementation of an n-density dry model that seems to be working reasonably well using energy as the prognostic thermodynamic variable ...
Energy, one gas
Energy, three gases
... but that doesn't work particularly well with entropy ...
Entropy, one gas
Entropy, three gases
I think the issue is representing entropy production by mixing, which is hard to compute accurately. The rate of entropy production by mixing depends non-linearly on mass mixing rates and the entropy of a specific constituent blow up at low densities, and those two things together make me suspect that it will be hard to model mass diffusion when using entropy as the thermodynamic variable without producing spurious energy/pressure/temperature sources. (It's also possible there's a bug in the code I wrote...)
I'm labeling this as work in progress (and think we should hold off on merging for now) because the code is getting a little clunky. Some additions to the code I think would be useful include:

a more robust approach to tracking which model tracers are the thermodynamic variable and the densities (right now the code assumes that they're at certain indices of model.tracers)
a more streamlined way to include diagnostic fields in operators. #59 would address this.
some way to enforce boundary conditions on diagnostic fields (e.g. pressure, diffusive fluxes). Right now this can be done indirectly by setting boundary conditions on prognostic fields, but having a more direct way to apply them would simplify the code (and let us avoid hacks like the one in diffusive_pressure_flux_z in compressible_operators.jl, which is currently there because I couldn't figure out any other way to ensure the flux was zero at the top and bottom boundaries). #59 could also address this.

@ali-ramadhan, let me know if you're interesting in working on any of these additions. (Might be worth doing some pair programming so we make sure we're both happy with the resulting code.)
It also might be worth adding some regression tests before we start working on any of the above so that we have a way of making sure we're not changing model behavior (apart from fixes to any bugs we find) while refactoring.

Also want to note that this branch has or will have changes that address #49 (but need to update the notes from that issue), #52, #54, #55, #56, #57, #58, and #59.

Pretty epic PR! Super cool to see the three gas thermal bubble and the difference between energy and entropy.
For entropy, I'm not familiar with the spurious mixing issue but it seems really bad with the three gas thermal bubble. Even with the single gas case (both energy and entropy) you get numerical artifacts but that could just be the advection scheme. Quick test would be to see if increasing the resolution reduces those numerical artifacts.
But in general, good advection scheme seems pretty important.
Just skimmed through the changes but I won't review yet as it's a WIP. If there's a bug with entropy hopefully we'll find it during review or pair programming.

a more robust approach to tracking which model tracers are the thermodynamic variable and the densities (right now the code assumes that they're at certain indices of model.tracers)

Hmmm, yeah this might warrant us taking a slightly different approach than for incompressible models. I think a big benefit of having them be part of model.tracers is that they can be processed and time-stepped together.
One solution could be to define model.thermodynamic_variables and model.densities to be a named tuple of pointers to specific tracers in model.tracers. So we can treat them all similarly when calculating right hand sides or time stepping, but can pass in just model.thermodynamic_variables when needed, for example in a pressure calculation or microphysics scheme.

a more streamlined way to include diagnostic fields in operators. #59 would address this.

True. Maybe model.tracers can hold the prognostic fields while properties like model.thermodynamic_variables and model.velocities can hold lazy primitive fields.

some way to enforce boundary conditions on diagnostic fields (e.g. pressure, diffusive fluxes). Right now this can be done indirectly by setting boundary conditions on prognostic fields, but having a more direct way to apply them would simplify the code (and let us avoid hacks like the one in diffusive_pressure_flux_z in compressible_operators.jl, which is currently there because I couldn't figure out any other way to ensure the flux was zero at the top and bottom boundaries). #59 could also address this.

Ah sorry yeah the boundary conditions in master are still hacked in as I was hoping to include each field's boundary conditions in the Field struct (https://github.com/climate-machine/Oceananigans.jl/issues/606).
But yeah we definitely want to avoid hard-coding in boundary conditions like that!
Hmmm, this one seems like it could be a bit tricky if we really need boundary conditions and halo filling on diagnostic fields. Filling halos on the fly or on demand for lazy computations could be expensive.
It would be nice if we can get away with just imposing boundary conditions on prognostic fields so that lazy evaluations of the diagnostic variables will never have to worry about boundary conditions.
Hmmm, looking into diffusive_pressure_flux_z, I think it should be able to diagnose a zero pressure gradient if the thermodynamic variable has no-flux boundary condition (zero gradient or no-flux at the boundary), which should produce p/ρ = constant at the boundary. Then ∂zᵃᵃᶠ(i, j, k, grid, p_over_ρ_diag, tvar, g, Ũ, ρ, ρ̃, C̃) = 0. But that must have not worked as you had to hard code in the no-flux boundary conditions. Looks like halo regions are filled correctly during time stepping too.
Worse case even if we had to "fill in halo regions" for lazy diagnostic fields I think we can do it generally and there will be a performance hit but it should be rather negligible.

It also might be worth adding some regression tests before we start working on any of the above so that we have a way of making sure we're not changing model behavior (apart from fixes to any bugs we find) while refactoring.

True. I guess we don't have easy output writers built in yet (#38) but should be easy to output/load a few fields for the purposes of regression testing.
Been thinking of moving diagnostics and output writers out of the model and into a Simulation type (https://github.com/climate-machine/Oceananigans.jl/issues/447) so I haven't added an output_writers property field to CompressibleModel yet.

@ali-ramadhan, let me know if you're interesting in working on any of these additions. (Might be worth doing some pair programming so we make sure we're both happy with the resulting code.)

Pair programming could be a good idea, especially to settle on design choices.
I'll try to push through a couple of upstream issues: https://github.com/climate-machine/Oceananigans.jl/issues/606 so it'll be much easier to add boundary conditions on all fields here, and https://github.com/climate-machine/Oceananigans.jl/issues/447 so we can start using diagnostics and output writers (and get rid of the time stepping loop).
But maybe more importantly, I can work on adding lazy fields (https://github.com/thabbott/JULES.jl/issues/59) which seem like they might be a much needed abstraction for a CompressibleModel.

Also want to note that this branch has or will have changes that address

I think if you include a list like
Resolves #49
Resolves #52
...

(but without the code block) in your original comment then those issues will be automatically closed when this PR is merged.

Even with the single gas case (both energy and entropy) you get numerical artifacts but that could just be the advection scheme. Quick test would be to see if increasing the resolution reduces those numerical artifacts.

I think the solutions in this branch might be more oscillatory than master in part because diffusion is less strong---master has κ = 0.5 m^2/s, but with a bug in the diffusion operators that makes this equivalent to κ = 500 m^2/s in this branch, whereas the videos I posted use κ = 75 m^2/s. I'm also using half the spatial resolution. So we could make the verification experiments look nicer pretty easily (but having better advection operators eventually would be good too).

One solution could be to define model.thermodynamic_variables and model.densities to be a named tuple of pointers to specific tracers in model.tracers. So we can treat them all similarly when calculating right hand sides or time stepping, but can pass in just model.thermodynamic_variables when needed, for example in a pressure calculation or microphysics scheme.

Something like this seems good to me---I agree that keeping the thermodynamic variable and densities in the list of tracers is convenient, and this would give a straightforward and relatively low-cost way to access them when we need them individually.

True. Maybe model.tracers can hold the prognostic fields while properties like model.thermodynamic_variables and model.velocities can hold lazy primitive fields.

It might make sense to store all the lazy diagnostic fields in another struct called something like model.diagnostic_variables so that we clearly distinguish them from variables that get stepped forward in time. But we can think more about the best way to organize things.

Hmmm, looking into diffusive_pressure_flux_z, I think it should be able to diagnose a zero pressure gradient if the thermodynamic variable has no-flux boundary condition (zero gradient or no-flux at the boundary), which should produce p/ρ = constant at the boundary. Then ∂zᵃᵃᶠ(i, j, k, grid, p_over_ρ_diag, tvar, g, Ũ, ρ, ρ̃, C̃) = 0. But that must have not worked as you had to hard code in the no-flux boundary conditions. Looks like halo regions are filled correctly during time stepping too.

The problem comes from the fact that energy includes kinetic energy and halos for the vertical velocity field aren't filled the same way as other variables at the top and bottom boundaries. So even though the energy gradient is zero at boundaries the temperature and pressure gradients aren't. Calculating diffusive fluxes using lazy field with its own boundary condition might let us do something like
struct LazyGradient{B}
    scalar :: Field
    bc :: B
end

getindex(A::LazyGradient, inds...) = gradient(i, j, k, A.scalar.grid, A.scalar, A.bc)

function gradient(i, j, k, grid, scalar, bc::AllPeriodic)
   xcomp = ∂x(i, j, k, grid, scalar)
   ycomp = ∂y(i, j, k, grid, scalar)
   zcomp = ∂z(i, j, k, grid, scalar)
   return (xcomp, ycomp, zcomp)
end

function gradient(i, j, k, grid, scalar, bc::NoGradientZ)
   xcomp = ∂x(i, j, k, grid, scalar)
   ycomp = ∂y(i, j, k, grid, scalar)
   zcomp = (k == 1 || k == grid.Nz) ? 0.0 : ∂z(i, j, k, grid, scalar)
   return (xcomp, ycomp, zcomp)
end

This doesn't require extra halo filling but also handles the case where halo filling doesn't do what we'd like it to.

I'll try to push through a couple of upstream issues: climate-machine/Oceananigans.jl#606 so it'll be much easier to add boundary conditions on all fields here, and climate-machine/Oceananigans.jl#447 so we can start using diagnostics and output writers (and get rid of the time stepping loop).
But maybe more importantly, I can work on adding lazy fields (#59) which seem like they might be a much needed abstraction for a CompressibleModel.

Sounds good to me!

Of course it took longer than expected but I've made some changes to Oceananigans.jl that will benefit JULES so I'll update JULES to use the latest version and start with building up some lazy fields.
A quick summary:

Grids now have a topology which help inform boundary conditions (https://github.com/climate-machine/Oceananigans.jl/pull/614, https://github.com/climate-machine/Oceananigans.jl/pull/620)
time_step! now takes a single time step. A high-level Simulation abstraction handles time stepping, diagnostics, and output writing (https://github.com/climate-machine/Oceananigans.jl/pull/621).
Oceananigans.jl now implements an IncompressibleModel so there's room for a CompressibleModel (https://github.com/climate-machine/Oceananigans.jl/pull/626).
Boundary conditions are now a field property so JULES can readily support arbitrary boundary conditions (https://github.com/climate-machine/Oceananigans.jl/pull/631).

I updated the time stepping a bit after the model reorganization.
I also updated the tests and noticed some out of bounds errors that probably weren't caught since they happen inside @inbounds blocks. Tests are nice like that as they force bounds checking.
How should potential energy and temperature be computed in the halo regions? Right now we have Φ = gravity * grid.zC[k] so we need to make a choice about zC[0] and zC[Nz+1] or about Φ[0] and Φ[Nz+1].
Right now I'm just doing
k′ = k
k < 1 && (k′ = 1)
k > grid.Nz && (k′ = Nz)
Φ = gravity * grid.zC[k′]

which is probably not what we want but is probably better than silently accessing ranges out of bounds.
Updated and ran the 3-gas dry rising thermal bubble with energy verification and movie still looks good so looks like I didn't break things too much.

Hmmm, I think something we changed led to a performance regression and lots of allocations (model should ~10x faster I think):
 ──────────────────────────────────────────────────────────────────────────────────
    Static atmosphere benchmarks           Time                   Allocations
                                   ──────────────────────   ───────────────────────
         Tot / % measured:              49.3s / 6.85%           15.5GiB / 6.87%

 Section                   ncalls     time   %tot     avg     alloc   %tot      avg
 ──────────────────────────────────────────────────────────────────────────────────
 32×32×32 [CPU, Float64]       10    3.38s   100%   338ms   1.06GiB  100%    109MiB
 ──────────────────────────────────────────────────────────────────────────────────

Probably would be hard to find the culprit. We think it was working well in commit https://github.com/thabbott/JULES.jl/pull/60/commits/6327e2f720c733b91e5d3034a9c5d6a16a83bf18

Good catch with the out-of-bounds accesses! We want boundary conditions on Φ to match boundary conditions on energy, so what you're doing is correct for no-flux top and bottom boundaries, but probably not something we want to keep hard-coded into the geopotential calculation. We could always add a new lazy field for Φ that can hold its own boundary conditions....
I can take a shot at tracking down the source of the allocations if you'd like. (It'd be a good way to get re-acquainted with the code after the changes you made.)

I can take a shot at tracking down the source of the allocations if you'd like. (It'd be a good way to get re-acquainted with the code after the changes you made.)

That would be sweet. We can then add a performance regression test to ensure we don't slow down the model by mistake.
Then I can make some more substantial changes so JULES can use the boundary conditions, diagnostics, output writers, etc. from Oceananigans.

Resolves #49
Resolves #52
Resolves #54
Resolves #55
Resolves #56
Resolves #57
Resolves #58
Resolves #59


Cognito User Group data

User and groups data created in Cognito, does Data All stores it in RDS or any other storage?

Hi @sandeephs1, thanks for the question.
Users and groups created in Cognito are only stored in Cognito. Data.all creates permissions for each group once at least one user belonging to that group has logged into data.all. In the table tenant_policy we store these "active" groups in the principalId column: dataall/core/permissions/db/permission_models.py:33. Users are not stored in RDS.
Let us know if you need more clarifications. Also, why are you looking into this information? Are you going to use it or do you need to ensure that those details are not collected?

Thanks @dlpzx
we are planning for external tool integration with Data all, which would require us to manage users and groups between Data all and External tool. So wanted to confirm if user group data is stored in any other object/source

Thanks for the context @sandeephs1, in your scenario we have seen that other users of data.all manage users and groups directly in the external tool . Do you have any other pending questions?


ladder: remove lmb, rgo3, ti-mo from foundations

I understand @lmb and @rgo3 are aware of this and you're making the change on their behalf. Makes sense. If anything changes, feel free to send another update to rearrange your team membership.


【Help!!!!!】Get paper pdf

hi ,I'm sorry to bother you，
I can't find the address to download the paper from the Internet，
Can you send a paper to my email?
paper: FISS: A Trajectory Planning Framework Using Fast Iterative Search and Sampling Strategy for Autonomous Driving，
email<EMAIL_ADDRESS>Thanks a lot!

Hi there,
Thank you for your interest in our work!
Our paper is accessible on IEEE Xplore: https://doi.org/10.1109/LRA.2022.3191940
You may log in and download the paper from there. 😃
Regards,
Shuo


Notes - Grants Backlog

Overview
This is an issue to jot down a running backlog of grant ideas.
These may include possible RFP ideas, retroactive funding items, or other details.
Ecosystem
Proposal Automation
Amount: $$$
Person: @abbey-titcomb
Description: something that makes creating on-chain proposals easier
Delegationi Automation
Amount: $$$
Person: @abbey-titcomb
Description: something that makes delegating easier
Radicle Core
Naming of orgs (retroactive funding)
Amount: 2000 USDC
PR: https://github.com/radicle-dev/radicle-upstream/pull/2180

Definitely see a grant in process for developing our governance "stack". We need an easier dashboard/tool for proposing, voting, and delegating. Have a ton of feedback here and honestly would love to put out a RFP to the ecosystem for helping us build out our governance infra. I think it could either be our own platform or a hodge-podge of different governance tools.

@abbey-titcomb cool!
Definitely can think of some iterative v1 -> v2 -> vN type ideas here
Because Radicle provides such an interesting primitive with NFT-based membership, I think a good north star here is having something that feels "natively Radicle" such that people can take care of all their code/development governance all within 1 app.
But yeah, I imagine v1 would be like...whatever works better than the current process :)


Failing specs

I've just cloned the project, run the specs and it seems 10 of them are failing right now:
mangopay2-ruby-sdk git:(master) rspec spec
.....................................FF.....FF...........FF.F......FFF........................

Failures:

  1) MangoPay::KycDocument CREATE PAGE accepts File instance
     Failure/Error: MangoPay::KycDocument.create_page(new_natural_user['Id'], new_document['Id'], file)
     MangoPay::ResponseError:
       One or several required parameters are missing or incorrect. An incorrect resource ID also raises this kind of error. File: The type of the file submitted could not be determined - please try using a different file type
     # ./lib/mangopay.rb:93:in `request'
     # ./lib/mangopay/kyc_document.rb:51:in `create_page'
     # ./spec/mangopay/kyc_document_spec.rb:72:in `create_page'
     # ./spec/mangopay/kyc_document_spec.rb:77:in `block (3 levels) in <top (required)>'

  2) MangoPay::KycDocument CREATE PAGE accepts base64-encoded string
     Failure/Error: MangoPay::KycDocument.create_page(new_natural_user['Id'], new_document['Id'], file)
     MangoPay::ResponseError:
       Internal Server Error
     # ./lib/mangopay.rb:93:in `request'
     # ./lib/mangopay/kyc_document.rb:51:in `create_page'
     # ./spec/mangopay/kyc_document_spec.rb:72:in `create_page'
     # ./spec/mangopay/kyc_document_spec.rb:83:in `block (3 levels) in <top (required)>'

  3) MangoPay::PayIn::Card::Direct REFUND refunds a payin
     Failure/Error: refund = MangoPay::PayIn.refund(payin['Id'], {AuthorId: payin['AuthorId']})
     MangoPay::ResponseError:
       Internal Server Error
     # ./lib/mangopay.rb:93:in `request'
     # ./lib/mangopay/http_calls.rb:71:in `refund'
     # ./spec/mangopay/payin_card_direct_spec.rb:42:in `block (3 levels) in <top (required)>'

  4) MangoPay::PayIn::Card::Direct CASH FLOW changes balances correctly
     Failure/Error: refund = MangoPay::PayIn.refund(payin['Id'], {AuthorId: payin['AuthorId']})
     MangoPay::ResponseError:
       Internal Server Error
     # ./lib/mangopay.rb:93:in `request'
     # ./lib/mangopay/http_calls.rb:71:in `refund'
     # ./spec/mangopay/payin_card_direct_spec.rb:63:in `block (3 levels) in <top (required)>'

  5) MangoPay::PreAuthorization CREATE creates a new card pre-authorization
     Failure/Error: expect(created['Status']).to eq('SUCCEEDED')

       expected: "SUCCEEDED"
            got: "FAILED"

       (compared using ==)
     # ./spec/mangopay/preauthorization_spec.rb:14:in `block (3 levels) in <top (required)>'

  6) MangoPay::PreAuthorization UPDATE updates a card pre-authorization
     Failure/Error: updated = MangoPay::PreAuthorization.update(created['Id'] ,{
     MangoPay::ResponseError:
       One or several required parameters are missing or incorrect. An incorrect resource ID also raises this kind of error. PaymentStatus: PaymentSatus value has to be 'WAITING' to edit a PreAuthorization
     # ./lib/mangopay.rb:93:in `request'
     # ./lib/mangopay/http_calls.rb:20:in `update'
     # ./spec/mangopay/preauthorization_spec.rb:24:in `block (3 levels) in <top (required)>'

  7) MangoPay::Refund FETCH fetches a refund
     Failure/Error: resource = MangoPay::Transfer.refund(transfer['Id'], { AuthorId: transfer['AuthorId'] })
     MangoPay::ResponseError:
       Internal Server Error
     # ./lib/mangopay.rb:93:in `request'
     # ./lib/mangopay/http_calls.rb:71:in `refund'
     # ./spec/mangopay/refund_spec.rb:7:in `block (3 levels) in <top (required)>'

  8) MangoPay::Transfer CREATE creates a new Transfer
     Failure/Error: expect(trans['Status']).to eq('SUCCEEDED')

       expected: "SUCCEEDED"
            got: "FAILED"

       (compared using ==)
     # ./spec/mangopay/transfer_spec.rb:11:in `check_type_and_status'
     # ./spec/mangopay/transfer_spec.rb:21:in `block (3 levels) in <top (required)>'

  9) MangoPay::Transfer FETCH fetches a Transfer
     Failure/Error: expect(trans['Status']).to eq('SUCCEEDED')

       expected: "SUCCEEDED"
            got: "FAILED"

       (compared using ==)
     # ./spec/mangopay/transfer_spec.rb:11:in `check_type_and_status'
     # ./spec/mangopay/transfer_spec.rb:30:in `block (3 levels) in <top (required)>'

  10) MangoPay::Transfer REFUND refunds a transfer
     Failure/Error: refund = MangoPay::Transfer.refund(trans['Id'], {AuthorId: trans['AuthorId']})
     MangoPay::ResponseError:
       Internal Server Error
     # ./lib/mangopay.rb:93:in `request'
     # ./lib/mangopay/http_calls.rb:71:in `refund'
     # ./spec/mangopay/transfer_spec.rb:38:in `block (3 levels) in <top (required)>'

Finished in 7 minutes 36 seconds (files took 0.25313 seconds to load)
94 examples, 10 failures

Failed examples:

rspec ./spec/mangopay/kyc_document_spec.rb:75 # MangoPay::KycDocument CREATE PAGE accepts File instance
rspec ./spec/mangopay/kyc_document_spec.rb:81 # MangoPay::KycDocument CREATE PAGE accepts base64-encoded string
rspec ./spec/mangopay/payin_card_direct_spec.rb:40 # MangoPay::PayIn::Card::Direct REFUND refunds a payin
rspec ./spec/mangopay/payin_card_direct_spec.rb:54 # MangoPay::PayIn::Card::Direct CASH FLOW changes balances correctly
rspec ./spec/mangopay/preauthorization_spec.rb:6 # MangoPay::PreAuthorization CREATE creates a new card pre-authorization
rspec ./spec/mangopay/preauthorization_spec.rb:22 # MangoPay::PreAuthorization UPDATE updates a card pre-authorization
rspec ./spec/mangopay/refund_spec.rb:5 # MangoPay::Refund FETCH fetches a refund
rspec ./spec/mangopay/transfer_spec.rb:18 # MangoPay::Transfer CREATE creates a new Transfer
rspec ./spec/mangopay/transfer_spec.rb:26 # MangoPay::Transfer FETCH fetches a Transfer
rspec ./spec/mangopay/transfer_spec.rb:36 # MangoPay::Transfer REFUND refunds a transfer

Top 10 slowest examples (193.05 seconds, 42.3% of total time):
  MangoPay::PayIn::Card::Direct REFUND refunds a payin
    40.36 seconds ./spec/mangopay/payin_card_direct_spec.rb:40
  MangoPay::PayIn::Card::Direct CASH FLOW changes balances correctly
    37.77 seconds ./spec/mangopay/payin_card_direct_spec.rb:54
  MangoPay::Event FETCH accepts filtering params
    22.12 seconds ./spec/mangopay/event_spec.rb:8
  MangoPay::PayIn::PreAuthorized::Direct REFUND refunds a payin
    16.22 seconds ./spec/mangopay/payin_preauthorized_direct_spec.rb:40
  MangoPay::PayIn::PreAuthorized::Direct CASH FLOW changes balances correctly
    15.78 seconds ./spec/mangopay/payin_preauthorized_direct_spec.rb:54
  MangoPay::Transaction FETCH accepts filtering params
    13.93 seconds ./spec/mangopay/transaction_spec.rb:35
  MangoPay::Wallet FETCH TRANSACTIONS fetches list with two transactions after payin and payout done
    12.73 seconds ./spec/mangopay/wallet_spec.rb:48
  MangoPay::Wallet FETCH TRANSACTIONS accepts filtering params
    11.69 seconds ./spec/mangopay/wallet_spec.rb:61
  MangoPay::Transaction FETCH fetches list with two transactions after payin and payout done
    11.31 seconds ./spec/mangopay/transaction_spec.rb:22
  MangoPay::Transfer CASH FLOW changes balances correctly
    11.12 seconds ./spec/mangopay/transfer_spec.rb:51

Top 10 slowest example groups:
  MangoPay::PayIn::Card::Direct
    23.86 seconds average (95.43 seconds / 4 examples) ./spec/mangopay/payin_card_direct_spec.rb:1
  MangoPay::Event
    22.12 seconds average (22.12 seconds / 1 example) ./spec/mangopay/event_spec.rb:1
  MangoPay::PayIn::PreAuthorized::Direct
    13.46 seconds average (53.84 seconds / 4 examples) ./spec/mangopay/payin_preauthorized_direct_spec.rb:1
  MangoPay::Transfer
    10.18 seconds average (40.72 seconds / 4 examples) ./spec/mangopay/transfer_spec.rb:1
  MangoPay::Refund
    9.8 seconds average (9.8 seconds / 1 example) ./spec/mangopay/refund_spec.rb:1
  MangoPay::Transaction
    7.63 seconds average (30.54 seconds / 4 examples) ./spec/mangopay/transaction_spec.rb:1
  MangoPay::PayOut::BankWire
    7.58 seconds average (22.75 seconds / 3 examples) ./spec/mangopay/payout_bankwire_spec.rb:1
  MangoPay::PreAuthorization
    7.41 seconds average (22.23 seconds / 3 examples) ./spec/mangopay/preauthorization_spec.rb:1
  MangoPay::Wallet
    5.31 seconds average (37.14 seconds / 7 examples) ./spec/mangopay/wallet_spec.rb:1
  MangoPay::Client
    2.71 seconds average (8.12 seconds / 3 examples) ./spec/mangopay/client_spec.rb:1

I'd like to work on a PR to allow changing the Net:HTTP's timeout, but with the specs failing it's going to be hard to be sure I'm not breaking anything.

Hi @javiercr, are your specs still failing with the latest version? Normally everything is updated and fixed now but do let me know if not.

@hobailey all tests are passing now with the latest changes from master. Closing this, thanks!

Great, thanks for the update :-)


kie-issue#210: Keyboard shortcuts stop working after blurring out of some components on the BeeTable

This PR fix the following issues:
Closes kie-issue#210: Keyboard shortcuts stop working after blurring out of some components on the BeeTable
Closes kie-issue#214: Saving a file using keyboard shortcuts (Ctrl + S) on VS Code is not working if DMN Editor's Boxed Expression Editor is focused
Video of the issue#210 fixed. We had to comment a block of the code which introduced the issue#210 otherwise we had issues with delete rows and add rows  (see FIXME removed in the code). After the fix in this PR, we should not have those issues anymore. Video bellow playing with remove/add rows/columns, edit and selection:
https://github.com/kiegroup/kie-tools/assets/7305741/d52d5f4c-2686-4b0b-bae2-7e38882b76f2

@danielzhe LGTM. Thanks!


Unable to cast object of type 'Microsoft.Dynamics.Nav.CodeAnalysis.SourceTableExtensionTypeSymbol' to type 'Microsoft.Dynamics.Nav.CodeAnalysis.Symbols.TableTypeSymbol'.'

After converting a number of objects to AL, including a table extension and a report, I received the following error on compile:
Microsoft (R) AL Compiler version 0.12.15355
Copyright (C) Microsoft Corporation. All rights reserved
[Error - 8:59:30 AM] Please report this issue to https://github.com/microsoft/al/issues including information on how to reproduce it, if possible.
Processing of message 'al/createPackage' failed with error: 'Unable to cast object of type 'Microsoft.Dynamics.Nav.CodeAnalysis.SourceTableExtensionTypeSymbol' to type 'Microsoft.Dynamics.Nav.CodeAnalysis.Symbols.TableTypeSymbol'.'
StackTrace:
at Microsoft.Dynamics.Nav.CodeAnalysis.Emit.ReportMetadataEmitter.<>c__DisplayClass13_1.b__1() in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\Emitter\ReportMetadataEmitter.cs:line 264
at Microsoft.Dynamics.Nav.CodeAnalysis.Emit.ObjectMetadataEmitHelper.WriteStartEndElement(XmlWriter xmlWriter, String elementName, Action action) in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\Emitter\ObjectMetadataEmitHelper.cs:line 25
at Microsoft.Dynamics.Nav.CodeAnalysis.Emit.ReportMetadataEmitter.<>c__DisplayClass13_0.b__0() in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\Emitter\ReportMetadataEmitter.cs:line 252
at Microsoft.Dynamics.Nav.CodeAnalysis.Emit.ObjectMetadataEmitHelper.WriteStartEndElement(XmlWriter xmlWriter, String elementName, Action action) in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\Emitter\ObjectMetadataEmitHelper.cs:line 25
at Microsoft.Dynamics.Nav.CodeAnalysis.Emit.ReportMetadataEmitter.b__1_0(XmlWriter xmlWriter) in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\Emitter\ReportMetadataEmitter.cs:line 39
at Microsoft.Dynamics.Nav.CodeAnalysis.Emit.ObjectMetadataEmitter1.EmitMetadata(Action1 content) in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\Emitter\GenericObjectMetadataEmitter.cs:line 37
at Microsoft.Dynamics.Nav.CodeAnalysis.MethodCompiler.CompileObject(ObjectTypeSymbol containingType) in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\Compiler\MethodCompiler.cs:line 248
at Microsoft.Dynamics.Nav.CodeAnalysis.MethodCompiler.<>c__DisplayClass17_0.b__0() in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\Compiler\MethodCompiler.cs:line 216
at Microsoft.Dynamics.Nav.CodeAnalysis.Utilities.UICultureUtilities.<>c__DisplayClass5_0.b__0() in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\Utilities\UICultureUtilities.cs:line 152
at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Dynamics.Nav.CodeAnalysis.MethodCompiler.WaitForWorkers() in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\Compiler\MethodCompiler.cs:line 161
at Microsoft.Dynamics.Nav.CodeAnalysis.MethodCompiler.CompileMethodBodies(Compilation compilation, ModuleBuilder moduleBeingBuiltOpt, Boolean hasDeclarationErrors, DiagnosticBag diagnostics, Predicate1 filterOpt, CancellationToken cancellationToken) in C:\Agent\_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\Compiler\MethodCompiler.cs:line 145    at Microsoft.Dynamics.Nav.CodeAnalysis.Compilation.CompileImpl(ModuleBuilder moduleBuilder, Boolean emittingPdb, DiagnosticBag diagnostics, Predicate1 filterOpt, CancellationToken cancellationToken) in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\Compilation\Compilation.cs:line 810
at Microsoft.Dynamics.Nav.CodeAnalysis.Compilation.Compile(ModuleBuilder moduleBuilder, Boolean emittingPdb, DiagnosticBag diagnostics, Predicate`1 filterOpt, CancellationToken cancellationToken) in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\Compilation\Compilation.cs:line 787
at Microsoft.Dynamics.Nav.CodeAnalysis.CommandLine.CommonCompiler.PackageTheModule(Compilation compilation, CompilerTextWriter consoleOutput, CancellationToken cancellationToken, ErrorLogger errorLogger, AnalyzerDriver analyzerDriver) in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\CommandLine\CommonCompiler.cs:line 531
at Microsoft.Dynamics.Nav.CodeAnalysis.CommandLine.CommonCompiler.RunCore(CompilerTextWriter consoleOutput, ErrorLogger errorLogger, CancellationToken cancellationToken) in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\CommandLine\CommonCompiler.cs:line 413
at Microsoft.Dynamics.Nav.CodeAnalysis.CommandLine.CommonCompiler.Run(CompilerTextWriter consoleOutput, CancellationToken cancellationToken) in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.CodeAnalysis\CommandLine\CommonCompiler.cs:line 312
at Microsoft.Dynamics.Nav.EditorServices.Protocol.LanguageServer.Extensions.CreatePackageRequestHandler.d__3.MoveNext() in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.EditorServices.Protocol\LanguageServer\Extensions\CreatePackageRequestHandler.cs:line 32
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Dynamics.Nav.EditorServices.Protocol.RequestRegistry.d__6.MoveNext() in C:\Agent_work\38\s\source\Prod\Microsoft.Dynamics.Nav.EditorServices.Protocol\Endpoints\RequestRegistry.cs:line 59
After progressively disabling parts of the report, I narrowed it down to this code:
                        column(EmployeeName;TempSalesInvoiceLine."Employee Name")
                        {
                            IncludeCaption = true;
                        }

After I removed the IncludeCaption property, the compile works.
The Table Extension does have a Caption however:
        field(50003;"Employee Name";Text[50])
        {
            Caption = 'Employee Name';
        }

My biggest issue was finding where this error originated. I had a small number of objects in my conversion, and even so, it took quite long to narrow down the error to what objects, and then what control what impacting the issue. It would be great if the compiler could at least indicate what object is involved in the error it finds.

Just had this issue in NAV2018CU2 with AL version 0.12.16641
Put all IncludeCaption = true; into comments and the package was created.

That's correct, the issue was only fixed for the latest version of the product (February Update of Developer PReview). For NAV 2018 please see: https://github.com/Microsoft/AL/wiki/Frequently-Asked-Questions#nav-2018-issues


Feature request: System Requirements

Is your feature request related to a problem? Please describe.
No, documentation related
Describe the solution you'd like
Are there any minimum/recommended hardware/software requirements? Or any suggestions regarding sizing?

Whilst some operations can be CPU intensive (e.g. calculating RAKE probabilities), the CyberChef application itself should be very light weight on both memory and CPU.  We have minimum browser requirements, but no minimum specs.


1 liner bug fix for game_recovery: name 'config' is not defined

as per the title

Guess I didnt review very well : )

humble :) but it was my bad really, I'm working with 2 IDEs and somehow I did this change on one and the rest of the changes on the other one so I missed it


Marking a scenario as failed in @afterScenario

Is it possible to mark a scenario as failed in an afterScenario function?

Nope.

@everzet Is there a way to do this somehow then?
Right now in the afterScenario we are checking if the called application had any errors, warnings or notices and if it had we throw an Exception. The problem is that though cli output contains this the test itself is not failed.

@emulgeator you can create step to check this

@spolischook But every step I have should check this at the end, as clearly no errors should be trigger between normal circumstances. That means if I have 300 scenarios I have to add 300 steps which is far from optimal

FWIW I have the same need. Adding 300 steps (see prior post) is a poor workaround.

I'm also trying to figure out how to catch any errors/warnings/notices that occur during a scenario and fail the scenario. I can't think of any reasonable way to do this other than via AfterScenario, which unfortunately doesn't actually fail the scenario. Steps doesn't feel like a reasonable workaround at all.

This is happening elsewhere for me as well, where another package is throwing a fatal (occasionally) during a behat build, and behat doesn't seem to recognize that the fatal should be treated as a failure.

I also have the case, would be nice to add something that could at least specify a result code from the AfterScenario hook to override the current scenario result code, no?

Is there any news about this issue please? It seems significant to me that an AfterScenario hook has the possibility to make fail the scenario.

We have a similar scenario, we use Behat for flow-testing where we mock external web services.
In a hook after each scenario we would want to assert that all expected requests have been made, and if not, fail the feature.
Having to specify an additional step in every scenario seems to be a very error-prone approach.
Also we write scenarios in more of a user story/non technical way, we would need to add an extra step just for technical purposes which goes against that.
I'm wondering how people solved issues like this.

This would totally break reported like the pretty reporter if you can change the outcome of a scenario after that scenario has ended, because it could have already been reported as successful (depending on the order of listeners between your mock and the reporter)

Luckily (maybe) I got a notification too :) I think things might have moved on since this was first filed, because hooks definitely now fail the build (e.g. make it exit with an exit code 1) - but as @stof says making them change the result of an actual scenario would be challenging due to the sequence of listeners.
As it stands, the output for both --pretty and --progress does show the hook failures, but it's not always easy to see what scenario they were from, and the summary at the end (e.g. "3 scenarios (3 passed)") doesn't tell you there were failing hooks.
If you want to open a new issue - or better yet, a pull request - to enhance the formatter output to make it easier to see when / where hooks have failed that could definitely be useful.
On your broader question, my opinion would be that:

if the assertions are actually part of the business requirements (e.g. the expected request is "and the destination bank should have received a transfer notification for £15.00") then they should be part of the scenario, even if that is repetitive across lots of scenarios.
if they are sanity checks for developers (e.g. "no errors were reported during the scenario") then actually they probably shouldn't fail the scenario itself - because all the business/user-defined requirements were satisfied. So an assertion in an AfterScenario hook might be valid, and it should be easy for developers to see which scenario they were related to.


feat: added support for spacecraft thrusters

🎉 New feature
Summary
Adds support for Spacecraft thrusters controlled with a duty cycle signal (such as PWM). Method ported from PX4 Gazebo-Classic SITL implemented by @Jaeyoung-Lim
Test it

Checklist

[X] Signed all commits for DCO
[X] Added tests
[X] Added example and/or tutorial
[ ] Updated documentation (as needed)
[ ] Updated migration guide (as needed)
[ ] Consider updating Python bindings (if the library has them)
[X] codecheck passed (See contributing)
[X] All tests passed (See test coverage)
[ ] While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

Thanks @ahcorde for the inputs! I had not yet started cleaning the branch as I had just finished the feature. I will address this in the coming days. Thanks!

@ahcorde should be good for a review check. Let me know what else I should improve on it!

Thanks for this really awesome contribution. I've marked out some things that need changing. It would be nice if you had an example SDF and some unit tests also. These are useful as they allow us to make sure we don't introduce regressions when we make changes to the core simulation platform.

Thanks @arjo129 ! Unfortunately right now I'm a bit swamped, so might take a bit more time before I add an example SDF and unit tests. Would this be a hard requirement for now, or is it ok if we can merge this in and then I add them?
On another note: I need this plugin to be available on gz-sim7 (since we are using gazebo garden for now). Should I do a PR targetting that branch too?

I would say unit tests can go in later but we should have an example. At the very least thatll allow me to test it.

@arjo129 I Just added a spacecraft world with the DART spacecraft (mass and inertia were severely reduced for faster visualization. Also updated the tutorial.
Moreover, I think that the mesh file should not be there but hosted on fuel, but I couldn't find my way to upload it there. Do I need extra permissions?

You do need to sign in to upload files to fuel. Tbh you can just use a box if you're strapped for time.

You do need to sign in to upload files to fuel. Tbh you can just use a box if you're strapped for time.

I'll try to do it now so that it's done properly at once. Standby,

I wont be able to test this till tomorrow. Ipl let you know how to fix it.

I wont be able to test this till tomorrow. Ill let you know how to fix it.

No worries, let me know and I'll fix it over the weekend.

Thanks @arjo129 , just fixed the above items!
Best,
Pedro

Also, to target gz garden, should I make a PR to that branch?

We try to avoid adding new features to older versions. If it really is needed we do do backports from time to time. Lets first merge it then it can be cherry picked later.

Sounds good @arjo129 !
Let me know if I should do anything else for this to be merged. Thanks!

Your DCO needs fixing also are you basing your work on gz-sim7 so you should target that branch else the abi checker wont be happy.

Your DCO needs fixing also are you basing your work on gz-sim7 so you should target that branch else the abi checker wont be happy.

Yes you are right, thanks for changing the target branch to gz-sim7! So later we cherry-pick this for main?

Yes. We will forward port later. First address the open issue a hand.
https://github.com/gazebosim/gz-sim/pull/2431/checks?check_run_id=26016983492

@Pedro-Roque you might want to look at this PR https://github.com/DISCOWER/gz-sim/pull/1 I submitted to your branch, it aligns more on the standard names/use of actuators, topics, and sub_topics. Note that the MulticopterMotorModel is getting refactored soon (thanks for reminder and catch on that @azeey) to make it camel_case and also follow all the other systems (so making it how other ones like Ackermann, Joint, JointPosition, etc are, is where this request is coming from).
I also cleaned up some repeated areas of the tutorial and made it more in line with the other existing ones.
In my opinion after making the suggested changes in the PR https://github.com/DISCOWER/gz-sim/pull/1 the only remaining items are:

Update the model revision number in examples/worlds/spacecraft.sdf after uploading the revised dart model to app.gazebosim with the model.sdf using the plugin with updated camel case.

Consider also uploading the 2D testbench model to app.gazebosim and linking that instead in the tutorial so it doesn't allow for any broken links.

Add a test to keep system from possibly breaking (can follow for instance follow how these tests are done)

@bperseghetti @azeey I've now updated everything with respect to the provided comments. Only thing that I'll have to do later is the unit tests. I'd be happy if this could be merged meanwhile, since I have another PR depending on this.

@arjo129 are we ready to merge this in?

Hey @Pedro-Roque  I would suggest still adding a basic test to get some level of code coverage started. Also I'm assuming your dependent PR is the one in PX4, I can help merge that PX4 PR in for you but first a release of garden would need to be scheduled after this current PR gets in and that can take some time since it's more than just getting this PR merged.
With that being said I would suggest the fastest way to get this current PR merged in and start the "release timeline clock" is to create that super basic test based off what I shared for Ackermann (or really any other test in tree as well) with the goal of testing the functionality and also as many lines of code of this PR as possible (for code coverage).

@bperseghetti I created a test and wanted to run it on my machine. However after following the instructions regarding code coverage, colcon only finds 7 tests in the whole workspace and no tests are found on the gz-sim7 package.
Is there something obvious I'm missing? I'll share here a log of the commands

How are you building your workspace? Generally I would run colcon build. Some times you may need to force cmake to reconfigure itself. Running colcon test will take very long. I suggest going inside the build/gz-sim7/bin directory. You should see the integration test binaries. You can run your test individually from there.

@bperseghetti I created a test and wanted to run it on my machine. However after following the instructions regarding code coverage, colcon only finds 7 tests in the whole workspace and no tests are found on the gz-sim7 package.
Is there something obvious I'm missing? I'll share here a log of the commands

Want to just push up to this PR your test that you wrote and we can look at/suggest changes from there?

How are you building your workspace? Generally I would run colcon build. Some times you may need to force cmake to reconfigure itself. Running colcon test will take very long. I suggest going inside the build/gz-sim7/bin directory. You should see the integration test binaries. You can run your test individually from there.

This is what is populated in the folder:
╭─  ~/gz_garden_ws/build/gz-sim7/bin ···························  07:38:29
╰─❯ pwd
/home/roque/gz_garden_ws/build/gz-sim7/bin

╭─  ~/gz_garden_ws/build/gz-sim7/bin ···························  07:38:30
╰─❯ ls
runGui

@bperseghetti I'd like to have a working test before I push it, but I'd also like to try it out locally first.

Seems like you are not building tests. Please use:
colcon build --merge-install

NOT:
colcon build --cmake-args -DBUILD_TESTING=OFF --merge-install

Seems like you are not building tests. Please use:
colcon build --merge-install

NOT:
colcon build --cmake-args -DBUILD_TESTING=OFF --merge-install

Consider deleteing build and install folders in your workspace for clean builds.

Hi @arjo129 the command I was running is
colcon build --cmake-args -DCMAKE_BUILD_TYPE=Coverage --merge-install --packages-select gz-sim7

I'll do a clean build and come back to you.

Remove the coverage do a vanilla build.

Remove the coverage do a vanilla build.

Ok, the vanilla build seems to be building the tests. Maybe this is something we should update the docs with. I'll fix the test and push changes later in the week.

@arjo129 @bperseghetti I've added a unit test now that checks for motion of the model upon setting a non-zero input. The test works as expected.
Let me know if there is anything else needed before merging this in.

Yeah just revert and ignore windows for now. I'm not sure about it.
On Thu, Jul 18, 2024, 9:03 PM Pedro Roque @.***> wrote:

@.**** commented on this pull request.
In test/integration/spacecraft.cc
https://github.com/gazebosim/gz-sim/pull/2431#discussion_r1682800396:

serverConfig.SetSdfFile(sdfFile);

auto server = std::make_unique(serverConfig);
EXPECT_FALSE(server->Running());
EXPECT_FALSE(*server->Running(0));

using namespace std::chrono_literals;
server->SetUpdatePeriod(1ns);
return server;
}
+};

+/////////////////////////////////////////////////
+// Test that commanded motor speed is applied
+// See https://github.com/gazebosim/gz-sim/issues/1175
+TEST_F(SpacecraftTest, GZ_UTILS_TEST_DISABLED_ON_WIN32(InputTest))
@arjo129 https://github.com/arjo129 test fails on Windows - seems like
the vehicle did not move at all there. Thoughts?
—
Reply to this email directly, view it on GitHub
https://github.com/gazebosim/gz-sim/pull/2431#discussion_r1682800396,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEEMQC3JXL4SF55RMVDFGLZM64LDAVCNFSM6AAAAABIXKU6BOVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDCOBVG42TOMZQGQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

Done. Will wait for @ahcorde and @azeey for inputs

I was trying to bring up and agent and I think gz_sim-ci-pr_any-focal-amd64  failure was my fault, sorry
@osrf-jenkins retest this please

Can you merge with gz-sim7 to make the ABI checker happy

@arjo129 just merged it in, thanks for noting that!

@Pedro-Roque Thanks for your contribution, note this will not be in the binaries till a release is cut and run (including all other needed backports/forwardports that the team thinks is needed to allow for a release. I will see about the timing of that this coming Monday.

Thanks @bperseghetti - how can I get that info on Monday? Reach out via Discord?

Thanks @bperseghetti - how can I get that info on Monday? Reach out via Discord?

I'll ping you on here.

@bperseghetti just pinging for further info. A release with these additions would strongly simplify our setup

@bperseghetti just pinging for further info. A release with these additions would strongly simplify our setup

You can follow it here: https://github.com/gazebosim/gz-sim/pull/2491

@bperseghetti just checking if there's any reason why the plugin was not propagated to newer gazebo versions - shall I do a PR for them? It is missing in harmonic, at least.

forward ported to harmonic in https://github.com/gazebosim/gz-sim/pull/2504. I think there just needs to be a new gz-sim8 release.

forward ported to harmonic in #2504. I think there just needs to be a new gz-sim8 release.

@Pedro-Roque I can open a release request PR this evening.

Thanks @bperseghetti and @iche033 ! When are the binaries going to be released for Ubuntu 24?

Thanks @bperseghetti and @iche033 ! When are the binaries going to be released for Ubuntu 24?

Should be covered by this: https://github.com/gazebosim/gz-sim/pull/2671


feature request: Long list style

Did you check existing requests?

[X] I have searched the existing issues

Describe the feature
List files with timestamp and size, similar to g:netrw_liststyle = 1 option in netrw.
Provide background
No response
Additional details
No response

  columns = {
    "icon",
    -- "permissions",
    -- "size",
    -- "mtime",
  },


Nullable field or schema not validating for None

Hi,
I've just upgrade openapi-core on my project to 0.13.3, after some months without upgrading, and now nullable schemas are not validating None values anymore.
I found a commit which seems to fix the issue: https://github.com/p1c2u/openapi-core/commit/7307794656106323b1f46cebd7525fcb265f5f28 - but this seems to have been fixed after the latest tag.
Can you release a new patch version, with the most recent fixes, please? I use nullable schemas quite a lot, and need that fix...
Thanks!

Actually, just since I already created this ticket, there's a very similar issue still going on in the latest version from master: nullable arrays don't work, if I pass None it just fails instead of accepting the value. The reason is, the ArrayUnmarshaller tries to list(map(self.items_unmarshaller, value)) when the value is None, hence raising a NoneType error.

Reproducing the same issue.
Unfortunately, this project going to be dead soon (I would say), it seems that author has no time at all, which is fully understandable, but if there is no release for longer time for some already fixed issues, not sure is it worth to contribute here at all (I usually fix issue locally, but that's not it).
If I got time, I will try to create PR for this.

I wouldn't say the project is "going to be dead soon"; It's just that the author, @p1c2u , hasn't been much active on it, but recently I got some replies on tickets here. Maybe it's worth considering adding more people as direct collaborators of the project.

Maybe it's worth considering adding more people as direct collaborators of the project.

That's the key! Without that, there is no too much sense in providing fixes without possibility to release, even potential smaller issues are better than issue with paths resolving. In any case, when (i.e. if) that happens, I'm here to help with PRs.

Looks like a new tag was cut in July, so this issue is moot? https://github.com/p1c2u/openapi-core/commit/0aeb14477bd881247856a8ab520b5adf4223573d

I do not think that fix for this issue is included in the last release, but you can check.

To correct myself (from previous comment) - handling nullable is partially fixed in the latest release, i.e. 0.13.4, but still having two issues:

When nullable is set for arrays (i.e. type: array & nullable: true), null value not being properly validated, instead, it raising error: 'NoneType' object is not iterable. So same issue that diogobaeder reported in the second post.
When nullable is set for date-time (i.e. type: string, format: date-time & nullable: true), null value sometimes not being properly validated, instead, it raising error: Parser must be a string or character stream, not NoneType. So, validating null value for date-times working fine in some cases, but in others raising mentioned error. Will report here if I figure out what leading to such behavior.

@stojan-jovic I address the first case in #276.
What do we think ?

@stojan-jovic I address the first case in #276.
What do we think ?

any news on this issue?

This is still an issue:
      "message": "None for not nullable"

Still an issue. I just updated openapi-core in my project and all of a sudden I started getting that error for a schema model that is nullable. This is the schema, with the description simplified:
{'description': 'something something.',
     'format': 'uuid',
     'nullable': True,
     'type': 'string'}

This leads me to:
None is not of type 'string'

Almost same error for:
    {'additionalProperties': True,
     'description': 'something something.',
     'nullable': True,
     'type': 'object'}

and almost the same for:
{'format': 'date-time', 'nullable': True, 'type': 'string'}

I see this as a pretty big issue on the current version.


17 add search and indexing

Please always provide the GitHub issue(s) your PR is for, as well as test URLs where your change can be observed (before and after):
Added /best-cigars-guide/search.docx as search results page
Updated search query param from 's' to 'q'
Set up custom indexing. New pages are indexed as they are published in /best-cigars/guide/index/query-index.xlsx and category-index.xlsx
The custom index was added to handle the different structure of category list pages.
You should be able to use the search bar in the header and find available posts. For example search for 'best'.
Fix #17
Test URLs:

Before: https://main--best-cigars-guide--famous-smoke.hlx.live/best-cigars-guide
After: https://17-add-search-and-indexing--best-cigars-guide--famous-smoke.hlx.live/best-cigars-guide

I was able to complete a search and get results.
I was able to add to the index by previewing a file on the branch for this ticket.
https://17-add-search-and-indexing--best-cigars-guide--famous-smoke.hlx.page/query-index.json
The js/css comes from an aem block template and will be customized and styled on another ticket
aem-block-collection/blocks/search at main · adobe/aem-block-collection


Add project description validation using the validateString helper

Relies on #64.

Codecov Report

Merging #70 into feature/validateString-helper will not change coverage.
The diff coverage is n/a.

@@                     Coverage Diff                      @@
##           feature/validateString-helper    #70   +/-   ##
============================================================
  Coverage                            100%   100%
============================================================
  Files                                  7      7
  Lines                                 39     39
  Branches                              10     10
============================================================
  Hits                                  39     39

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9b38ec9...6abbe9a. Read the comment docs.


not in gzip format when unpacking file output from curl

curl -u testuser:welcome1 -k -i -H "Content-type: application/json" -o /tmp/mardl.tar.gz -X GET https://adc.com:7002/apis/getResource/jdk
The above fetches me a .tar.gz file & file gets stored & the size of the file seems correct.
But when I run tar -xvzf on the file, it says not in gzip format.
Please help.

Sorry, what is this link referring to?  I am lost.

That's the REST api which returns the tar.gz file
It is not a link.

But how is this URL related to bedtools?: https://adc.com/iaps/getResource/jdk
Sorry if I am being dim.

Sorry, it is a local API. Since there was a similar issue reported here, I thought someone might help with the curl command. Sorry if this is not the right forum for this.


Depth camera bit precision

Documentation questions:
I see from the documentation
that depth values are 16 bit integers, units in mm.
From the hardware specs operating ranges vary based on the depth camera mode.

What's the depth bit precision? Is it linear throughout the range, or do closer values have higher precision? Is it the same or different for the different camera modes?
The footnote on the hardware specs page notes that values may be possible outside the range. Which means the maximum and minimum bit values are presumably outside the stated range? Is zero the minimum possible depth reading?
Is it possible to adjust the precision? I'm working on a close range robotics application, and would be happy to trade away maximum range in exchange for greater precision up close.

Thanks!

The temporal standard deviation of the range measurements is not constant over range. It is a function of

Distance (closer is better, up until sensor saturation)
Reflectivity (higher is better)
Angle of incidence (lower angle is better)
Location in the FOV
Ambient light levels (lower spectral content ~850 nm is better)
Field of illumination (narrower is better)

Yes. It is impossible to get a depth reading corresponding to the maximum value of the 16 bit integer. For most applications the minimum distance is limited by saturation and the given specs are merely a guide to minimum performance parameters, not the actual device performance (see the above list).
The ability to freely change the precision and/or dynamic range of the system is not exposed to the user, because it completely invalidates the calibration parameters. The only available precision trade-offs are by changing between the different binning and narrow/wide configurations. The wide configuration will have worse temporal standard deviation, but will be able to range closer to a given high reflectivity object with valid depth. Binning should have only limited impact on minimum imaging distance.

Thank you


On Board Screen Logic

When the user has downloaded the app for the first time add an option to display the onboarding screen otherwise skip that screen direct to the home screen.

please assign me this issue

Assigned for 3 days.

Kindly assign me this if its still open

Sorry @dhivya17 . This issue was resolved


Reponse to WriteSingleCoil is malformed

The response to WriteSingleCoil is not correct.
I was testing with the Node.js module 'modbus-serial' as a TCP client and this crate as the Modbus TCP server, when I got an error on writeCoil (aka WriteSingleCoil).
I then traced the packets using WireShark and it confirmes that Data is missing from the Response.
Modbus
    .000 0101 = Function Code: Write Single Coil (5)
    [Request Frame: 2612]
    [Time from request: 0.000282788 seconds]
    Reference Number: 20
    Data: <MISSING>
[Malformed Packet: Modbus]
    [Expert Info (Error/Malformed): Malformed Packet (Exception occurred)]
        [Malformed Packet (Exception occurred)]
        [Severity level: Error]
        [Group: Malformed]

I am not sure on the validity of this page https://www.modbustools.com/modbus.html#function05, but it specifies that the response must be an echo of the request.
I have clone the project and will try to fix it. I would like here, however, if you agreed with my findings or I'm wrong.
Regards,
Thomas

I think I have sorted out now. I am not able to test it, however.
I have checked the master branch out, made the changes and it builds.
When I try to reference to it from my own project it states:
error[E0433]: failed to resolve: could not find `Server` in `tcp`
  --> src/main.rs:22:14
   |
22 |         tcp::Server::new(socket_addr).serve(|| Ok(modbus::MbServer));
   |              ^^^^^^ could not find `Server` in `tcp`

This is the dependency in Cargo.toml:
tokio-modbus = { version = "0.3.5", default-features = false, features = ["tcp"] }

This is the path override just below:
[patch.crates-io]
tokio-modbus = { path = "../tokio-modbus" }

I would like to test before making a pull request. Any help is appreciated, thanks.
/Thomas

@thosaa thanks for reporting this bug and fixing it.

I cannot see what has changed since tag v0.3.5 to now, since I cannot run my code with the newest version:
I have now set my dep to:
tokio-modbus = { version = "0.3.5", default-features = false, features = ["tcp", "server"] }

and the same patch.crates-io as before.
I now get this error:
23 |         tcp::Server::new(socket_addr).serve(|| Ok(modbus::MbServer));
   |                                       ^^^^^ the trait `tokio_modbus::server::service::Service` is not implemented for `modbus::MbServer`

And this is the way I call it:
    let _server = thread::spawn(move || {
        tcp::Server::new(socket_addr).serve(|| Ok(modbus::MbServer));
    });

bug fixed with 8284940bae40ead5410f0abbe9942f0be9db2239

@thosaa I'll have a look at your questions at the weekend.


How to manually update MC after using this script?

The built-in server updater doesn't work after installing using this script (it kills the install). meshcentral-data folders etc aren't installed in the "correct" location (user home folder) so simply running npm install meshcentral doesn't work from user prompt.
How to manually update the server after installing with this script?

I know the update on mesh itself is a little ropey right now but can check it, it should be fine where it's installed and is the "correct" location as far as linux is concerned (maybe not NPM 😜)

The install location is /opt/meshcentral/ folder. I tried running npm install meshcentral in that folder and it killed the install (again!)
Currently my server is sitting on 1.1.9 with the latest 1.1.11 and unable to update :(

no worries, Ill take a look at it

Any update on this @dinger1986 ?

no, I dont use mesh anymore so need to spin up a server etc to look at it and figure out whats wrong, its on my list

Update yet on this issue?

No, I might just archive this, long list not got loads of time and don't need getting chased as again I don't use meshcentral


Bug: Automatic Mode Made False Claims Of Perf Improvements In This Case

What version of million are you using?
2.6.4
Are you using an SSR adapter? If so, which one?
None
What package manager are you using?
pnpm
What operating system are you using?
Mac
What browser are you using?
Chrome
Describe the Bug
A user complained about not getting any differences in lighthouse perf scores, even when automatic mode in the terminal told them that certain components in their application had performance improvements:
"I inherited React app done with create-react-app. I added craco and setup million on it.
The code base is really awful. So, quick win? Hypothesis: Add Million and get a 10% boost?
I'm the lead for FE here, so my job is to select our next tech stack. I'd prefer to stay on React (Next.js) and add Million to boost it to Svelte speed numbers. Thus, I was using our current site as a guinea pig to prove we should use React + Million.
I could share more, but not publicly." - User
Before:

After:

Even when Million at build time said this:

What's the expected result?
The expected results is to know that the results Million provides in the terminal tallies to a huge extent as to results testing softwares provides and stuff
Link to Minimal Reproducible Example
https://media.discordapp.net/attachments/938129049539186761/1192906374355619840/image.png?ex=65aac7a2&is=659852a2&hm=68cf439de606a17e6a60287856fa3ac94c369708f0aef077e7d553712f07ecf1&=&format=webp&quality=lossless&width=1062&height=150
Participation

[ ] I am willing to submit a pull request for this issue.

Can we take a look at this @aidenybai, it's kind of important.


Release sdk/resourcemanager/customerlockbox/armcustomerlockbox v0.1.0

[ ] The purpose of this PR is explained in this or a referenced issue.
[ ] The PR does not update generated files.

These files are managed by the codegen framework at Azure/autorest.go.

[ ] Tests are included and/or updated for code changes.
[ ] Updates to CHANGELOG.md are included.
[ ] MIT license headers are included in each file.

/azp run prepare-pipelines

/azp run go - armcustomerlockbox - ci

/check-enforcer override


Wasted few hours trying to import it propery.

Dude, this library wasted my few hours bcz even after importing the library poperly using jCenter, bintray, jitpack.
I Was unable to import the classes in the activity. Plz check.

Hey Rahul, sorry to hear it's not working for you. Does the sample app compile for you? That uses the jcenter version of the library rather than the local so if that works, it should work in your project as well.
I'm currently traveling so I won't be able to investigate for a few days but feel free to leave more info that you may find in the meantime

Hey dear, thanks a lot for such a quick response. No I haven't complied the sample app. Will try to do that and let you know. I would just recommend to make the library easily importable using jitpack.  Happy journey.

Hi, I wanted to check if you had a chance to try the sample project?
I can consider also publishing through jitpack as well though no urgent plans to do so yet.

I've updated the README to specify the requirement of adding the jcenter() repository. Hopefully you've been able to get this sorted out in the meantime.
I'm closing the issue since the importing from jcenter appears to be working


Vncserver not starting

$ vncstart                           DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-go4ptt5iXX,guid=88e71b2eb0caa0c7928b92ad6184fbf4               DBUS_SESSION_BUS_PID=21152
/usr/local/bin/vncstart: line 3: vncserver: command not found

@BDhackers009

show output of
cat $(which vncstart)
On Fri, Nov 5, 2021, 15:51 Meharaj007 @.***> wrote:

[image: Screenshot_20211105-151013796]
https://user-images.githubusercontent.com/93766839/140491875-d5cab833-e5d1-45df-a273-711eef157efd.jpg
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/modded-ubuntu/modded-ubuntu/issues/40#issuecomment-961758760,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQAS3WY5FQC74MUBXHNOJJLUKOSJFANCNFSM5HNN5OKQ
.

 Its vncserver-start or vncserver-stop

Ar you guys fool ot what its not vncstart. Its vncserver-start or vncserver-stop

vncserver-start && vncserver-stop aren't any command they are just alias for running vncserver. The developers of andronix && anlinux created vncserver-start && vncserver-stop so that the users can use run && stop vncserver easily. In the same way we made vncstart && vncstop so that the users can easily run && stop vncserver. Got it sir?


Could not find the JSON path of a referenced schema: Manually referenced schemas must be added to the 'Definitions' of a parent schema

I'm getting this error in our project and I have no idea how do debug it and there is absolutely no information at all as to what is actually the issue. Right before this I get the following info, which is why I'm reporting it here and not in the nswag repository:
No action descriptors found. This may indicate an incorrectly configured application or missing application parts.

This would seem to indicate that there is no openapi definitions loaded, but the endpoints all work in fast-endpoints as you'd expect.
What does all of this mean with the swagger (other than it doesn't work) and how do I go about diagnosing it?
I can't find anything other than 2 open issues in nswag and nsjonschema with no information on how to find this and address it.
Thanks!
PS: I might be doing something wrong because this is confusing me in the docs:
Program.cs
Copy code
using FastEndpoints;
using FastEndpoints.Swagger; //add this

var bld = WebApplication.CreateBuilder();
bld.Services
   .AddFastEndpoints()
   .SwaggerDocument(); //define a swagger document

var app = bld.Build();
app.UseFastEndpoints()
   .UseSwaggerGen(); //add this
app.Run();
You can then visit /swagger or /swagger/v1/swagger.json to see the generated Swagger document.

Configuration
Swagger generation/document settings can be configured by providing an action to SwaggerDocument():
Program.cs
bld.Services.SwaggerDocument(o =>
{
    o.DocumentSettings = s =>
    {
        s.Title = "My API";
        s.Version = "v1";
    };
});

Note that in the first section it references AddFastEndpoints().AddSwaggerDocument() and then in the second builder.Services.SwaggerDocument. If you do both, I get an error telling me that there's a duplicate. And some of the options to do with RemoveEmptyRequestSchema etc. (which I've tried not touching) don't show up in the AddFastEndpoints().AddSwaggerDocument() so it appears I need to use the later?

No action descriptors found. This may indicate an incorrectly configured application or missing application parts.

that you can safely ignore or suppress the log message as it's output by nswag because they're not aware of FE.
as for the other issue, i tried to replicate it but did not succeed.
could you possibly attach a small repro project so i can have a look?
btw, AddSwaggerDocument() is not something from FE. only .Services.SwaggerDocument(...) should be used.

No action descriptors found. This may indicate an incorrectly configured application or missing application parts.

that you can safely ignore or suppress the log message as it's output by nswag because they're not aware of FE.
OK.

as for the other issue, i tried to replicate it but did not succeed. could you possibly attach a small repro project so i can have a look?

The problem is that I have no way to start making an Repo because nswag is giving me no information at all to work from.  I can tell you that Swashbuckle works fine without error.

btw, AddSwaggerDocument() is not something from FE. only .Services.SwaggerDocument(...) should be used.

Ok, well the FE docs reference it. Might want to fix.
Also of note, on app.ExportSwaggerJsonAndExitAsync it would be nice to supply the file name. I.e. we like to put ours in /help/v1/openapi.json but the document name is v1 so this right now forces /help.v1.json instead.

on app.ExportSwaggerJsonAndExitAsync it would be nice to supply the file name.

v<IP_ADDRESS>-beta can do this. thanks for the suggestion.

No action descriptors found. This may indicate an incorrectly configured application or missing application parts.

that you can safely ignore or suppress the log message as it's output by nswag because they're not aware of FE.
as for the other issue, i tried to replicate it but did not succeed. could you possibly attach a small repro project so i can have a look?
btw, AddSwaggerDocument() is not something from FE. only .Services.SwaggerDocument(...) should be used.

https://fast-endpoints.com/docs/swagger-support
The first section and then second section. It may be the same thing, but it's very confusing.
I still can't figure out any way to identify why nswag is having an issue. If you have any insight, I'd greatly appreciate it.

here's a freshly scaffolded project with a working swagger/nswag setup. run it and check.
SwaggerRepro.zip
if you can reproduce the issue there with your own endpoint(s), send back the updated project and i'll look in to it.

I found the issue. NSwag doesn't yet support Generics and throws without telling you what it's throwing on when it encounters a generic. (OpenApi 3.1 supports generics)


Convert users from per-user MFA to Conditional Access based MFA

I'm finding that though this script info is helpful, it is not really clear what it does.  I understand the first two commented commands to disable MFA or enable MFA for all.  -KeepMethods seems to be a new switch which is awesome.   Where I am lost is:
Wrapper to disable MFA with the option to keep the MFA
methods (to avoid having to proof-up again later)
The wrapper first comment is odd as one of the previous commands does that.  The methods comment is unclear because I have no idea what "proof-up" means.
I'm also completely unsure how I would run this.  I understand the short disable commands, but the function...will that run against all users?  Why would I choose that vs one of the shorter commands?  Usually Microsoft provides some examples with the commands. With the function, I would definitely needs some clarity.
Lastly, I believe no matter which commands are run...it appears application passwords are wiped out.  We have a handful of folks that use these to authenticate with apps that don't support MFA.  Is that expected behavior?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f132318f-8670-fd0d-2524-01305ccff2a1
Version Independent ID: 0849b4a9-ff14-1d61-47f2-a2daecf3cde9
Content: Plan and execute an Azure Multi-Factor Authentication deployment - Azure Active Directory
Content Source: articles/active-directory/authentication/howto-mfa-getstarted.md
Service: active-directory
Sub-service: authentication
GitHub Login: @MicrosoftGuyJFlo
Microsoft Alias: joflore

@xcactusx Thank you for your feeedback . We will investigate and update the thread further.

@xcactusx It should be a single comment "Wrapper to disable the MFA with the option to keep MFA methods (to avoid having to proof-up again later)". I have updated the article accordingly.
The function is Disable-MFA is described under this comment. Proof-up basically means having to register for MFA again. This includes registering your phone or mobile app etc.
Both the commands and the function need to be run together in your script to get this to work. The shorter commands do not work directly. The shorter commands are in fact calling the function.
If you run the command "Get-Msoluser -All " then this will run against all users. If you want to run it against a subset of users, you would have to filter them accordingly.
Conditional access is applicable to modern authentication supported clients. So, if you have users who are using legacy flow (App passwords), conditional access will not work for them. You will have to use per-user MFA for that set of users.

Thank you, your response was incredibly helpful...even giving me a way to test on a subset of users vs running universally the first time.

@xcactusx I am glad it helped you. We will now proceed to close this thread. If you have further questions, please tag me in the comments and I will gladly continue the conversation.


Button accessibility?

I saw that primer is requiring to add type attributes to all <button> tags.
My question is:
Does adding type attributes to <button> tags improve accessibility?

If so, how does it improve accessibiilty?
If not, are there any other reasons for using the type attribute other than distinguishing between, "button", "submit" and "reset" behaviors?

Tried googling around to answer my question but haven't been able to find any answers.
So far, I could only find documentation regarding usage of role="button" for elements made clickable, like <a> tags.

It's not about accessibility, it's about specific usage. type will default to submit (within forms at least) and that changes behavior. Adding an explicit type will ensure folks don't run into accidental form submissions.

Thanks @mdo


Segfault when copying directories

when copying directories, I receive a segfault.
mkdir a
mkdir b
wcp a b

[1]    7072 segmentation fault (core dumped)  wcp a b               0.00 B / ???                                     8388608.00 TiB/s   ETA: ???

I understand this program doesn't support directories? It would be nice to give an error instead of segfaulting

No, it does definitely support directories, the issue is probably a bug when the source directory is empty. Will look into it, thanks for the report!
On Mon, Mar 8, 2021, at 1:04 PM, Vítor Luís dos Santos Trindade wrote:

when copying directories, I receive a segfault.
mkdir a
mkdir b
wcp a b
[1]    7072 segmentation fault (core dumped)  wcp a b
0.00 B / ???                                     8388608.00 TiB/s
ETA: ???
I understand this program doesn't support directories? It would be nice
to give an error instead of segfaulting
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/wheybags/wcp/issues/4, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAIK5GVXVBZENGTW7WPOEKTTCTDN7ANCNFSM4YZLWYOA.

Should be fixed by https://github.com/wheybags/wcp/commit/fbc63458fc3a9f7dd122429b97072b3863624bde
Let me know if you have any more problems!


"Manage dependencies" should be a separate chapter

<EMAIL_ADDRESS>Currently this chapter is very short and not in-depth, whilst the subject at hand is huge.
We should have a separate chapter for how to manage dependencies, including:

examples of existing community tools
anti-patterns
how it's different than Node dep management
how it's similar to Node dep management

We should mention udd.
Also there is an automated bot https://github.com/hasundue/denopendabot


Add nested-span hack to work around display: inherit;

Sorry @kentonv for horrible CSS here.
Rationale:

show-above-phone contains display: inherit;
which resulted in the SPAN inheriting display: block;

I'm going to self-merge.

@kentonv I request your deployment of this!


Port WebEncoders to use new Span APIs

https://github.com/dotnet/corefx/tree/master/src/System.Text.Encodings.Web/src/System/Text/Encodings/Web
@rynowak showed interest in doing this.
@KrzysztofCwalina do you have any thoughts on this?

We should once Span is "stable"/productized
For now, we should just write extensions that consume spans.

cc @GrabYourPitchforks

@KrzysztofCwalina, @muratg, @joshfree  - do we need this for 2.1?

@ahsonkhan this is more @rynowak and @Eilon territory... folks?

I assume no APIs would change (except maybe adding new ones), but there are no specific requirements from ASP.NET Core 2.1 as far as I know. However, if new more-efficient APIs are available, we could consider using them opportunistically.

Re-pathing to future since there's no specific request.

Tracking this in https://github.com/dotnet/corefx/issues/27253 since that's the correct repo.


concat can accept RDMs as separate arguments

Addresses Issue  #170: concat can accept either a list (or other iterable) of RDMss, or can accept individual RDMss as arguments.

Codecov Report

Merging #172 (a160cf9) into main (56b90ff) will increase coverage by 0.08%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #172      +/-   ##
==========================================
+ Coverage   76.21%   76.29%   +0.08%
==========================================
  Files          40       40
  Lines        3906     3911       +5
==========================================
+ Hits         2977     2984       +7
+ Misses        929      927       -2

Impacted Files
Coverage Δ

pyrsa/rdm/rdms.py
81.36% <100.00%> (+0.43%)
:arrow_up:

pyrsa/vis/model_plot.py
60.00% <0.00%> (+0.40%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 56b90ff...a160cf9. Read the comment docs.

Codecov Report

Merging #172 (a160cf9) into main (56b90ff) will increase coverage by 0.08%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #172      +/-   ##
==========================================
+ Coverage   76.21%   76.29%   +0.08%
==========================================
  Files          40       40
  Lines        3906     3911       +5
==========================================
+ Hits         2977     2984       +7
+ Misses        929      927       -2

Impacted Files
Coverage Δ

pyrsa/rdm/rdms.py
81.36% <100.00%> (+0.43%)
:arrow_up:

pyrsa/vis/model_plot.py
60.00% <0.00%> (+0.40%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 56b90ff...a160cf9. Read the comment docs.

Oh, I just tried this out. There seems to be a problem with the tests at least. Due to the reference character of the concat we get running concat twice does something interesting here. It seems the output of the two concats you compare in the test in the end actually link to the same object which then has more than 16 entries.
I noticed this, because == should not work properly for RDMs objects so far
A possible solution would be to do a copy of the first RDM at the beginning before appending RDMs.

Ok, I'll have a look at this.  If == is doing something unexpected I'll open a separate issue to address that.

Oh, I just tried this out. There seems to be a problem with the tests at least. Due to the reference character of the concat we get running concat twice does something interesting here. It seems the output of the two concats you compare in the test in the end actually link to the same object which then has more than 16 entries.
I noticed this, because == should not work properly for RDMs objects so far
A possible solution would be to do a copy of the first RDM at the beginning before appending RDMs.

Ok, I'll have a look at this.  If == is doing something unexpected I'll open a separate issue to address that.

It does not do anything unexpected, but it checks that the two objects are exactly the same, i.e. here the output creates the exact same object

It does not do anything unexpected, but it checks that the two objects are exactly the same, i.e. here the output creates the exact same object


Find the causality between sentiment index and bitcoin market returns

Hello Dang,
First of all, a great project and idea proposed have a lot of scopes. In search of that, I would like to get an acceptable correlation between bitcoin market return and sentiment index
To be more precise, we have found the relation of sentiment index on a composite market return instead of that I need with bitcoin.
okay, so  this is how I want to approach it

GAther the crix data only for bitcoin
follow the procedures with the crix_bitcoin data

I m not able to under the n in the above equation.
PLease provide a suggestion on my idea and above question
Thanks in advance

okay, I believe they are the number of coins taking part in the calculation.

Hey mate,
In case you want to find causality between the sentiment index and Bitcoin returns, you can simply use Bitcoin historical data for the dependent variable. For the sentiment index, if your purpose is solely Bitcoin, I suggest you only pull social messages only related to Bitcoin (from StockTwits or Reddit) to create the index. I also suggest you use more complicated models than random forests (e.g. XGBoost, RNNs, etc.) to find the patterns (I was forced to use RF as it's my university project.)
On part of the CRIX, I'm not sure what you mean here but CRIX is just a market-cap-weighted index for the entire crypto market (because I wanted to do my analysis on the market scale). Put it simply, CRIX is like S&P500 for the US stock market or DAX30 for the German one. This index is created by a German professor named Härdle, you can read more about it here: https://thecrix.de/.
Hope this helps, cheers!

Hey thanks,
Yes, I agree with the first point on getting bitcoin historical data and trying complex algorithms. But what would you say, will it be a better idea to use the market return based on the bitcoin market capitalization instead of CRIX(saying, it's an index for the entire market).
My final aim is to predict signal for the bitcoin trading.

Hello again!
I have few questions ( for a sanity check)!

How to find the Bitcoin market return series?
i was using this equation RM_t = Bitcoin_MarketCap_t - Bitcoin_MarketCap_t-1 / Bitcoin_MarketCap_t . Is this right?

What did you mean by models that require stationarity?
---> Suppose if I want to use a Random forest regressor to fit a model on the basis of sentiment data features and Bitcoin prices as the dependent variable, Do I want to do some preprocessing on the data? Since the features and bitcoin prices are non-stationary right?

What evaluation metrics can be used in the above case?
The MSE is enough?

Thanks in advance!

Hi Zack,
For your questions:

BTC returns should be calculated using Price, not Market Cap (and it's only Returns, not Market Returns). So:
R_t = ( BTC_Price_t - BTC_Price_(t-1) ) / BTC_Price_(t-1)

Regarding stationarity, to my knowledge, only some old-school time-series models, say ARIMA or VAR, requires this. So with random forests, it doesn't matter if you do preprocessing or not.

On evaluation metrics, I suppose that it depends on your needs. For example, to me, when I build a model to predict returns, my ultimate goal is to generate a trading strategy that outperforms simple holding. Therefore, in this case, I choose the evaluation metric as the returns of the trading strategy that use signals generated by my model predictions.
Tbh, MSE would do fine as well since generally speaking, when your model predictions are closer to the real values, your strategy returns are better at the same time.

Hope this helps.

Thanks mate,
That a lot of information I could have asked for. I will keep you in the loop with my advancements.
Best

Good luck with your work!


fix attempt

Merged in https://github.com/userver-framework/userver/commit/e8a7560b405713366ead73e5c25b735ce76b17c8


Fixed a bug with wrong rows being edited in reference author fields

closes #392

Using this same branch for more tasks since the bug fixes are pretty central and I don't want to mess with merging
closes #265
closes #394


MeshLine.refine

While working on the point source #632, trouble was encountered with MeshLine().refined() and this was traced to element_finder:
>>> m = MeshLine().refined()
>>> m.element_finder()(np.array([.7]))
array([0])
>>> MeshLine(m.p[0]).element_finder()(np.array([.7]))
array([1])

I'm not sure which of these is right.  I mean the first is wrong since 0.7 is definitely in the second element, but I'm not sure about the second.  Are the points passed to the constructed assumed increasing?

There is Mesh.sort_t flag which defaults to False. MeshTri1 has sort_t=True which is explained in the docstring of Mesh.

But given m = MeshLine().refined(), should m be valid?  It has
 >>> m.p
array([[0. , 1. , 0.5]])
>>> m.t
array([[0, 2],
       [2, 1]])

which looks O. K., not sorted, but O. K., but then m.element_finder() is wrong.
>>> m.element_finder()(np.array([.2]))
array([1])
>>> m.element_finder()(np.array([.7]))
array([0])

So element_finder seems to do argsort so that's not the issue.

I think there is some sort of assumption inside finder regarding the order of self.t.

It does assume that the start point of interval is self.t[0].

There is Mesh.sort_t flag which defaults to False. MeshTri1 has sort_t=True which is explained in the docstring of Mesh.

But given m = MeshLine().refined(), should m be valid? It has
 >>> m.p
array([[0. , 1. , 0.5]])
>>> m.t
array([[0, 2],
       [2, 1]])

which looks O. K., not sorted, but O. K., but then m.element_finder() is wrong.
>>> m.element_finder()(np.array([.2]))
array([1])
>>> m.element_finder()(np.array([.7]))
array([0])

The original version before #630 gave the correct result here. I'll have to check #630 carefully.

Alright, so it was a change from self.t[1] to self.t[0]. Maybe we shouldn't assume anything.

Does #634 fix the issue in the point source?

No, it gives different answers for
mesh = skfem.MeshLine(np.linspace(0, 1, 2 ** 1 + 1))

and
mesh = skfem.MeshLine().refined(1)

and indeed
>>> MeshLine(np.array([0, .5, 1])).element_finder()(np.array([.7]))
array([0])
>>> MeshLine().refined(1).element_finder()(np.array([.7]))
array([1])

The first of these is wrong, since
>>> m = MeshLine(np.array([0, .5, 1]))
>>> m.p[0, m.t]
array([[0. , 0.5],
       [0.5, 1. ]])

(Both meshes look the same in terms of m.p[0, m.t].)


Recommended upgrade path from 0.x app?

Hey,
What would you recommend as upgrade path from a 0.68 project.
Going to 2 or it would be better upgrading to 1 than 2.
Is it at all possible you think, or it's not very recommended?
Any pointers would be highly appreciated,
Thanks

Hey @dstpierre.
To be clear, v1 was released more as a milestone release and does not have any breaking changes (v2 has them and has the guide). The problem is that every minor release from 0.68.0 till 0.88.2 has breaking changes (it's least 20 breaking changes 💥).
If you have time, I suggest to jump to v2 directly via the migration guide and check release notes for previous releases as there were a lot of fixes 🙄


Environment variable for runtime-interface-emulator-address

Please allow setting the runtime-interface-emulator-address via an environment variable.
At the moment it's only possible to configure it through the command line.
Use case:
Trying to run a lambda locally (e.g. public.ecr.aws/lambda/ruby:3.2-x86_64) which needs to connect to a port on the host using the --network host parameter.
At the moment I need to manually patch the lambda-entrypoint.sh file, e.g. change the line to configure the port
From:
exec /usr/local/bin/aws-lambda-rie $RUNTIME_ENTRYPOINT

To:
exec /usr/local/bin/aws-lambda-rie $RUNTIME_ENTRYPOINT --runtime-interface-emulator-address=<IP_ADDRESS>:9999

Duplicate of #120


AgonElectronHAL & EZ80F91

I'm trying to use the AgonElectronHAL on an ESP32 [board = nodemcu-32s] with a new eZ80F91 board I'm bringing up and I am having issues reading multiple bytes of data over ZDI.
I see you have quite a few commits for ZDI stability and one issue on the use of out0 vs out. I was wondering if you could provide some pointers on what next steps to take for debugging this issue based on your experience?

ZDSII debug tool works - can read write FLASH and memory to the eZ80F91 board okay.
Custom bootloader loaded via ZDS can read and write memory and print the contents over UART, and using ZDS I can download register status, so board seems good.
I am running at 18.18MHz to mimic EZ80F92 speeds and timing. Faster speeds behave the same too.
I can read product id = 8 and revision = 2 which is correct.
Reading internal flash memory using ZDI & Electron at 0x0 reads the same data continuously
10000000C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0F0
Reading the registers reads garbage as well - 02C0 in this case.

A=C0 BC=0002C0 DE=0002C0 HL=0002C0 IX=0002C0 IY=0002C0 SPL=0002C0
F=02 N..VA. STAT=.HAMI PC=0002C0 MB=00 IFF=0 I=C0 R=C0 SPS=02C0 (registers)

I tried envenomator/agon-recovery tool as well and again I see the same issue. I can read product ID and version but reading flash or SRAM memory reads garbage.
An oscilloscope shows ZCL and ZDA toggling okay.
I also tried two different ESP32-S3 modules using different pins and had the exact same behaviour.

Any ideas on how to proceed with debugging ZDI communication would be greatly appreciated?

Hi,From the looks of it, it could be timing or speed issue? I don’t have F91 but a F92 so can’t test that out. I used to hook up my cheap logic analyzer to pulseview and made a filter for it to see the commands, etc. Then I checked both the original Zilog cable and mine to spot differences. It’s quite a bit of work but it worked in the end.I am busy with a new Agon with a F91. Still have to create the boards and solder them but will probably run into the same problem you described. Then I can debug it better. From a distance it’s difficult to guess what’s wrong. Sorry.Sent from my iPadOn 23 Aug 2024, at 20:06, dennowiggle @.***> wrote:
I'm trying to use the AgonElectronHAL on an ESP32 [board = nodemcu-32s] with a new eZ80F91 board I'm bringing up and I am having issues reading multiple bytes of data over ZDI.
I see you have quite a few commits for ZDI stability and one issue on the use of out0 vs out. I was wondering if you could provide some pointers on what next steps to take for debugging this issue based on your experience?
ZDSII debug tool works - can read write FLASH and memory to the eZ80F91 board okay.
Custom bootloader loaded via ZDS can read and write memory and print the contents over UART, and download registers, so board seems good.
I am running at 18.18MHz to mimic EZ80F92 speeds and timing. Faster speeds behave the same too.
I can read product id = 8 and revision = 2 which is correct.
Reading internal flash memory using ZDI & Electron at 0x0 reads the same data continuously
10000000C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0F0
Reading the registers reads garbage as well - 02C0 in this case.
F=02 N..VA. STAT=.HAMI PC=0002C0 MB=00 IFF=0 I=C0 R=C0 SPS=02C0
(registers)```

I tried envenomator/agon-recovery tool as well and again I see the same issue. I can read product ID and version but reading flash or SRAM memory reads garbage.
An oscilloscope shows ZCL and ZDA toggling okay.
I also tried two different ESP32-S3 modules using different pins and had the exact same behaviour.

Any ideas on how to proceed with debugging ZDI communication would be greatly appreciated?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

Thanks for the feedback. Would it be possible to share your ZDI protocol decoder for PulseView and I can start with my little analyzer.
That's exciting that you are working on an F91 Agon. I couldn't find many designs using the F91 that share info so will be great to have another one. Exciting.

I found a typo below. The second call to zdi_write_registers should contain ZDI_ISL1 for a two byte command.
void zdi_cpu_instruction_out (uint8_t regnr, uint8_t value)
{
    // ld a, nn
    uint8_t instructions[2];
    instructions[0]=value;
    instructions[1]=0x3e;
    zdi_write_registers (ZDI_IS1,2,instructions);
    // out0 (nn),a
    instructions[0]=regnr;
    instructions[1]=0xd3;
    zdi_write_registers (ZDI_IS2,2,instructions);
}

Looking further I realised that out(n),a will not work as the contents of A are placed on the address bits 15->8. Those bits need to be 0. This is an excerpt from the eZ80 CPU user manual for out(b),a, page 269.
The n operand is placed on the lower byte of the address bus, ADDR[7:0]. The CPU
places the contents of the accumulator, A, onto the middle byte of the address bus,
ADDR[15:8]. The upper byte of the address bus, ADDR[23:16] is undefined for I/O
addresses. The CPU next outputs the contents of the accumulator to this I/O address.

An alternate way to out0(regnr), value is this as the contents of b=0 are placed on bits A15->A8.
``
ld a, value
ld bc, regnr,
out (c), a

I now have ESP32/nodemcu and ESP32-S3 working with ZDI to F91.
There were a few issues, mostly on my side. Documenting here in case you want to implement improvements.

My boot code switching between ADL and Z80 mode was wrong which caused the CPU to encounter illegal commands and perhaps enter a strange state,. I added a zdi reset command followed by a break command which helped that issue but I encountered another caused by this change.

ZDI was hanging after reset. I added a 100us delay after issuing a reset command and then polled ZDI_MASTER_CTL on zdi until bit 7 was cleared, waiting 100us between each poll. Accessing ZDI straight after the reset was unstable. So far so good.

I needed to switch back to using the out0(n),a instruction as the out(n),a did not work for a > 0, and thus the internal CPU registers were not configured correctly. I erased the internal memory several times with the ZDSII tool (per other closed issue) and each time I could access the CPU over ZDI from ESP32 fine, internal registers seemed configured okay, and I was able to flash a new image using ZDI.

You can close this issue.


Debug Option?

I can't seem to be able to get this to start on my raspberry pi.  The listener never starts, is there a debug option to starting it? This is what I'm currently seeing when starting:
root@pi2:~# smartthings-mqtt-bridge
info: Starting SmartThings MQTT Bridge - v1.3.4
info: Loading configuration
info: Loading previous state
info: Perfoming configuration migration
info: Saving current state
info: Connecting to MQTT at localhost:1883

If it doesn't get further than here, it looks like it can't connect to your
mqtt server. Did you verify the host and ports?
On Sun, Aug 27, 2017 at 16:12 jon102034050<EMAIL_ADDRESS>wrote:

I can't seem to be able to get this to start on my raspberry pi. The
listener never starts, is there a debug option to starting it? This is what
I'm currently seeing when starting:
root@pi2:~# smartthings-mqtt-bridge
info: Starting SmartThings MQTT Bridge - v1.3.4
info: Loading configuration
info: Loading previous state
info: Perfoming configuration migration
info: Saving current state
info: Connecting to MQTT at localhost:1883
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/stjohnjohnson/smartthings-mqtt-bridge/issues/102, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAl98XqxJ5DG62tLo2NoKuckLsJHtNm5ks5scffsgaJpZM4PD80l
.

--
-St. John

resolved my issue, it had to do with my connection to MQTT within the config.yml.  I had defined the host:port, and apparently it didn't like that.  This is what worked:
host: localhost