persist state filling up local storage very quickly

Hey there,
I have a fairly large redux store, and my chrome debugger gets full quite quickly resulting in this message:
Could not write debug session to localStorage: DOMException: Failed to execute 'setItem' on 'Storage': Setting the value of 'redux-dev-session-1' exceeded the quota.(…)"

Does anyone know of a good way of dealing with this issue? Can localStorage be auto-cleared if it fills up? Maybe there could also be an option to have persistState only send a smaller portion of the action history if the redux store is above some threshold?
Any help would be appreciated!

Well, it depends on the browser preference and disk space. What you want to do is to whether click Commit button on log monitor every time to clear the history or even better to limit the history by adding maxAge parameter as following:
const enhancer = compose(
  applyMiddleware(d1, d2, d3),
  DevTools.instrument({ maxAge: 50 })
);

Sweet thanks for the maxAge tip!

A follow up question: I would like to blacklist some of the reducers from being persisted. Is that possible to do with persistState? Or should I look at a different implementation of persistState to do that?

@zalmoxisus @gaearon ?

I'm still unfortunately hitting this issue much more often than I'd like (often after changing pages through redux-router). Is there something else I should look into for fixing this issue?

When the persisted lifted state is loaded, the reducers are recomputed with those values. Not having all the data, will cause unexpected results.
However, if you want to filter some data you can add a store enhancer as following:
const persistSelector = createStore => (reducer, preloadedState, enhancer) => {
  const store = createStore(reducer, preloadedState, enhancer);
  const getState = store.getState;
  store.getState = () => {
    const liftedState = getState();
    // Here you can filter what to be persisted
    return liftedState;
  };
  return store;
};

Then apply it:
const enhancer = compose(
  applyMiddleware(thunk),
  DevTools.instrument(),
  persistSelector,
  persistState(
    window.location.href.match(
      /[?&]debug_session=([^&#]+)\b/
    )
  )
);

In 3.0 version of Redux DevTools Extension it will be possible to save and load a state history (locally or remotely). For now, you can use Export / Import button there, which would solve the localStorage issues.

Hey @zalmoxisus , thanks for the tips! When do you think version 3.0 will drop?

@tnrich, I'd like to have it at the end of the year, but cannot guarantee yet.

Hey @tnrich, I'm still not ready for 3.0, but I went ahead and made a PR for addressing this issue. I'll publish 2.12 these days along with other changes.

That's great to hear. Thanks!
On Wed, Dec 28, 2016, 5:20 AM Mihail Diordiev<EMAIL_ADDRESS>wrote:

Hey @tnrich https://github.com/tnrich, I'm still not ready for 3.0, but
I went ahead and made a PR for addressing this issue
https://github.com/zalmoxisus/redux-devtools-extension/pull/276. I'll
publish 2.12 these days along with other changes.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/gaearon/redux-devtools/issues/298#issuecomment-269476029,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACmqcZIPK0TYpJI5doRNLb3wgCmQfAp5ks5rMmIQgaJpZM4JGllz
.


include_tmax not considered in mne.io.Raw.crop to check tmax in bounds

Description of the problem
When cropping a Raw to its exact duration without including tmax, the code crashes.
This is clearly an edge case, but I would expect this to work.
Am I wrong on this?
Steps to reproduce
import numpy as np
import mne

n_seconds = 1
sfreq = 100
ch_names = ['1']
data = np.ones((len(ch_names), sfreq*n_seconds))
info = mne.create_info(ch_names=ch_names, sfreq=sfreq)
raw = mne.io.RawArray(data, info)
raw.crop(tmax=n_seconds, include_tmax=False)

Link to data
No response
Expected results
The code should run and the raw should not change, since it is cropped for the entire data.
Actual results
Code crashes.
Additional information
Platform:         Linux-5.15.63-flatcar-x86_64-with-glibc2.31
Python:           3.9.7 | packaged by conda-forge | (default, Sep  2 2021, 17:58:34)  [GCC 9.4.0]
Executable:       /opt/conda/bin/python3
CPU:              x86_64: 64 cores
Memory:           251.8 GB
mne:              1.1.dev0
numpy:            1.23.3 {}
scipy:            1.9.1
matplotlib:       3.6.0 {backend=module://matplotlib_inline.backend_inline}
sklearn:          1.1.2
numba:            Not found
nibabel:          Not found
nilearn:          Not found
dipy:             Not found
cupy:             Not found
pandas:           1.5.0
pyvista:          Not found
pyvistaqt:        Not found
ipyvtklink:       Not found
vtk:              Not found
qtpy:             Not found
ipympl:           Not found
pyqtgraph:        Not found
pooch:            Not found
mne_bids:         Not found
mne_nirs:         Not found
mne_features:     Not found
mne_qt_browser:   Not found
mne_connectivity: Not found
mne_icalabel:     Not found

@gemeinl with what you do you have
>>> raw.times
array([0.  , 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1 ,
       0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2 , 0.21,
       0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3 , 0.31, 0.32,
       0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4 , 0.41, 0.42, 0.43,
       0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.5 , 0.51, 0.52, 0.53, 0.54,
       0.55, 0.56, 0.57, 0.58, 0.59, 0.6 , 0.61, 0.62, 0.63, 0.64, 0.65,
       0.66, 0.67, 0.68, 0.69, 0.7 , 0.71, 0.72, 0.73, 0.74, 0.75, 0.76,
       0.77, 0.78, 0.79, 0.8 , 0.81, 0.82, 0.83, 0.84, 0.85, 0.86, 0.87,
       0.88, 0.89, 0.9 , 0.91, 0.92, 0.93, 0.94, 0.95, 0.96, 0.97, 0.98,
       0.99])

so the max time is not 1s. Even with include_tmax=True it complains.
Not sure how to proceed here...

But the rec has a duration of 1s. I should be able to crop it to 1s, no? That it crashes with include_tmax=True is obvious, I think. It asks for one additional time when it already crashes without it.
From my point of view, if I call crop with tmax=1 and include_tmax=False, internally tmax should be reduced to tmax - 1 / raw.info["sfreq"]. Does this make sense to you?

can you try to open a PR about this and see if it breaks anything?

But the rec has a duration of 1s. I should be able to crop it to 1s, no?

It appears we're off by one sample here, yes.
btw there's a bug in the HTML repr that is not present in the text repr:


feat: add directly imported deps to --mark-module

With the current logic there might be arrows between these first-degree imports if they directly import each other, but there are no arrows if one indirectly imports another...

Here is the current behaviour now.
Q: The implication --mark-module ==> --include-direct-deps means there are some nodes like Lean in the second image. Is this fine?
Image 1: --mark-module

Image 2: --mark-module --include-deps

Image 1: --mark-module

Image 2: --mark-module --include-direct-deps

Image 3: --mark-module --include-deps

Image 3: --mark-module --include-deps --include-direct-deps

Image 1: --mark-module

Image 2: --mark-module --include-direct-deps

Image 3: --mark-module --include-deps

Image 4: --mark-module --include-direct-deps --include-deps


A new bug is detected by LAW. How to send this event to ADO ?

@xyz-mtl Please help with the link to the document on which the above document feedback request is created.
In case your query is not related to document content, open a new forum thread in Microsoft Q&A or Stack Overflow as it could be better place to get help on your scenario. These forum community members could provide their expert guidance on your scenario based on their experience. Thanks.

link


Simplify SA1605 tests using [Theory]

Code looks good :+1:
Have you considered using the MemberData attribute, instead of the multiple InlineData ones?

I had not. I'll take a look and see.

@vweijsters I find that it makes it harder to simply look at the test and understand its purpose. It seems the main advantage would be the following situations:

You have a very large number of test cases
Your test methods have inputs that can't be expressed as a constant
You need to generate your test cases at runtime for some reason

Okay, I'm fine with leaving it the way it is.


Add eslint with eslint-config-airbnb

(Please look at the individual commit descriptions for more context)

Thanks for the review and final fixes! The inputProps being nested is apparently an equivalent workaround for the lint rule conflicting: https://github.com/mui/material-ui/issues/10064#issuecomment-373190215 but disabling it is also fine 👍


[Logs]: Add change notice for Cloudflare Logpush User-Agent

Howdy @cmackenzie1, this seems like something that also might warrant being included in the changelog.
Onboarding instructions in our style guide.

@cmackenzie1 any updates on this PR? It's showing some conflicts because of the migration.

@angelampcosta Closing the PR for now. Will reopen once we are closer to making this change.


[Decompiler] Stack Variables

Also cleaning up some IR1 stuff related to type analysis.

Pull Request Test Coverage Report for Build 691636105

164 of 205 (80.0%)  changed or added relevant lines in 17 files are covered.
4 unchanged lines in 2 files lost coverage.
Overall coverage increased (+0.05%) to 68.901%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

decompiler/IR2/AtomicOpTypeAnalysis.cpp
10
12
83.33%

decompiler/analysis/type_analysis.cpp
16
20
80.0%

decompiler/IR2/Form.cpp
10
15
66.67%

decompiler/config.cpp
0
5
0.0%

decompiler/IR2/Env.cpp
16
23
69.57%

decompiler/ObjectFile/ObjectFileDB_IR2.cpp
0
9
0.0%

decompiler/util/config_parsers.cpp
25
34
73.53%

Files with Coverage Reduction
New Missed Lines
%

decompiler/config.cpp
1
0%

decompiler/IR2/Env.h
3
94.74%

Totals

Change from base Build 687768475:
0.05%

Covered Lines:
32542

Relevant Lines:
47230

💛  - Coveralls


Documetanation: airline-predefined-parts confusing expression

I am reporting on the latest github checkout copy.
I think following change should make it easier for many of us.
I see nothing "Before" but the list is "After" this line.
$ git diff
diff --git a/doc/airline.txt b/doc/airline.txt
index 1316b5f..8382d88 100644
--- a/doc/airline.txt
+++ b/doc/airline.txt
@@ -1433,7 +1433,7 @@ same `foo` part.
 Note: Look at the source code and tests for the full API.

 -------------------------------------             *airline-predefined-parts*
-Before is a list of parts that are predefined by vim-airline.
+The following list of parts are predefined by vim-airline.

 * `mode`           displays the current mode
 * `iminsert`       displays the current insert method

By the way, it is not clear that we can use vim native statusline strings with airline#create _right command etc. in ADVANCED CUSTOMIZATION.  I could only guess from examples.  Maybe in addition to mentioning ones provided by the extension in the following part, this fact may be worth stressing.
Also, you may want to update your example in the following part to use  AirlineInitOverride() instead of  AirlineInit() to clarify what exactly it does.
FYI: I am using the following after figuring out this for my local setting.
" Display Unicode code point (for utf-8)
function! AirlineInitOverride()
  let g:airline_section_y = airline#section#create_right(['ffenc','U+%04B'])
endfunction
autocmd User AirlineAfterInit call AirlineInitOverride()

@chrisbra
If @osamuaoki don't send PR, I'd like to send it instead.
Specifically, I would like to make the following changes.
Before
Before is a list of parts that are predefined by vim-airline.
After
The following list of parts are predefined by vim-airline.

Sure go ahead

Am 04.10.2019 um 05:36 schrieb IK<EMAIL_ADDRESS>
@chrisbra
If @osamuaoki don't send PR, I'd like to send it instead.
Specifically, I would like to make the following changes.
Before
Before is a list of parts that are predefined by vim-airline.
After
The following list of parts are predefined by vim-airline.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Thanks. I will do.

On Thu, Oct 03, 2019 at 10:52:12PM -0700, IK wrote:

Thanks. I will do.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.*

Thanks


Q: Getting an invalid_request_error triggering the HTTP Error 404 and 429?

I came across with this problem where if I try to use a simple curl command to make an API call to OpenAI with an unusable API Key with this specific return from OpenAI:
{
    "error": {
        "message": "The model `gpt-4o` does not exist or you do not have access to it.",
        "type": "invalid_request_error",
        "param": null,
        "code": "model_not_found"
    }
}

Then, when I tried to use that same API key on vim-ai using AIRun, I received a 404 error, and when I attempting to use it via AIChat command, I'll get the 429 error.
Here's the API Key I used for above experiment, which is an old and no longer unusable: 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.
Here's the exact curl command I executed:
curl "https://api.openai.com/v1/chat/completions" \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $API_KEY" \
    -d '{
        "model": "gpt-4o",
        "messages": [
            {
                "role": "user",
                "content": "How are you?"
            }
        ]
    }'

Hi, I have just improved error handling in the plugin. In addition to the status code, it will show the error message. That should help you better debug issues like this


[CARBONDATA-3680]Fix add segment issue with SI table

Why is this PR needed?
What changes were proposed in this PR?
Does this PR introduce any user interface change?

No
Yes. (please explain the change and update document)

Is any new testcase added?

No
Yes

Build Success with Spark 2.4.4, Please check CI http://<IP_ADDRESS>:12545/job/ApacheCarbon_PR_Builder_2.4.4/607/

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12545/job/ApacheCarbonPRBuilder2.3/2314/

Build Success with Spark 2.4.4, Please check CI http://<IP_ADDRESS>:12545/job/ApacheCarbon_PR_Builder_2.4.4/636/

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12545/job/ApacheCarbonPRBuilder2.3/2343/

Build Failed  with Spark 2.4.4, Please check CI http://<IP_ADDRESS>:12545/job/ApacheCarbon_PR_Builder_2.4.4/773/

Build Failed  with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12545/job/ApacheCarbonPRBuilder2.3/2480/

@akashrn5 Please fix the CI failures

Build Success with Spark 2.4.4, Please check CI http://<IP_ADDRESS>:12545/job/ApacheCarbon_PR_Builder_2.4.4/806/

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12545/job/ApacheCarbonPRBuilder2.3/2511/

@kunal642 fixed all test cases, please review and merge

LGTM


SEOImageUpdates: User Story 1463880, Small PR 5

Summary
Making a smaller PR.
Fixes #1463880

@mairaw This is the last of the small PRs. There are a lot of warnings on this one, but they are resolved with the previous four PRs. I'll finish this one up once the others are signed off and merged. I'll then pull those updates into there and hopefully all the warnings will be resolved.

@mairaw Feedback incorporated. Updated CATS report:
https://contentqacats.azurewebsites.net/CheckResults/2019051600053/Report/

@mairaw Feedback incorporated. I went with the second option for the image change recommendation you provided. Updated CATS (2 errors for files not included in this project):
https://contentqacats.azurewebsites.net/CheckResults/2019051800003/Report/

@mairaw I've made the changes. CATS report:
https://contentqacats.azurewebsites.net/CheckResults/2019052100049/Report/

Thank you so much @damabe with all your help in this task. We're done! 🎆

Yay!! @mairaw Thanks for your patience and direction.


[Provisioning ACC] Add an event recorder.

What would you like to be added:
An Event recorder in the Provisioning Admission Check Controller.
Why is this needed:

Ease tracing the Admission Check State changes in the workloads lifecycle.
Ease tracing the Admission Check Active condition status changes.

Completion requirements:
This enhancement requires the following artifacts:

[ ] Design doc
[ ] API change
[ ] Docs update

The artifacts should be linked in subsequent comments.

/assign

/assign

Who is working on this issue?

@tenzen-y, I will
/unassign @trasc

@tenzen-y, I will /unassign @trasc

Thank you for letting me know.


Failed: Update autorest version to prerelease 3.0.0-alpha.20240606.5

Generated by Autorest Regen Preview build 20240606.5Triggered from pull request: https://github.com/Azure/autorest.csharp/pull/4800

API change check
API changes are not detected in this pull request.


[Hosts] Word-wrap in additional content window not resets and looks confusing

Microsoft PowerToys version
0.80.1
Installation method
GitHub, PowerToys auto-update
Running as admin
None
Area(s) with issue?
Hosts File Editor
Steps to reproduce

Open Hosts
Open "additional content" window
Resize Hosts window small and then back to wide

✔ Expected Behavior
With word-wrap: The word-wrap turns back when the main window becomes bigger again.
Without word-wrap: A scroll bar is shown.
❌ Actual Behavior
The text is wrapped when the flyout gets smaller and the wrapping is kept even if I make the window big again.
Btw, the wrapping is confusing because each line needs to be commented.

Other Software
No response

This is released with PowerToys 0.82.0


 After this PR, there shouldn't be any more warnings in the console.
Test Plan
Click on all the places on the site and ensure that no warnings appear in the console upon doing any action.
Issues
Closes #199, #163

This is a draft because I don't have the .env yet and there are still some errors when I run the frontend locally that I want to confirm will go away once I run the client with the backend.

Still getting a key error for the map, when I click on "Tuesday"

I think this is probably where the issue is coming from but I'm not totally sure:
https://github.com/icssc-projects/AntAlmanac/blob/fix-warnings/client/src/components/Map/UCIMap.js#L119
Also important note - @falsidge updated UCIMap.js in #223. So you'll probably need to merge in the latest version of main.


Background color

Can I change the background color? I looked for a way to do this but couldn't find anything.

Neovide cant do that.
Instead from neovim use:
" Change #000000 to whatever color you want
hi Normal guibg=#000000


Expose the AllProps and  PickerBaseProps objects

Describe the problem that you have
I am trying to create a rapper to VueDatePicker, the issue that I'm having is that as soon as I use Vue's withDefaults and defineProps, and v-bind the props objects to VueDatePicker, I rightfully lose the default values of the props, like this:
<template>
  <VueDatePicker
    v-bind="props"
    v-model="date"
  />
</template>

<script setup lang="ts">
import type { ModelValue as DatePickerModelValue, VueDatePickerProps } from '@vuepic/vue-datepicker'
import VueDatePicker from '@vuepic/vue-datepicker'
import<EMAIL_ADDRESS>
const props = withDefaults(defineProps<VueDatePickerProps>(), {
})

const date = defineModel<DatePickerModelValue>('')
</script>

It would be nice to have them available somewhere, either via exports or via the VueDatePicker exposes.
Describe the solution you'd like
I noticed that there are already 2 objects defined in your source code that define the defaults for the component, AllProps and PickerBaseProps.
The problem is that you don't export them in entry.esm.ts, and, when we install the package, the source code isn't included, therefore I don't have a real way to access these objects.
You could either export those and any other useful constants/objects in the entry.esm.ts file, include the source code in the npm registry, or expose them using Vue's defineExpose in VueDatePicker, as you are already doing for the following methods:
    defineExpose({
        closeMenu,
        selectDate,
        clearValue,
        openMenu,
        onScroll,
        formatInputValue, // exposed for testing purposes
        updateInternalModelValue, // modify internal modelValue
        setMonthYear,
        parseModel,
        switchView,
        toggleMenu,
        handleFlow,
        dpWrapMenuRef,
    });

Describe alternatives you've considered
I already tried the following, but nothing worked:
1:
const props = withDefaults(defineProps<VueDatePickerProps>(), {
  ...VueDatePicker.props
})

TypeError: parent.insertBefore is not a function

2:
const props = withDefaults(defineProps<VueDatePickerProps>(), {
  ...new VueDatePicker().$options.props,
})

 VueDatePicker is not a constructor

Are there any other ways to do this, without changing the library that might work? Please let me know, thanks

For anyone wondering, the workaround is this:
const props = withDefaults(defineProps<BsdkNewDatePickerProps>(), {
  ...Object.entries(VueDatePicker.props).reduce((acc, [key, prop]: any[]) => {
    acc[key] = prop.default
    return acc
  }, {} as Record<string, any>),
  enableTimePicker: false,
})

Still, it would be nice to have access to those interfaces, or to the source code directly


issue: source amount > balance. However, Kyber does not show noti "Source amount is too high. Your balance is insufficient"

Steps:
Access kyber > exchange  > select RCN token (From side) > input 0.001 eth (To side) > issue.

Expected:
should show noti "Source amount is too high. Your balance is insufficient".

Not bug


Pull http backend RPC from zk and consul.

Pull http backend RPC from zk and consul.
Dubbo backend PRC support pull the API config from reg-center.  HTTP also needed.

Agree. e.g: springcloud. this need to do in next step.


Dropping empty chunks for multi-page apps

Clear and concise description of the problem
When building multi-page apps, especially when using templating, it's possible to generate an HTML file that doesn't include any JavaScript. When trying to get those files through the rest of Vite's transforms as an input, Vite will generate an empty JavaScript file/chunk.
Suggested solution
When generating a JavaScript chunk, whose only dependent is HTML, and it's empty, instead don't generate the chunk and don't inject the source tag for it.
Alternative
Pushing these files out after Vite runs, but then it prevents Vite optimizations from happening

Duplicate of https://github.com/vitejs/vite/issues/3127, let's discuss there about the different aproaches


Added documentation on starting Selenoid under Windows (fixes #239)

Codecov Report

Merging #240 into master will decrease coverage by 0.29%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master     #240     +/-   ##
=========================================
- Coverage   75.88%   75.59%   -0.3%
=========================================
  Files           5        5
  Lines         676      676
=========================================
- Hits          513      511      -2
- Misses        141      145      +4
+ Partials       22       20      -2

Flag
Coverage Δ

#unittests
75.59% <ø> (-0.3%)
:arrow_down:

Impacted Files
Coverage Δ

selenoid.go
75.07% <0%> (-0.58%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b8f0c62...e3b4f2c. Read the comment docs.


shift-home

Is it the intent that shift-home should move the cursor to the start of the line on MacOS?  This behavior seems to be gone from zsh as opposed to bash so perhaps it's no longer expected.
Wondering if there's some key definition I have missed.

@carljmosca Thanks for the feedback! We could add the support for this.

Also, Kui always interprets Home and End to be history navigation. This is consistent with macOS Terminal, but not with Linux or Windows, or these keys navigate the command line.

Also, Kui always interprets Home and End to be history navigation. This is consistent with macOS Terminal, but not with Linux or Windows, or these keys navigate the command line.


Fix Forecast plot method

Issue #, if available:
Description of changes: changes in matplotlib broke the plot method.
from gluonts.model import SampleForecast
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

forecast = SampleForecast(
    samples=np.random.normal(size=(100, 24)),
    start_date=pd.Period("2023-04-05", freq="D"),
)

forecast.plot()
plt.show()

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Please tag this pr with at least one of these labels to make our release process faster: BREAKING, new feature, bug fix, other change, dev setup

Do we want to try both or require matploblib 3.8?

The PR introducing this line has been there for almost a year, and appears (if I read GitHub's UI correctly) to be part of the 3.7.x tags as well.
Also this has been there for 7 years.

@jaheba tested it manually with matplotlib~=3.6, added constraints to the requirements where it appears


Add git-core PPA to whitelist

This enables people to install the latest Git version (currently 2.7) on the Travis CI Ubuntu Linux containers.
More info here:
https://launchpad.net/~git-core/+archive/ubuntu/ppa

Thank you @BanzaiMan @joshk ! :+1: :smile:


Responsive NavBar with better Hover Property and Suitable Color For Website And Font

Prerequisites

[X] I am an Open Source Contributor
[X] I checked to make sure that this issue has not already been filed

Description
I would Like To Modify The Footer To Look More Responsive and Better Hover Property also I Will Work On the Font And Background Color.
Screenshots
No response
Code of Conduct

[X] I follow Contributing Guidelines of this project.

@Rakesh9100 Assign This Issue To Me  Under SWOC

Issue already exist for the UI enhancement. Pls create new issues @Kapileswar-Moharana


trident forge should possibly alter the Nr_autosomal_SNPs entry in Janno files

trident forge can result in fewer SNPs listed in the target of the forge compared to the sources. This may particularly happen if you use the --intersect option. I would consider it currently a bug that this isn't done, as users clearly expect the number in Nr_autosomal_SNPs to be related to the genotype data, and not to some abstract original calling.

Uff, yes of course. Goodness. This needs to be fixed! I'll work on it.

Thanks Stephan - though maybe we can merge #107 first. Probably more easy this way round.


Blastn

PR checklist

[ ] This comment contains a description of changes (with reason).
[ ] If you've fixed a bug or added code that should be tested, add tests!
[ ] If you've added a new tool - have you followed the pipeline conventions in the contribution docs
[ ] Make sure your code lints (nf-core lint).
[ ] Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
[ ] Usage Documentation in docs/usage.md is updated.
[ ] Output Documentation in docs/output.md is updated.
[ ] CHANGELOG.md is updated.
[ ] README.md is updated (including new tool citations and authors/contributors).

@alxndrdiaz Pending tasks, convert blastn module to execute the commannd in run_blastn rule:
shell:
     """if [ -s {input.fasta} ]; then \
         blastn -task megablast \
             -query {input.fasta} \
             -db {params.db} \
             -outfmt "6 qseqid staxids bitscore std" \
             -max_target_seqs {params.max_target_seqs} \
             -max_hsps 1 \
             -evalue {params.evalue} \
             -num_threads {threads} {params.taxid_flag} \
             -lcase_masking \
             -dust "20 64 1" \
             > {output} 2> {log} || \
         sleep 30; \
         if [ -s {log} ]; then \
             echo "Restarting blastn without taxid filter" >> {log}; \
             > {output}; \
             blastn -task megablast \
                 -query {input.fasta} \
                 -db {params.db} \
                 -outfmt "6 qseqid staxids bitscore std" \
                 -max_target_seqs {params.max_target_seqs} \
                 -max_hsps 1 \
                 -evalue {params.evalue} \
                 -num_threads {threads} \
                 -lcase_masking \
                 -dust "20 64 1" \
                 > {output} 2>> {log}; \
         fi \
     else \
         > {output}; \
     fi"""

Also fix the linting issues related to the blastn databases format.

@alxndrdiaz Pending tasks:

Output file (GCA_922984935.2.blastn.txt) is empty, this is not the expected output (?), so, the module should be modified to get the expected output.
Fix linting errors related to bin/nohitlist.sh script and blastn databases.
Once the linting errors are fixed add the blast database to run the full test (gfLaeSulp1.1).

Small test runs successfully using Nextflow version 22.10.1, but the expected version is >=23.04.1.

Closing and opening a new one for a final merge. Give me a couple days.


Bad port '0' on boot2docker ssh comand on windows 8

Hello,
I have installed boot2docker on windows 8 and when i try ssh from the git bash i'm unable to connect.
I would like to know what setting shall i set on profile in order to be able to use boot2docker.exe command
The boot2docker VM works and docker too because i am able to connect using putty
$ boot2docker.exe ssh
Bad port '0'
error in run: exit status 255
$ boot2docker.exe config
boot2docker profile filename: C:\Users\ioan.pop.boot2docker\profile
Init = false
Verbose = false
Driver = "virtualbox"
Clobber = false
ForceUpgradeDownload = false
SSH = "ssh"
SSHGen = "ssh-keygen"
SSHKey = "C:\Users\ioan.pop.ssh\id_boot2docker"
VM = "boot2docker-vm"
Dir = "C:\Users\ioan.pop.boot2docker"
ISOURL = "https://api.github.com/repos/boot2docker/boot2docker/releases"
ISO = "C:\Users\ioan.pop.boot2docker\boot2docker.iso"
DiskSize = 40000
Memory = 2048
CPUs = 4
SSHPort = 2022
DockerPort = 2376
HostIP = "<IP_ADDRESS>"
DHCPIP = "<IP_ADDRESS>"
NetMask = [255, 255, 255, 0]
LowerIP = "<IP_ADDRESS>"
UpperIP = "<IP_ADDRESS>"
DHCPEnabled = false
Serial = false
SerialFile = "\.\pipe\boot2docker-vm"
Waittime = 300
Retries = 75
$ boot2docker.exe info
{
"Name": "boot2docker-vm",
"UUID": "a627244e-4ace-43c4-a4eb-a6f54182b618",
"Iso": "C:\Users\ioan.pop.boot2docker\boot2docker.iso",
"State": "running",
"CPUs": 4,
"Memory": 2048,
"VRAM": 8,
"CfgFile": "C:\Users\ioan.pop\Desktop\Ioan\Virtual Machin
es\boot2docker-vm\boot2docker-vm.vbox",
"BaseFolder": "C:\Users\ioan.pop\Desktop\Ioan\Virtual Machines\boo
t2docker-vm",
"OSType": "",
"Flag": 0,
"BootOrder": null,
"DockerPort": 0,
"SSHPort": 0,
"SerialFile": "\.\pipe\boot2docker-vm"
}
Docker command works
$ docker info
Containers: 4
Images: 66
Storage Driver: aufs
Root Dir: /mnt/sda1/var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 74
Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.0.7-boot2docker
Operating System: Boot2Docker 1.7.1 (TCL 6.3); master : c202798 - Wed Jul 15 00:
16:02 UTC 2015
CPUs: 4
Total Memory: 1.955 GiB
Name: boot2docker
ID: DAS4:V3HW:JAOX:4VW3:VBRL:BXZB:ETMJ:34E7:JILH:IGWR:VDZ6:Y7T7
Debug mode (server): true
File Descriptors: 27
Goroutines: 43
System Time: 2015-08-10T22:56:03.880915914Z
EventsListeners: 1
Init SHA1:
Init Path: /usr/local/bin/docker
Docker Root Dir: /mnt/sda1/var/lib/docker
$ docker.exe ps
CONTAINER ID        IMAGE                      COMMAND                CREATED
STATUS              PORTS
NAMES
3eb26a706041        shipyard/shipyard:latest   "/controller"          3 days ago
Up 10 minutes       <IP_ADDRESS>:8080->8080/tcp
shipyard
bd9c83790398        shipyard/rethinkdb         "/usr/bin/rethinkdb    3 days ago
Up 10 minutes       <IP_ADDRESS>:32770->8080/tcp, <IP_ADDRESS>:32769->28015/tcp,
<IP_ADDRESS>:32768->29015/tcp   shipyard-rethinkdb
6e226d6ea753        shipyard/rethinkdb         "/bin/bash -l"         3 days ago
Up 10 minutes       8080/tcp, 28015/tcp, 29015/tcp
shipyard-rethinkdb-data

Sorry, this repository is long-since deprecated in favor of Docker Toolbox (whose usage is now also discouraged in favor of Docker for Windows and Docker for Mac).


Added channel type to @Param

Currently, when creating params for a command, the @Param's type field doesn't support the channel type, despite it being a possible option type with the official Discord API.
This pull request adds the CHANNEL type to ParamType, updates the NonParamOption type and added a case for it when converting to Discord API types.

https://github.com/fjodor-rybakov/discord-nestjs/tree/master/packages/core#ℹ-channel- how about that?

This was unfortunately only mentioned in the docs in the definitions list, and I didn't see it. I don't exactly understand why it has to be a different decorator, but I'll use that. Thanks.

use @Channel then got issue like this:
node_modules/@discord-nestjs/core/dist/explorers/option/option.explorer.js:34
name: paramDecoratorMetadata.name ?? propertyKey,
^
TypeError: Cannot read properties of undefined (reading 'name')
any comments @fjodor-rybakov thanks


cargo build: dependency conflict

the issue appeared after commit.. https://github.com/chiselstrike/chiselstrike/commit/b8880a6a67178280ae3c70b4a1661709791d7961

You need to update the submodule: git submodule update.

still same..

Something doesn't add up
% grep rsa server/Cargo.toml
rsa = "0.7.0-pre"

What do you get with the above grep?

Something doesn't add up
% grep rsa server/Cargo.toml
rsa = "0.7.0-pre"

What do you get with the above grep?

Thanks.. indeed it was outdated "0.5.0".. works now..


Integrate Read the Docs; fixes #25

Codecov Report

Merging #33 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #33   +/-   ##
=======================================
  Coverage   95.15%   95.15%
=======================================
  Files           5        5
  Lines         681      681
=======================================
  Hits          648      648
  Misses         33       33

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f10f1b8...1a1d191. Read the comment docs.


Decide whether regex is worthwhile as a feature

Regex in the router only makes sense if we can try, fail, then continue matching trying to route a path.
Otherwise, we're better off handling the value as a dynamic parameter, then running the regex after the fact.
Struggling to think of a scenario where we'd want this.

Here's a complex route that GitLab has:
--[ Route 2047 ]--------------------------------------------------------------------------------------------
Prefix            |
Verb              | POST
URI               | /*namespace_id/:project_id/*all(.:format)
Controller#Action | application#route_not_found {:project_id=>/(?!((?i-mx:\-|badges|blame|blob|builds|commits|create|create_dir|edit|environments\/folders|files|find_file|gitlab\-lfs\/objects|info\/lfs\/objects|new|preview|raw|refs|tree|update|wikis))\/)(?-mix:(?:[a-zA-Z0-9_\.][a-zA-Z0-9_\-\.]{0,254})(?-mix:(?<!\.git|\.atom)))/, :namespace_id=>/(?-mix:(?!((?i-mx:\-|\.well\-known|404\.html|422\.html|500\.html|502\.html|503\.html|admin|api|apple\-touch\-icon\.png|assets|dashboard|deploy\.html|explore|favicon\.ico|favicon\.png|files|groups|health_check|help|import|jwt|login|oauth|profile|projects|public|robots\.txt|s|search|sitemap|sitemap\.xml|sitemap\.xml\.gz|slash\-command\-logo\.png|snippets|unsubscribes|uploads|users|v2))\/)(?-mix:(?:[a-zA-Z0-9_\.][a-zA-Z0-9_\-\.]{0,254}[a-zA-Z0-9_\-]|[a-zA-Z0-9_])(?-mix:(?<!\.git|\.atom))))(?:\/(?!(?i-mx:\-|badges|blame|blob|builds|commits|create|create_dir|edit|environments\/folders|files|find_file|gitlab\-lfs\/objects|info\/lfs\/objects|new|preview|raw|refs|tree|update|wikis)\/)(?-mix:(?:[a-zA-Z0-9_\.][a-zA-Z0-9_\-\.]{0,254}[a-zA-Z0-9_\-]|[a-zA-Z0-9_])(?-mix:(?<!\.git|\.atom))))*/}

Would we be able to support this, without regex built into the router?

We could change our insert to work like:
let mut router = Router::new();
router.insert(
  Route::new("/user/<name>.<ext>")
    .constraint("name", r"[a-z]+")
    .constraint("ext", r"png|jpg"),
  1,
)?;

For simple routes, we could implement From<&str>, so it would only be verbose for regex routes.
This would still work fine for config driven approaches.


Test Workflow

Testing...

Thank you for creating this issue!
I'll get back to you as soon as I can.
Have a great day!
--
This comment was created automatically.


Meta-config files (e.g. Binderfile)

At the Jupyter sprint we discussed setting up a "meta-config" file called a Binderfile. I think we should think about implementing this sooner than later, as it will take care of many user concerns (e.g. running shell commands) and will be helpful for extending binder functionality (e.g., telling Binder to start in JupyterLab mode).
General behavior

multiple file types supported in a hierarchical fashion (e.g., reqs.txt, env.yml etc)
for these, if one if found, then it triggers a build and anything lower in the hierarchy doesn't happen

Specific to binderfile

if you want MORE than one of these to be triggered, you need a binder.yml file. This has key: value pairs.

the key is the name of one of the files that could be in the hierarchy (e.g. pip, conda)
the value is a list of items, one per line, that mimics exactly the content that would have been inside of the file if were a standalone file instead of being inside of binder.yml

In this case, each key will be triggered

Example
inside binder.yml:
image: <path-to-image>

pip:
    package1
    package2==2.0
conda:
    package3
bash:
    touch myfile.txt
    python myfile.py
    nbconfig etc etc

Outstanding questions

Should each section name be exactly the same as the name of the file if it were standalone? E.g., of the form e.g, use environment.yml: and requirements.txt: above instead of conda: and pip:, respectively.
How to support that the value can also point to a file of the same nature (e.g., ./docs/requirements.txt)

Links
Original etherpad

Going to edit this in a while, but we definitely don't want the general behavior listed here, and that's not what we ended up agreeing on. I want them to all be able to trigger (so you can have an environment.yaml file and a requirements.txt and an apt.txt file and it's all ok). I'll edit task later (I meant to edit original etherpad but never got around to it)
Most of the rest is ok :D

See the linked etherpad at the bottom...I basically just copy/pasted what was in there into this issue.

Yeah, am pretty sure we evolved quite a bit at the end and then I forgot to update the notes. Have updated the task instead now.

I've merged this into master now.


{iot} HTML tags are rendering as empty parameter values in reference

Describe the bug
Problem to solve:
HTML tags in source code are causing build warnings in MicrosoftDocs/azure-docs-cli. More importantly, customer-facing reference docs are not instantiating properly -- parameter values formatted with `<>` are missing. Here's an example:

Requested solution:
These values should be escaped with a backtick. Here's an example from az webpubsub.

Published reference guidelines
Guidelines for Parameters used in examples
Related GitHub issues:
This is the same issue as these GitHub issues.
List of HTML tag instances that need update:

File name
Build error

latest/docs-ref-autogen/iot/central/app.yml
HTML tag 'subdomain' isn't allowed.  Replace it with approved Markdown or escape the brackets if the content is a placeholder.

latest/docs-ref-autogen/iot/central/device-template.yml
HTML tag 'Device' isn't allowed.  Replace it with approved Markdown or escape the brackets if the content is a placeholder.

latest/docs-ref-autogen/iot/central/device-template.yml
HTML tag 'Device' isn't allowed.  Replace it with approved Markdown or escape the brackets if the content is a placeholder.

latest/docs-ref-autogen/iot/central/export.yml
HTML tag 'Export' isn't allowed.  Replace it with approved Markdown or escape the brackets if the content is a placeholder.

latest/docs-ref-autogen/iot/central/export/destination.yml
HTML tag 'Destination' isn't allowed.  Replace it with approved Markdown or escape the brackets if the content is a placeholder.

latest/docs-ref-autogen/iot/central/job.yml
HTML tag 'Job' isn't allowed.  Replace it with approved Markdown or escape the brackets if the content is a placeholder.

latest/docs-ref-autogen/iot/device.yml
HTML tag 'name' isn't allowed.  Replace it with approved Markdown or escape the brackets if the content is a placeholder.

latest/docs-ref-autogen/iot/device.yml
HTML tag 'name' isn't allowed.  Replace it with approved Markdown or escape the brackets if the content is a placeholder.

latest/docs-ref-autogen/iot/device.yml
HTML tag 'name' isn't allowed.  Replace it with approved Markdown or escape the brackets if the content is a placeholder.

latest/docs-ref-autogen/iot/device.yml
HTML tag 'name' isn't allowed.  Replace it with approved Markdown or escape the brackets if the content is a placeholder.

Related command
az iot
Errors
n/a
Issue script & Debug output
n/a
Expected behavior
n/a
Environment Summary
Azure CLI 2.62.0
Additional context
No response

Thank you for opening this issue, we will look into it.


Remove references to deprecated trait in README

@brendt i think @vpratfr  has taken care of the request you made of me on #50 concerning the README update via this PR. You can give it a look when you're free.

Thanks!


need help with

I am stuck. I am stuck.
**I wish to run a basic hello_world.py program from apache.
I am trying to build apache mod_python using this steps mentioned in this link

I tried by building / making / installing python 3.9.6

executing below command is giving pyobject error. It states, PyObject shoudl actually be PyGenObject
apxs -I ~/src/Python-3.9.6/Include -I ~/src/Python-3.9.6
-c mod_python.c ~/src/Python-3.9.6/libpython3.9.a -lm
error -
command failed with rc=65536
can someone help please ?
**I wish to run a basic hello_world.py program from apache.
Screenshot here -

last line with error code - after execution-

Sadly it looks like this project is dead. FWIW I created an alternative that doesn't require apache at all.
https://github.com/nggit/httpout

Sadly it looks like this project is dead.

What give you this impression?

What gives you this impression?

"There is no plan for additional releases, ..."
This implies "The project is stable, but I've lost the passion/idea to push forward which could make it break."

The maintainer, you. are now primarily using Go. https://grisha.org/blog/2017/04/27/simplistic-go-web-app/
"Go is gradually disrupting the the world of web applications."

Another factor is that now everyone is switching to ASGI frameworks, Starlette, etc.


Enable using Tesseract if it is installed

Need a more detailed settings area for this feature to maybe show what is installed and how to add this option.


pip / setup.py

Explore the possibility to use a setup.py and to have kdmp available on pip

Any update on this by any chance? I can adjust the work I did for udmp-parser to use nanobind for this project too if this help. LMK

I don't have any short term plan to look at this - I looked at it a little bit right after your work on udmp-parser but I had forgotten  that a custom Python extension already existed for kdmp-parser and I couldn't get the CMakefile / pip building working easily so I kind of dropped it.
It might not be hard - it was more that I don't know those technologies very well and working with them can be frustrating :-D.
But yeah if you want to give it a shot, knock yourself out!
Cheers

Thanks for the reply. I should be able to prep you something quickly.

Hey everyone :wave: Thanks a lot for your work Axel, and for your contributions @hugsy. I just wanted to take this opportunity to say that I would really be interested in having kmdp-parser available on pip, if possible. Thanks :)

Hey @neitsa !
I think
python -m pip install git+https://github.com/hugsy/kdmp-parser.git

Should work out of the box for the time being

Hey all,
Yep I am planning to take a look & working to merge crazy hugsy's change in
December - I am currently traveling and that is why I haven't gotten a
chance to do anything about it :)
But I have not forgotten and I'll get on it as soon as I get back.
Cheers and thanks for dropping us a line Neitsa!
On Tue, Oct 31, 2023 at 3:37 PM crazy hugsy @.***>
wrote:

Hey @neitsa https://github.com/neitsa !
I think
python -m pip install git+https://github.com/hugsy/kdmp-parser.git
Should work out of the box for the time being
—
Reply to this email directly, view it on GitHub
https://github.com/0vercl0k/kdmp-parser/issues/14#issuecomment-1787348675,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AALIORJWBVAXG3ALDE4TKLTYCEEJDAVCNFSM4VLEVYNKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZYG4ZTIOBWG42Q
.
You are receiving this because you authored the thread.Message ID:
@.***>

Enjoy your travels buddy 😀
@neitsa if you happen to try the non-yet-merged version let me know how it goes! I can use that time to fix bugs to make the review faster for @0vercl0k
🍻

No worries Axel! Enjoy :)
@hugsy : compilation and usage went without any hassle 👍 . Fought a bit with pip to get it to compile from the src/python directory (otherwise it awaits for a setup.py or .toml file at the root of the repo) until I read the fine manual.
Works like a charm :) Thanks a lot guys!
(.venv) PS G:\git\python> python -m pip install -e "git+https://github.com/hugsy/kdmp-parser.git/#egg=kdmp-parser&subdirectory=src/python"
Obtaining kdmp-parser from git+https://github.com/hugsy/kdmp-parser.git/#egg=kdmp-parser&subdirectory=src/python
  Cloning https://github.com/hugsy/kdmp-parser.git/ to G:\git\python\.venv\src\kdmp-parser
  Running command git clone --filter=blob:none --quiet https://github.com/hugsy/kdmp-parser.git/ 'G:\git\python\.venv\src\kdmp-parser'
  Resolved https://github.com/hugsy/kdmp-parser.git/ to commit c48d801d6fcb296b028e620b1d0bdf37f4c1da34
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: kdmp-parser
  Building editable for kdmp-parser (pyproject.toml) ... done
  Created wheel for kdmp-parser: filename=kdmp_parser-0.5.0-cp311-cp311-win_amd64.whl size=77896 sha256=28e24a04301fa2382224fd2771fb8d254c37c3d149c4146898dfdc1b9f3bb550
  Stored in directory: C:\Users\neitsa\AppData\Local\Temp\pip-ephem-wheel-cache-jl97kngr\wheels\78\08\93\918a183b38d3dfb1b3beb8d66733a5f307ca03f2b1442c8a3e
Successfully built kdmp-parser
Installing collected packages: kdmp-parser
Successfully installed kdmp-parser-0.5.0

(.venv) PS G:\git\python> python
Python 3.11.6 (tags/v3.11.6:8b6ee5b, Oct  2 2023, 14:57:12) [MSC v.1935 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import kdmp_parser
>>> import pathlib
>>> p = pathlib.Path(r"d:\tmp\system_dump.dmp")
>>> fdmp = kdmp_parser.FullDump(p)
>>> fdmp.type
<DumpType.FullDump: 1>
>>> dir(fdmp)
['_KernelDumpParser__dump', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'context', 'directory_table_base', 'filepath', 'read_physical_page', 'read_virtual_page', 'translate_virtual', 'type']
>>> hex(fdmp.directory_table_base)
'0xd99cd000'

Only thing to report atm is, I guess, that the readme could possibly get a bit of refresh since class names changed a bit. I'll continue to test in the upcoming days and report any problems.

Only thing to report atm is, I guess, that the readme could possibly get a bit of refresh since class names changed a bit.

It's important, most people only read the README (if anything 😀) It should be correct.

I'll continue to test in the upcoming days and report any problems

Much appreciated

Hi guys!
Not sure if this warrants a new issue or not. Let me know.
It's probably just a minor nitpick, but the code (haven't tested if it's from python or C++) fails if reading from a physical address that is in large page and is not aligned on a page boundary. If that doesn't makes sense, here's a simple example from a full kernel dump.
Note: testing with KdDebuggerDataBlock as it has an easily recognizable signature.
PS G:\reverse\utils\kdmp-parser> .\parser.exe -a D:\_tmp\system_dump.dmp
--------------------------------------------------------------------------------
Dump structures:
  HEADER64
    +0x0000: Signature                : 0x45474150.
    +0x0004: ValidDump                : 0x34365544.
    +0x0008: MajorVersion             : 0x0000000f.
    +0x000c: MinorVersion             : 0x000023f0.
    +0x0010: DirectoryTableBase       : 0x00000000d99cd002.
    +0x0018: PfnDatabase              : 0xfffff90000000000.
    +0x0020: PsLoadedModuleList       : 0xfffff8017fe2a190.
    +0x0028: PsActiveProcessHead      : 0xfffff8017fe1df80.
    +0x0030: MachineImageType         : 0x00008664.
    +0x0034: NumberProcessors         : 0x00000004.
    +0x0038: BugCheckCode             : 0x00000000.
    +0x0040: BugCheckCodeParameter
    +0x0080: KdDebuggerDataBlock      : 0xfffff8017fe00b20.

KdDebuggerDataBlock is at  0xfffff8017fe00b20.
Opening the dump in kd, translating (v to p), checking the content and the table entries:
0: kd> !vtop d99cd000 fffff8017fe00b20
Amd64VtoP: Virt fffff8017fe00b20, pagedir 00000000d99cd000
Amd64VtoP: PML4E 00000000d99cdf80
Amd64VtoP: PDPE<PHONE_NUMBER>409028
Amd64VtoP: PDE<PHONE_NUMBER>40aff8
Amd64VtoP: Large page mapped phys<PHONE_NUMBER>800b20
Virtual address fffff8017fe00b20 translates to physical address 3800b20.

0: kd> !db 3800b20
# 3800b20 70 05 e4 7f 01 f8 ff ff-70 05 e4 7f 01 f8 ff ff p.......p.......
# 3800b30 4b 44 42 47 80 03 00 00-00 00 20 7f 01 f8 ff ff KDBG...... .....
# 3800b40 70 f0 5f 7f 01 f8 ff ff-00 00 00 00 00 00 00 00 p._.............
# 3800b50 00 00 00 00 00 00 01 00-c0 a7 5f 7f 01 f8 ff ff .........._.....
# 3800b60 00 00 00 00 00 00 00 00-90 a1 e2 7f 01 f8 ff ff ................
# 3800b70 80 df e1 7f 01 f8 ff ff-d0 b5 ef 7f 01 f8 ff ff ................
# 3800b80 00 69 e1 7f 01 f8 ff ff-00 00 00 00 00 00 00 00 .i..............
# 3800b90 00 00 00 00 00 00 00 00-68 b4 ef 7f 01 f8 ff ff ........h.......

0: kd> !pte fffff801`7fe00b20
                                           VA fffff8017fe00b20
PXE at FFFFBADD6EB75F80    PPE at FFFFBADD6EBF0028    PDE at FFFFBADD7E005FF8    PTE at FFFFBAFC00BFF000
contains<PHONE_NUMBER>409063  contains<PHONE_NUMBER>40A063  contains 8A000000038008E3  contains<PHONE_NUMBER>000000
pfn 4409      ---DA--KWEV  pfn 440a      ---DA--KWEV  pfn 3800      --LDA--KWEV  LARGE PAGE pfn 3800

0: kd> dt _mmpte_hardware FFFFBADD7E005FF8 Large*
nt!_MMPTE_HARDWARE
   +0x000 LargePage : 0y1

KdDebuggerDataBlock is somewhere in a data section of the kernel , mapped as a large 2MB (?) page as given by the PDE.
Simple python code:
import pathlib

import kdmp_parser

def main():
    file_path = pathlib.Path("d:/_tmp/system_dump.dmp")
    fdmp = kdmp_parser.KernelDumpParser(file_path)

    virt_addr = 0xfffff8017fe00b20  # KdDebuggerDataBlock virt addr.
    phy_addr = fdmp.translate_virtual(virt_addr)
    if not phy_addr:
        print("failed to get phy addr.")
        return
    print(f"VIRT 0x{virt_addr:016x} --> PHY: 0x{phy_addr:016x}")

    content_virt = fdmp.read_virtual_page(virt_addr)
    if not content_virt:
        print('Failed to read virtual address.')

    # this fails
    content_phy = fdmp.read_physical_page(phy_addr)
    if content_phy is None:
        print("[-] Reading from phy addr failed.")

    # this works (if aligned down to a page boundary).
    content_phy = fdmp.read_physical_page(phy_addr & 0xfffffffffffff000)
    if content_phy is not None:
        print("[+] Reading from phy addr succeeded!")

    print("[*] Exiting")

if __name__ == "__main__":
    main()

Now, if I test the same code with an address somewhere in a 4KB page, it works (e.g PfnDatabase + 0x20 converted to a physical address) even if the physical address is not aligned on a page boundary.
Not a big deal though, once you know it.

Good catch!
It seems that C++ kdmpparser::GetPhysicalPage makes no assumption on the alignment of its address parameter, which may result in the page being not found if the parameter is not aligned.
https://github.com/0vercl0k/kdmp-parser/blob/3bec915e6f5304c187765be7ce3cfde713d7c29b/src/lib/kdmp-parser.h#L233-L246
A little everywhere in kdmp-parser GetPhysicalPage is called with Page::Align which you made me realized this part is not completely exposed to Python. So this is now fixed (see hugsy/kdmp-parser@71d28b9), thanks!
And as for non-alignment part (it's part of the C++ code) I'll let @0vercl0k answer on it as this is an implementation choice.
Cheers!

Happy to fix more bugs/add more improvements that you spot. Don't be shy 😎

+1 for README being cleaned up / reflecting accurate information. And I'll
also check the alignment thing in the C++ part; I can't remember 😅
But yes, keep them coming Neitsa - thank a lot 🙏!
Cheers
On Sun, Nov 5, 2023 at 1:28 AM crazy hugsy @.***> wrote:

Happy to fix more bugs/add more improvements that you spot. Don't be shy 😎
—
Reply to this email directly, view it on GitHub
https://github.com/0vercl0k/kdmp-parser/issues/14#issuecomment-1793591712,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AALIORKRGGYO67WNU7FVVK3YC3MS3AVCNFSM4VLEVYNKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZZGM2TSMJXGEZA
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hey guys, I hope you're doing well :)
Nothing to report, I have been using the library lately without any problems.
Now that Santa is (soon to be) around the corner, a few stuff I'd like to see (should I make a new issue?).
Please note that I can try to implement them if you think some of the propositions mentioned below are worth it. It's more an open discussion than "Pretty please implement it, I'm too lazy to do anything" :p
Feature Requests
What I usually await from a parser is to give me access to the structural shape of the file format, not necessarily parsing the data contained inside those structures (i.e it's not the job of the dump parser to give me the list of processes or threads or whatever you might find inside a dump; I can use the primitive given by the parser to build another library on top). From my PoV, which may be limited by my use cases, the more information I have about the structural format, the better.

Ability to access the header(s) from Python (at least the main header, HEADER64).

I think without the header's information there's not much to do. I guess that most use cases would benefit from having access to all the information contained in the header (e.g. kernel base, loaded module list, active process head, etc.). It's not that hard to re-implement the header parsing on top of the library, but since kdmp-parser already does it, having access to this header would be great. Possible access to other parts of the header (memory runs for ex.) would be great.

(very optional) Access to the KdDebuggerDataBlock.

It's not complicated once you have kdmp-parser (and access to the header) to get the content of that structure (available from wdbgexts.h ). I seem to understand that it might be sometime encoded (from the various full dumps I gathered it was always in clear, though) and can be decoded.
I guess that if it was available from the get go it could be a real plus :) Most of the interesting info is already in the header proper, tough, so no big deal if you decide it's not worth the hassle.
Question
Following our previous discussion, what should a user asking for an address in a large page should be given?

The 4KB page in which the address lies?
The whole 2MB or 1GB page (ouch!)?

Another possibility would be to give some help to the end-user with an additional function that would return the base address of the page / page table table entry / PFN from a physical address. Note sure if it's really helpful, though. I don't have a precise use case for that...
Possible Optimization
I mean, we all know the saying:

Premature optimization is the root of all evil -- Donald Knuth

Now, I was thinking (it happens!): the underlying C++ code is using the dump as a mapped file, and the kernel is doing a great job and not mapping the whole file in one go, but only really mapping the pages you touch in the mapped file.
 You end up, as far I can surmise, with the whole file mapped in memory. Depending on the size of the dump, it can become a burden on the computer memory and you may not even have enough memory.
Since physical runs are quite limited in their number (i.e. PHYSMEM_DESC and PHYSMEM_RUN in the code;, something like around 80 of them at max) and are relatively easy to parse it could technically be possible to just open the file as a "regular" file (i.e not a mapped one) and translate physical address to flat file offset (or virt -> phy -> offset). On Windows it's possible to this in one go (and avoid a SetFilePointer syscall) with just a synchronous ReadFile and a OVERLAPPED structure.
I guess it would nonetheless be slower - no real idea of how much, though - than having a memory mapped file, but it could be a trade-off compared to the memory requirements of a (very?) big memory dump.
Possible implementation could be a base class from which (e.g.) KernelDumpMemoryParser and KernelDumpFileReader could both subclass, with a factory function to decide which one the end-user would like to use.
Or maybe just memory mapping the paging tables (although I guess they could be quite big too :| ) to do the translation, and do the actual page reading by reading the flat file.
Thanks a lot  for reading this wall of text :D

Yo!

Ability to access the header(s) from Python (at least the main header, HEADER64).

I think that's exactly 2 lines to add. Should be doable. Maybe after #19 is reviewed and (hopefully) merged.
I can't help for the rest, I'm not the chief engineer here 😁 But maybe open different issues on Github for your suggestions, it'll make the tracking easier.

Thanks for the feedback neitsa - it makes a lot of sense.
Maybe let's open a issues then:

One to expose more information from the dump to the user
One to explore parsing the dump w/o a memory map but reading it like a
regular file

What do you think?
Regarding the question; I guess the ideal case is we give what the user
asks for (I think the Read API kinda does this by taking in a size
argument). But maybe it's a good time to revisit the  interface a bit and
how it works to make it useful for everybody.
For history, I made this library for https://github.com/0vercl0k/wtf and
the only thing it needed was getting access to the physical memory
addresses & the page contents; so that's kinda why the API is probably not
very well suited for anything else 😅 Happy to improve it though, so maybe
another issue for that as well?
Cheers
On Fri, Nov 10, 2023 at 6:36 AM crazy hugsy @.***>
wrote:

Yo!
Ability to access the header(s) from Python (at least the main header,
HEADER64).
I think that's exactly 2 lines to add. Should be doable. Maybe after #19
https://github.com/0vercl0k/kdmp-parser/pull/19 is reviewed and
(hopefully) merged.
I can't help for the rest, I'm not the chief engineer here 😁 But maybe
open different issues on Github for your suggestions, it'll make the
tracking easier.
—
Reply to this email directly, view it on GitHub
https://github.com/0vercl0k/kdmp-parser/issues/14#issuecomment-1805128904,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AALIORMLQHLVCYSXVOJL773YDW4NDAVCNFSM4VLEVYNKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBQGUYTEOBZGA2A
.
You are receiving this because you were mentioned.Message ID:
@.***>

the only thing it needed was getting access to the physical memory addresses & the page contents

TBF that's the only thing you really need when you think about it. The rest is sugar coating 😀

Oops, merging the PR closed this but there's still some work to be done and I still owe you guys some answers; I haven't forgotten
:).
I'll be re-reading this thread and create new issues to get answers and experiment!
Cheers


Add publish-workflow tests

The PR adds tests for the following scenarios:

Request a review
Cancel a review
Approve/Deny a review
Conclude review
Request publication
Publish document

Also makes changes to the long command to

set the correct user ids
be able to select different user roles

For some reason I haven't been able to get the tests to pass even though clearly the functionality is there and they should be passing tests. I cleared my volumes and pulled the latest develop of frontend and backend. I would be comfortable passing this, and will let @TimMcCauley  see if the tests pass on his end.


Create demographr

After creating the pull request, the R-multiverse GitHub bot will review your contribution on its next scheduled run. Thank you for your contribution!

Closing for the time being as the GitHub release has not yet been made.

Published release https://github.com/humaniverse/demographr/releases/tag/0.0.1


case control sampling questions

Hi @lisu-stats @RoonakR
What do you think is the best approach when for the case-control sampling when there are not enough controls for the specified ratio of case:controls in a period? Should the cases be sampled with replacement? Or just use them all and have fewer than requested?

@gravesti I think we shall use the available controls if the nubmer of controls requrested is larger than the avaiable ones.


Add bibliographic localities to citations

Microformat within citation:
<<citation,section 5,page 7:text>>

Extracted into ISOXML, now to export in Isodoc

Done isodoc


Graphical presentation Of Dynamic Configuration (Control Plane)

Title: Graphical presentation of feature "Dynamic Configuration (Control Plane)"
Description:

Similar to the way we have graphical presentation Of FrontProxy
https://www.envoyproxy.io/docs/envoy/latest/start/sandboxes/front_proxy

can we also have similar presentation for Dynamic Configuration (file system) and for Dynamic Configuration (Control Plane)
It gives better understanding for new users.
https://www.envoyproxy.io/docs/envoy/latest/start/sandboxes/dynamic-configuration-control-plane
https://www.envoyproxy.io/docs/envoy/latest/start/sandboxes/dynamic-configuration-filesystem

It would definitely be nice to have, but I'm not sure anyone with understanding of the system has the cycles to do it :-/
cc @phlax


Woopecker prometheus metrics not getting exposed

Component
server
Describe the bug
Hi team
I have deployed Woodpecker next version in kubernetes but metrics are not getting exposed as it is returning 404
Do we need to provide any extra flag/env variable Can someone pls help me in exposing woodpecker prometheus metrics ?
System Info
Version info {"source":"https://github.com/woodpecker-ci/woodpecker","version":"next-f6811a20"}

Curl output
curl -i --location --request GET 'https://woodpecker.<domain>.com/metrics' \
--header 'Authorization: Bearer <Token>'
HTTP/1.1 404 Not Found

Prometheus scrape config
- job_name: "woodpecker"
  bearer_token: <token>
  static_configs:
  - targets: ["woodpecker.<domain>.com"]

Prometheus response 404; expecting 200; response body: ""

Additional context
No response
Validations

[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Checked that the bug isn't fixed in the next version already [https://woodpecker-ci.org/faq#which-version-of-woodpecker-should-i-use]
[X] Check that this is a concrete bug. For Q&A join our Discord Chat Server or the Matrix room.

You need to set WOODPECKER_METRICS_SERVER_ADDR, or configure authenticated access for https. See https://woodpecker-ci.org/docs/next/administration/prometheus#metric-reference.


SQLAlchemyCRUDRouter - With joined tables

fastapi                  0.63.0
fastapi-crudrouter       0.6.1
Trying to use SQLAlchemyCRUDRouter for joined tables ,
All endpoints created automatically (amazing:)).
Tried using the "create one " for input:
{
  "customer_no": 1,
  "subscriber": [
    {
      "subscriber_no": 1,
      "is_active": false,
      "owner": 1
    }
  ]
}

End up with error:
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/state.py", line 434, in _initialize_instance
    return manager.original_init(*mixed[1:], **kwargs)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/decl_base.py", line 1086, in _declarative_constructor
    setattr(self, k, kwargs[k])
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 427, in __set__
    self.impl.set(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 1512, in set
    collections.bulk_replace(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/collections.py", line 817, in bulk_replace
    appender(member, _sa_initiator=initiator)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/collections.py", line 1131, in append
    item = __set(self, item, _sa_initiator)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/collections.py", line 1096, in __set
    item = executor.fire_append_event(item, _sa_initiator)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/collections.py", line 727, in fire_append_event
    return self.attr.fire_append_event(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 1345, in fire_append_event
    value = fn(state, value, initiator or self._append_token)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 1675, in emit_backref_from_collection_append_event
    child_state, child_dict = instance_state(child), instance_dict(child)
AttributeError: 'dict' object has no attribute '_sa_instance_state'

Code:
Routes:
from config.config import postgress_engine,SessionLocal
from fastapi_crudrouter impor SQLAlchemyCRUDRouter as SQLDBCrudRouter

customer_router = APIRouter()

def get_db():
    session = SessionLocal()
    try:
        yield session
        session.commit()
    finally:
        session.close()

customer_router = SQLDBCrudRouter(schema=Customer,create_schema=CustomerCreate,
             db_model=CustomerModel,db=get_db,prefix='customer')

schemas.py

from pydantic import BaseModel
from typing import List, Optional

class SubscriberBase(BaseModel):
    subscriber_no: int
    is_active: bool = False

class SubscriberCreate(BaseModel):
    pass

class Subscriber(SubscriberBase):
    owner: int

    class Config:
        orm_mode = True

class CustomerCreate(BaseModel):
    customer_no: Optional[int] = None
    subscriber: Optional[List[Subscriber]] =None

class Customer(CustomerCreate):
    id: int
    class Config:
        orm_mode = True

models.py

from sqlalchemy import Boolean, Column, ForeignKey, Integer, String,DateTime
from sqlalchemy.orm import relationship
from config.config import Base

class CustomerModel(Base):
    __tablename__ = 'customer'
    id = Column(Integer, primary_key=True, index=True)
    customer_no= Column(Integer,index=True)
    subscriber= relationship("SubscriberModel", back_populates="owner")

class SubscriberModel(Base):
    __tablename__ = 'subscriber'
    id = Column(Integer, ForeignKey("customer.id"))
    subscriber_no= Column(Integer, primary_key=True, index=True)
    owner = relationship("CustomerModel", back_populates="subscriber")

Hi @avico78. So the CRUDRouter currently does not support inserting nested objects.  This is something I am looking at potentially implementing in the future.
Currently you have the option of overloading routes to provide custom functionality (docs). You could do something as below to get around your issue.
customer_router = SQLDBCrudRouter(...)

@custom_router.post("")
def custom_create_router(customer: CustomerModel)
   #my custom db logic
   ...

app.include_router(customer_router)

Thanks @awtkns ,
Support of nested object would be awesome and actually sound like a real challenge ,
not sure if this can help solving the nested objects ,
but it maybe it may interest you :
https://github.com/brokenloop/jsontopydantic
it building a pydantic classes from a Json - even a real nested one.
The core of this project coming from datamodel-code-generator (v 0.5.39 )
import json
from pydantic import Json
from typing import Dict, Any
from genson import SchemaBuilder
from datamodel_code_generator.parser.jsonschema import JsonSchemaParser
from datamodel_code_generator.model.pydantic import (
    BaseModel,
    CustomRootType,
    DataModelField,
)

def translate(
    input_text: Dict,
) -> str:
    builder = SchemaBuilder()
    builder.add_object(input_text)
    schema = json.dumps(builder.to_schema())
    parser = JsonSchemaParser(
        BaseModel,
        CustomRootType,
        DataModelField,
        text=schema,
    )

    return parser.parse()

if __name__=="__main__":
    data2 = {"nested.":{"a":{"b":{"c":{"dList":[{"e":{"f":"val"}}]}}}}}
    result = translate(data2)
    print(result)

Thanks I will keep that in mind.  Ok if I close this for now?

Thanks I will keep that in mind. Ok if I close this for now?

sure,i will follow :)
hopefully a solution will be find for this ,
Thanks @awtkns ,great initiative


Add github actions for build / test

I'm going to assign that for myself , since you are focused on check command.

https://github.com/nodeshift/oratrix/pull/20/commits/edd39ddf7b5686dce6591b2a8e1d4a0e5fc7133a


Merge workflow.Await support

Pull Request Test Coverage Report for Build 47e2ef52-3eec-410a-9bfe-4b9a80daa0c8

13 of 16 (81.25%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.03%) to 74.818%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

workflow/deterministic_wrappers.go
0
3
0.0%

Totals

Change from base Build 27a0d8f1-f870-4d93-9ebe-75b9cac8a44f:
0.03%

Covered Lines:
9044

Relevant Lines:
12088

💛  - Coveralls


OC-1.2 Default address families & PushConfig() method

Current version of the test seems to rely on PushConfig() method for DUT that is not implemented for staticDUT type in featureprofiles binding.go:
https://github.com/openconfig/featureprofiles/blob/edb20b68c532cb40f7e1e545c3f29bbfe2f021ee/feature/networkinstance/ate_tests/defaults_test/defaults_test.go#L135-L137
Therefore we get an "unimplemented" error message from embeded PushConfig() method of AbstractDUT type.
Could you please provide some guidance here, how this should be resolved?
I see two main options:

the test could be refactored to remove the call to this method, and instead:
a) use gNMI replace, and/or
b) rely on the fact that configs are reset before test exectuion
this method should be implemented

What do you think?
cc @Hobbydos @trathod1 @xw-g

@liulk maybe can help comment from the infra side? Thanks.

I guess #1041 that has just been merged resolves this according to option#1.b. All right then.


[Bug]: Cannot invoke resource methods which contains path parameter with only the parameter type in services

import ballerina/http;

const WELL_KNOWN = ".well-known";

service on new http:Listener(9090) {
    resource function get a/[WELL_KNOWN]() returns string {
        return "Hello, World!";
    }
}

Consider the above service declaration which contains a resource method that has a path parameter only with the parameter type
when I try to invoke it with the following client it gives no matching resource found for path : //a/.well-known , method : GET"
import ballerina/io;
import ballerina/http;

http:Client client1 = check new("http://localhost:9090/");
const string WELL_KNOWN = ".well-known";

public function main() returns error? {
    string unionResult = check client1->/a/[WELL_KNOWN];
    io:println(unionResult);
}

Checked on this issue. We are using the ResourceMethodType:getResourcePath() method to create and register the resource. Seems like this constant value is not properly populated as the resource path. (See the screenshot below)
import ballerina/http;

const WELL_KNOWN = ".well-known";
const PATH = "path";

service on new http:Listener(9090) {
    resource function get a/[WELL_KNOWN]/[PATH]() returns string {
        return "Hello, World!";
    }
}

Had a chat with @HindujaB regarding this seems like the Path Symbol name is not properly updated for constant values. @rdulmina Can you check this on the compiler FE side?

Checked on this issue. We are using the ResourceMethodType:getResourcePath() method to create and register the resource. Seems like this constant value is not properly populated as the resource path. (See the screenshot below)
import ballerina/http;

const WELL_KNOWN = ".well-known";
const PATH = "path";

service on new http:Listener(9090) {
    resource function get a/[WELL_KNOWN]/[PATH]() returns string {
        return "Hello, World!";
    }
}

Had a chat with @HindujaB regarding this seems like the Path Symbol name is not properly updated for constant values. @rdulmina Can you check this on the compiler FE side?

Did u mean the $ in the path? That is the expected behavior

Currently http module does not support this. It requires an improvement from lang side as mentioned. So marking this issue as an improvement


Util: Make_timeline fix for when ACT wasn't open at start of instance

This should resolve #1061. We may have to add more 21 lines to is_instance_restarted, but from the More Than One logs I tested this on, it seems to be happy regardless.

Thanks for this! I've been using a bunch of user submitted logs so having this be more robust will be super helpful.  I can't live without -lf these days!


feat: show info button for last modified times

Fix #179 (continuation of #180)

:tada: This PR is included in version 6.7.0 :tada:
The release is available on:

GitHub release
v6.7.0

Your semantic-release bot :package::rocket:


Add support of suspendable answers to BDD

Post suspendable answer support to BDD mockito as well.
#357

Can we mirror all other methods listed in OngoingStubbing.kt to mirror them to BDDMockito.kt?

Yes sure. But I found only a few cases missing. All others are covered by mockito itself. Also, changed the commit name.


fix: SameFileError #1153

Description
Pull request to fix bug #1153.
When relative paths were provided to rules with the --generate-unit-tests flag the target path and the tests were successfully created. However, when full paths were used, the automatic unittest generation failed. This was caused by behavior of the Path.parent() property. As path and f.parent are matching this leads to f and target matching and the SameFileError raised by shutils.
Behaviour of Path.parent:
>>> from pathlib import Path
>>> f = Path("/dummy/test/path")
>>> f.parent
PosixPath('/dummy/test')
>>> path = Path("/dummy/test/path/")
>>> rulename = "test"
>>> content_type = "expected"
>>> target = path / rulename / content_type / f.parent
>>> target
PosixPath('/dummy/test')

Removing f.parent creates a successful, matching folder structure with both relative and full paths and allows unittests to be generated.
QC

[x] The PR contains a test case for the changes or the changes are already covered by an existing test case.
[x] The documentation (docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).

Sorry for the delay - I did not get a notification from Github about activity. Thanks for running the pipeline. I must have set my local environment incorrectly as I didn't spot this when I ran the tests - I will investigate.

Thanks for finishing and merging this @johanneskoester ! Getting the tests working was proving challenging given the time constraints and being new to the codebase.


Multiple Configuration Crash

Video Hub fails to load after changing the $HOME environment variable to a non-default value.
The Linux version is distributed as an AppImage and saves its data to $home/.config. It is common for Linux users to export a custom $config$ or $home$ environment variable to either

A: specify where the config should be saved, or
B: have multiple configs depending on which shortcut is used to open the AppImage.

An additional feature of AppImage in Linux is to automatically export the $home or $config variable when a folder with the AppImage's name exist in its folder. So if I try to open /wherever/VideoHub.App and the folder /wherever/VideHub.App.home exists, then the app will use /wherever/VideoHub.App.home as the home directory and store its files in /wherever/VideoHub.App.home/.config
Whenever I try to do this with Video Hub App a dialog pops up and immediately vanishes. The app never opens.

I'm sorry I'm not well-versed with how Linux works.
I would be thrilled if someone fixes the problems you describe 🙇

I believe the variable you're looking for is $XDG_CONFIG_HOME - this is what we currently use to store the config files.
You should also be able to use VideoHub.App.config as this exports $XDG_CONFIG_HOME, while VideoHub.App.home only exports $HOME, which we currently don't use.

After some experimenting on my linux box, XDG_CONFIG_HOME takes priority over HOME if it's set, but if it's not, $HOME/.config is used instead.
Also, both .home and .config folders work for me with AppImage - can you confirm you see this when running the AppImage in a terminal?
Setting $HOME to /home/asd/Desktop/Video Hub App 2-2.0.0.AppImage.home
Setting $XDG_CONFIG_HOME to /home/asd/Desktop/Video Hub App 2-2.0.0.AppImage.config

Also, please post any error messages, as I'm unsure what would cause it to crash in any case! 😄

I don't get any error message. I have tried with the folder method and by manually exporting the $home and $XDG_CONFIG_HOME variables.
All that happens is that a small dialog box pops up for a fraction of a second and then vanishes before I can get a good look. The video-hub-app-2 folder is created, along with the three folders and a Network Persistent State file, but not settings.json. I copied over a settings.json but the app still won't start
$ './Video Hub App 2-2.0.0.AppImage'
>Setting $XDG_CONFIG_HOME to /media/ThumbA/Apps/Video Hub App 2-2.0.0.AppImage.config
$

I'm closing this issue as I don't know how I could help with it and the discussion has dried up.
Perhaps with all the Electron updates since the issue was opened, the app works on Linux again 😅 🤞  -- it certainly worked on Ubuntu when I built the AppImage and installed it 🤷♂
Please feel free to comment or open a new issue 🤝


GUI HA with SSL enabled: PVC in pending state, when one gui node is down

Describe the bug
PVC in pending state, when one gui node is down
How to Reproduce?
Please list the steps to help development teams reproduce the behavior

Configured GUI HA in CSI CSO with ssl enabled

  clusters:
    - id: "6576648620455564292"
      secrets: "guisecret"
      secureSslMode: true
      primary:
        primaryFs: "fs1"
      cacert:  "spectrumscale-trusted-ca"
      restApi:
        - guiHost: "monica-multi-scalegui-1.fyre.ibm.com"
        - guiHost: "monica-multi-scalegui-2.fyre.ibm.com"
        - guiHost: "monica-multi-scalegui-3.fyre.ibm.com"

Take first GUI down and tried to create pvc

[root@monica-multi-master yaml]# oc describe pvc pvctest3
Name:          pvctest3
Namespace:     default
StorageClass:  sctest
Status:        Pending
Volume:
Labels:        <none>
Annotations:   volume.beta.kubernetes.io/storage-provisioner: spectrumscale.csi.ibm.com
               volume.kubernetes.io/storage-provisioner: spectrumscale.csi.ibm.com
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode:    Filesystem
Used By:       <none>
Events:
  Type     Reason                Age                  From                                                                                Message
  ----     ------                ----                 ----                                                                                -------
  Normal   Provisioning          84s (x8 over 3m31s)  spectrumscale.csi.ibm.com_ibm-spectrum-scale-csi-provisioner-6d55c779bb-xdnws_a9685963-eccd-43d2-a1f6-98b49d636284  External provisioner is provisioning volume for claim "default/pvctest3"
  Warning  ProvisioningFailed    84s (x8 over 3m31s)  spectrumscale.csi.ibm.com_ibm-spectrum-scale-csi-provisioner-6d55c779bb-xdnws_a9685963-eccd-43d2-a1f6-98b49d636284  failed to provision volume with StorageClass "sctest": rpc error: code = Internal desc = unable to get details of Primary Fileset [spectrum-scale-csi-volume-store]. Error : [Get "https://monica-multi-scalegui-3.fyre.ibm.com:443/scalemgmt/v2/filesystems/fs1/filesets/spectrum-scale-csi-volume-store": x509: certificate signed by unknown authority]
  Normal   ExternalProvisioning  8s (x15 over 3m31s)  persistentvolume-controller                                                                                waiting for a volume to be created, either by external provisioner "spectrumscale.csi.ibm.com" or manually created by system administrator

logs :
I0716 13:07:52.674943       1 rest_v2.go:1454] rest_v2 GetNodeHealthStates, nodeName: monica-multi-worker-2.fyre.ibm.com, component: GPFS
I0716 13:07:52.674950       1 rest_v2.go:999] rest_v2 doHTTP: urlSuffix: scalemgmt/v2/nodes/monica-multi-worker-2.fyre.ibm.com/health/states?filter=state=HEALTHY,entityType=NODE,component=GPFS, method: GET, param: <nil>
I0716 13:07:52.674955       1 rest_v2.go:1001] rest_v2 doHTTP: endpoint: https://monica-multi-scalegui-3.fyre.ibm.com:443/
I0716 13:07:52.674961       1 http_utils.go:58] http_utils HttpExecuteUserAuth. type: GET, url: https://monica-multi-scalegui-3.fyre.ibm.com:443/scalemgmt/v2/nodes/monica-multi-worker-2.fyre.ibm.com/health/states?filter=state=HEALTHY,entityType=NODE,component=GPFS, user: csiadmin
E0716 13:07:52.682886       1 rest_v2.go:1026] rest_v2 doHTTP: Error in authentication request on endpoint https://monica-multi-scalegui-3.fyre.ibm.com:443/: Get "https://monica-multi-scalegui-3.fyre.ibm.com:443/scalemgmt/v2/nodes/monica-multi-worker-2.fyre.ibm.com/health/states?filter=state=HEALTHY,entityType=NODE,component=GPFS": x509: certificate signed by unknown authority
E0716 13:07:52.682907       1 identityserver.go:60] Probe: GPFS component on node monica-multi-worker-2.fyre.ibm.com is not healthy. Error: unable to get health states for nodename monica-multi-worker-2.fyre.ibm.com
I0716 13:07:52.682915       1 utils.go:57] GRPC response: ready:<value:true >

Expected behavior
PVC should be in bound state

Deferral Template
Scrum Master signing off on deferral: Yes
Problem Description in customer terms (what activity is being performed and what problem was encountered): This issue occurs when multiple GUIs are passed in CSI CR along with SSL enabled.
Root cause: GUI HA with SSL enabled is still not handled.
Potential Fix Information: We have 2 options here: 1. Use single certificate for all GUIs or 2. Enable multiple cert inputs in CSI CR.
Frequency of occurrence: This issue is seen when GUI HA is used with SSL enabled for multiple GUIs.
Number of duplicate defects: 0
Does this problem exist with Spectrum Scale container native storage access in the field today (a NO answer means this is NEW BREAKAGE)?: No, this is a new breakage observed only with GUI HA with SSL enabled (but won't be visible in the field as GUI HA won't be used in the filed as it won't documented as delivered in 2.7.0)
In what level of code was this problem introduced?: CSI 2.7.0
Reason for risking defect: GUI HA is not targeted for this release, and it can still be used in SSL disabled mode.
Mitigation in the Field (how can problem be avoided): This can be avoided if SSL is disabled.
Recovery Actions (what should customer do if problem is encountered?): Use CSI in SSL disabled mode.

With using a single certificate for multiple GUIs, this issue shouldn't be seen. @Jainbrt, Monica, please verify and if verified already close this. Thanks!

Sure make sense. Closing now. I will open an enhancement for multi cert support for next release


@page "/contentscript.html" is not working

@page "/contentscript.html"
@inherits BasePage

ContentScript

@code {

}

 public partial class ContentScript
    {

        [Inject] public IJsRuntimeAdapter JsRuntimeAdapter { get; set; }

        protected override async Task OnInitializedAsync()
        {
            Console.WriteLine("dddd");

            await base.OnInitializedAsync();
        }
    }

dddd is not log in console

Have the manifest.json and Program.csbeen updated accordingly?

manifest.json

App.razor

Program.cs

Just one last thing to check, perhaps the ContentScript in App.razor is not referring to the actual component. If you are using Visual Studio you can check easily by hovering the component, it should say fishGo.Pages.ContentScript.
You can try adding @using fishGo.Pages in the App.razor and see if the content scripts is rendered in the site you are expecting.

you are right.


Endless 'TTL exceeded' error log

I have some jobs in failed state due to ttl exceeded, but their zids are still in the jobs:active sorted set, which leads to writing the error log again and again and finally exhausts all redis memory.

There's checkActiveJobTtl which use setInterval to check the active job ttl, see here
The failed job's zid is still in the zset and it's timeouted already
Then job.failedAttempt will be called and it writes a log to {q}:job:id:log
Cause the old state and new state are both failed the zid wouldn't be removed here

My question is how could this inconsistency happen? and any suggestion to remedy it?

There may be a race in job ttl handling,
TTL is dropped in Kue v1 in favor of automatic stalled job handling by Kue.


LinalgRefactor - SGVector - setconst

Generally fine, but pls also apply the changes discussed in the matrix PR here

@vigsterkr ready to merge


Set Thread name for running HTTP server threads

Setting the thread name on the server mode, it can be used to set a unique name for a thread, which can be useful for debugging  multi-threaded applications, if you are starting more than one server on different ports (the threads on each port cab be set by one name) .
.threadsName("MyHTTPServer")
Usage :
std::shared_ptr<Http::Endpoint> mHttpEndpoint_;

auto opts = Http::Endpoint::options()
          .threads(3)
          .threadsName("MyHTTPServer")
      mHttpEndpoint_->init(opts);

Codecov Report

Merging #575 into master will decrease coverage by 0.19%.
The diff coverage is 63.15%.

@@           Coverage Diff            @@
##           master    #575     +/-   ##
========================================
- Coverage   70.99%   70.8%   -0.2%
========================================
  Files          49      49
  Lines        6957    6966      +9
========================================
- Hits         4939    4932      -7
- Misses       2018    2034     +16

Impacted Files
Coverage Δ

include/pistache/listener.h
0% <ø> (ø)
:arrow_up:

include/pistache/endpoint.h
88.88% <ø> (ø)
:arrow_up:

include/pistache/reactor.h
93.33% <100%> (ø)
:arrow_up:

src/server/listener.cc
70.53% <100%> (+0.13%)
:arrow_up:

src/server/endpoint.cc
60.31% <20%> (-4.09%)
:arrow_down:

src/common/reactor.cc
79.51% <70%> (-0.49%)
:arrow_down:

src/client/client.cc
83.39% <0%> (-2.1%)
:arrow_down:

src/common/os.cc
56.49% <0%> (+0.64%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b5b1711...9b82001. Read the comment docs.

i will review your comments, amd fix the issues at weekend.
thanks

Hello Dennis,
I made the changes, you mentioned above about the code style and using std::string instead of char*.
about your last comment

Do not assume that all threading implementations use the same max length thread name. iirc, Windows lets you set a 10-char thread name. IDK about FreeBSD or Solaris. Someday we'll port pistache to an OS beyond Linux. Just use an arbitrary length string.

i dont know how can i handle multi OS support for this PR, i am not familiar with Windows nor Mac, BSD systems.
if you don't specify the length for thread name and this string length is more than 16 chars including '\0'  (in Linux), the pthread_setname_np function will return the error (code:34), it must be have in the specify length.
Thanks.

Dear Dennis,
All requested changes has been done.
Thanks

Ah, almost done.  Please exercise this code (call your new method) in a unit test.  I'm not asking for the test to verify that the thread name was set to the given value (inspecting that from the unit test might be cumbersome), just that the call will not crash.  Please test the following values:

""
"a"
"0123456789abcdef"  // max length string.
"0123456789abcdefghi" // exceed max length string.

Dear Dennis,
i add unit test for initializing the endpoint with EXPECT_NO_THROW.

I tested this PR code in my local machine and it works perfectly, without any exception in different name lengths, but you need the full test case to prove this, lets make the full scenario the same as  http_server_test.cc with setting name in thread with your preferred values.
i will do it in few days.

Dear Dennis,
i create the sever thread with name and client example to send & receive data, this is actual and real test case that show full functionality of HTTP server and client with setting the name in server side app.

Pushed to unstable PPA.


[scale-test] Diagnose exponential growth explicitly.

This, somewhat questionably, fits the polynomial model and the
exponential model, and then chooses the one with the best R^2. However,
no matter how statistically valid this is, it works reasonably in
practice.
Slow growing things sometimes get classified as 1.0^n or 1.1^n, but
these are either spurious or not relevant, and so a similar thresholding
to the polynomial fit is used.

cc @rudkx

@swift-ci Please smoke test and merge.

@swift-ci Please smoke test and merge.

@swift-ci Please smoke test and merge.

@swift-ci Please smoke test


Automatically install/update dependencies

It would be nice if dependencies are automatically installed/updated when installing ancp-bids via pip.

Dependencies are now automatically installed on pip install (--upgrade) ancpbids


Hygieia Product Dash

I have spent the good part of the last two weeks playing around with the Jenkins Publisher.  It has been a bit of a struggle trying to figure out why I couldn't get commits to move from the build phase, to any of my environments, but I think I finally tracked it down.  The issue seems to stem from a recent change made for the dynamic pipeline.  (Sidenote, what is the need for having the standard pipeline, and the dynamic pipeline, and which should I be using?) I am currently using the standard pipeline.  The major problem seems to be when I have changes in the build env, and I run the deploy publisher, it tries to match to artifacts with the same artifact name and extension.  However, a recent change was made to the DeployBuilder which pulls the version and extension out of the artifact name, instead of just the version.  This means that an artifact containing a change set will never match up to the environment, as the extension is always null, stranding the change sets in the build env forever.  Has anyone else encountered this issue?
@MarkRx @tabladrum

Going to be my weekend project :-)

Hopefully I pointed you in the right direction.

Boils down to difference between these two methods:
    public static String determineArtifactName(FilePath file, String version) {
    	String fileName = file.getBaseName();

        if ("".equals(version)) return fileName;

        int vIndex = fileName.indexOf(version);
        if (vIndex <= 0) return fileName;
        if ((fileName.charAt(vIndex - 1) == '-') || (fileName.charAt(vIndex - 1) == '_')) {
            vIndex = vIndex - 1;
        }
        return fileName.substring(0, vIndex);
    }

    public static String getFileNameMinusVersion(FilePath file, String version) {
        String ext = FilenameUtils.getExtension(file.getName());
        if ("".equals(version)) return file.getName();

        int vIndex = file.getName().indexOf(version);
        if (vIndex <= 0) return file.getName();
        if ((file.getName().charAt(vIndex - 1) == '-') || (file.getName().charAt(vIndex - 1) == '_')) {
            vIndex = vIndex - 1;
        }
        return file.getName().substring(0, vIndex) + "." + ext;
    }

The first one does not seem to return the ext of the file. Not sure yet.
@rob-miller-777 Can you quickly change the following code in Deploybuilder  locally and test:
String artifactName = HygieiaUtils.determineArtifactName(f, artifactVersion);

to
String artifactName = HygieiaUtils.getFileNameMinusVersion(f, artifactVersion);

@nireeshT @satishc1 FYI... if you can check too.

Had to do that, in combination with flipping two lines in the EnvironmentComponentEventListener class.  It is currently:
artifactName = artifactName.substring(0, dotIdx);
artifactExtension = artifactName.substring(dotIdx);

Changed it to:
artifactExtension = artifactName.substring(dotIdx + 1);
artifactName = artifactName.substring(0, dotIdx);

The artifact extension was always null because it had already been removed.  I also increased the index to remove the period prefix.

After making those changes, I can get a commit through different environments, but it isn't going away in the previous one.  If you look at this dash here, those all reference the same commit.

Is it the responsibility of the angular app, or the api to filter out the commits per stage?

I'm having a difficult time figuring out where that occurs.

Looks like there was a commit a month ago that removes removes filtering out commits when they are in later stages. I'm assuming this logic would have been moved to the UI, but can't find it.

We are looking into it right now.

The dynamic pipeline is an incomplete POC that I will get back to in the near future. It's purpose is to correlate data on hand dynamically instead of doing so when the data is collected as that has a number of shortcomings https://github.com/capitalone/Hygieia/issues/804#issuecomment-247391585.
        	artifactName = artifactName.substring(0, dotIdx);
        	artifactExtension = artifactName.substring(dotIdx);

Ha good catch. Looks like I wrote that too.
In regards to commits showing up in each stage - actually I think I prefer that. If a commit makes it to A, B, and C I as a user would want to see it in A, B, and C. Otherwise, as a number of people have brought up, some stages will appear to be empty with no data and this is confusing.

HygieiaUtils.determineArtifactName

The logic looks correct. It assumes a format of [[-_]][.ext]. The artifactName corresponds to the maven/ivy concept of an artifact name. If a user passes in the wrong data for artifactName in the BinaryArtifactService the EnvironmentComponentEventListener attempts to compensate for this using that logic that @rob-miller-777 corrected.

@rob-miller-777  I think I found the issue with the stage commits not vanishing in before stages. Working on it now.

As far as the the commits that have moved to a further stage, I'm cool with them being visible in each stage, but maybe have some way to differentiate it. Maybe show a greyed out box for the ones that have already moved on?  Currently, you just see them all as if they are all still in that stage.
The issue with the determine artifact name is that later on, you need the artifact extension, and this method is throwing it away. The logic works correctly to get the file name, just doesn't put the extension back on.

Is DeployBuilder.java missing this (from ArtifactBuilder)?
bac.setArtifactModule(artifactName); // for now assume maven artifact
bac.setArtifactExtension(FilenameUtils.getExtension(f.getName()));

Yeah, the deploy request object doesn't even have an artifact extension property.

See my updated comment

I am adding those.
Commit movement: Working to get it back to where it was - i.e moving them from one stage to another. Then will discuss if there is a better way.

I take that back.
EnvironmentComponent model has a generic componentName.
It is set to ArtifactName. It should contain the extension.
If we incorporate @rob-miller-777 's suggested corrections, we should be good.
@rob-miller-777 let me know if you want to do the PR or want us to do it.

You guys can take care of it, you know more about the pipeline than I do.

@tabladrum for clarity I suggest renaming the deploy service parameters to "componentName" and "componentVersion". The componentName has no semantic meaning - it may or may not contain a version string or an extension depending on how it is set up.

@MarkRx makes sense. Will do it in two separate PR's

@rob-miller-777  - I have made changes for commits movement through stages. Please test this https://github.com/capitalone/Hygieia/pull/1278 and let me know if you see issues.

@rob-miller-777  update to latest master. Reopen if still issue.


[Dep] Update vendor patches files to vendor-patches repo path

I am closing it as already applied.


Resource usage reports and cost

Resource usage reports and cost. Auto-generate after terraform plan command
To understand the cost of infrastructure, we need to explore the possibility of creating a report on the cost of infrastructure. Perhaps it will be a plugin and instructions for use (examples: https://github.com/binbashar/terraform-aws-cost-budget or https://github.com/antonbabenko/terraform-cost-estimation)

another one tool https://github.com/infracost/infracost

another one tool https://github.com/infracost/infracost

Actually, we use it https://github.com/provectus/swiss-army-kube/runs/2784335576?check_suite_focus=true and I have a conversation in issue https://github.com/infracost/infracost/issues/425#issuecomment-871650534


Code in headings breaks On this page text

Hi there, just stumbled over this issue. If I have inline code in a heading, all succeeding text will be truncated.
To reproduce create a file with the contents:
# Foo `bar` baz

The right On this page navigation will have a single item of Foo , missing the rest of the title.

Thanks for reporting this. Let me investigate a bit. I have a suspicion I know what the issue is here.

Are you still seeing this issue? I just tried to reproduce what you described but I'm seeing what I think is the expected output:

Oh - my mistake. You're talking about the right side navigation. I'm seeing the issue too.

Thanks a lot, looks good!