[R] CRAN packaging checklist for version 18.1.0

Describe the enhancement requested
Packaging checklist for CRAN release
For a high-level overview of the release process see the
Apache Arrow Release Management Guide.
cc @jonkeane @thisisnic @nealrichardson @assignUser @paleolimbot @eitsupi
Before the release candidate is cut

[x] Create a GitHub issue entitled [R] CRAN packaging checklist for version X.X.X and copy this checklist to the issue.
[x] Review deprecated functions to advance their deprecation status, including removing preprocessor directives that no longer apply (search for ARROW_VERSION_MAJOR in r/src).
[x] Evaluate the status of any failing nightly tests and nightly packaging builds. These checks replicate most of the checks that CRAN runs, so we need them all to be passing or to understand that the failures may (though won't necessarily) result in a rejection from CRAN.
[x] Check current CRAN check results
[x] Ensure the contents of the README are accurate and up to date.
[x] Run urlchecker::url_check() on the R directory at the release candidate.
commit. Ignore any errors with badges as they will be removed in the CRAN release branch.
[x] Polish NEWS but do not update version numbers (this is done automatically later). You can find commits by, for example, git log --oneline <sha of last release>..HEAD | grep "\[R\]"
[x] Run preliminary reverse dependency checks using archery docker run r-revdepcheck.
[ ] For major releases, prepare tweet thread highlighting new features.

Wait for the release candidate to be cut:
After release candidate has been cut

[x] Create a CRAN-release branch from the release candidate commit, name the new branch e.g. maint-X.X.X-r and push to upstream

Prepare and check the .tar.gz that will be released to CRAN.

[x] git fetch upstream && git checkout release-X.X.X-rcXX && git clean -f -d
[x] Run make build. This copies Arrow C++ into tools/cpp, prunes some
unnecessary components, and runs R CMD build to generate the source tarball.
Because this will install the package, you will need to ensure that the version
of Arrow C++ available to the configure script is the same as the version
that is vendored into the R package (e.g., you may need to unset ARROW_HOME).
[x] devtools::check_built("arrow_X.X.X.tar.gz") locally
[x] Run reverse dependency checks using archery docker run r-revdepcheck.

Release vote

[x] Release vote passed!

Generate R package to submit to CRAN

[x] If the release candidate commit updated, rebase the CRAN release branch
on that commit.
[x] Pick any commits that were made to main since the release commit that
were needed to fix CRAN-related submission issues identified in the above
steps.
[x] Remove badges from README.md
[x] Run urlchecker::url_check() on the R directory
[x] Create a PR entitled WIP: [R] Verify CRAN release-10.0.1-rc0. Add
a comment @github-actions crossbow submit --group r to run all R crossbow
jobs against the CRAN-specific release branch.
[x] Run Rscript tools/update-checksums.R <libarrow version> to download the checksums for the pre-compiled binaries from the ASF artifactory into the tools directory.
[x] Regenerate arrow_X.X.X.tar.gz (i.e., make build)

Check binary Arrow C++ distributions specific to the R package

[x] Upload the .tar.gz to win-builder (r-devel only)
and confirm (with Jon, who will automatically receive an email about the results) that the check is clean.
[x] Upload the .tar.gz to MacBuilder
and confirm that the check is clean
[x] Check install.packages("arrow_X.X.X.tar.gz") on Ubuntu and ensure that the
hosted binaries are used
[x] devtools::check_built("arrow_X.X.X.tar.gz") locally one more time (for luck)

CRAN submission

[x] Upload arrow_X.X.X.tar.gz to the
CRAN submit page
[x] Confirm the submission email

Wait for CRAN...

[x] Accepted!
[ ] Tag the tip of the CRAN-specific release branch with r-universe-release
[ ] Add a new line to the matrix in the backwards compatability job
[ ] (patch releases only) Update the package version in ci/scripts/PKGBUILD, dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb, r/DESCRIPTION, and r/NEWS.md
[ ] (CRAN-only releases) Rebuild news page with pkgdown::build_news() and submit a PR to the asf-site branch of the docs site with the contents of arrow/r/docs/news/index.html replacing the current contents of arrow-site/docs/r/news/index.html
[ ] (CRAN-only releases) Bump the version number in r/pkgdown/assets/versions.json, and update this on the the asf-site branch of the docs site too.
[ ] Update the packaging checklist template to reflect any new realities of the
packaging process.
[ ] Wait for CRAN-hosted binaries on the
CRAN package page to reflect the
new version
[ ] Tweet!

Use Bryce's script for contributor calculation.

Component(s)
R

Awesome, thanks!

There are a few failures over in the verification PR that need investigating (test-r-devdocs, test-r-rhub-debian-gcc-devel-lto-latest, r-recheck-most). I'll plan to work on them later today/tomorrow.

I looked into the two crossbow job failures and neither looked simple to diagnose and fix. @assignUser can you have a look? It's also not clear to me whether these two failures should hold up submitting to CRAN.
The recheck job above fails but it looks like that's fine since the failure is just because the recheck result for a couple of packages changed from 17.x to 18.x.

Strangely, the submission to release reports itself as r-devel-macosx-arm64 whereas the submission to devel reports itself as r-oldrel-macosx-arm64. Is that backwards?

Oh that is very strange! And even stranger the version numbers are the right way around 4.4 for release and 4.5 for devel. And in the first few lines of the check log for devel there is using R Under development (unstable) (2024-04-24 r86482) too. So weird!

Is this a transient mac builder issue?

Yeah, that looks like something is up with the environment there — you could try again and see if it just works?
Ideally r-devel passes mac builder, but this kind of failure looks much more like a macbuilder env issue than an arrow-the-package issue, so I would be ok submitting to CRAN still even if this hasn't passed. Some reasoning behind that: we haven't changed our DESCRIPTION file or dependencies, which is why something like that would change because of us.

I just submitted the tar to CRAN 🤞 Thank you @amoeba for all of your help on this, I couldn't have done it without you

package arrow_18.1.0.tar.gz is on its way to CRAN

And we are already seeing checks roll into https://cran.r-project.org/web/checks/check_results_arrow.html

a. wow css (colors) on cran
b. what's up with the non-informative error output? That looks like testthat.r echoed?

a. wow css (colors) on cran

😂

b. what's up with the non-informative error output? That looks like testthat.r echoed?

I would guess that means something crashed or exited unexpectedly somewhere. Maybe even an oom-kill? Something where stdout isn't informative.
IIRC macOS checks will automatically rerun so let's see if it happens again?

IIRC macOS checks will automatically rerun so let's see if it happens again?

It did not :tada:

Hey all, it looks like the CRAN binaries have been built (with the exception of windows r-devel) so I think we can post to socials. I've started a Google Doc. Due to the small changeset in this release, I decided not to do a long tweet thread with screenshots. However, I could be convinced to hype this up more if anyone feels differently. I'll do a final copy-edit before posting anywhere but please have a look. I'll plan to post to Twitter, Mastodon, and Blue Sky.
I've also submitted a PR to sync the docs if anyone wants to do a quick review.
Once those two things (social posts, website PR) are done, I think we can close this release out.

@jonkeane do you want to have a look at the social media google doc? I'll plan to post regardless but a review would be great.

Thank you for wrangling this. I made a couple of suggestions in the doc. Jon is on vacation now so I'm not sure they will be available to respond (or skeet).

Thanks for the edits on the social media doc, it's much appreciated @nealrichardson. Shoulda pinged you in the first place.
I'll schedule my posts to go out tomorrow.

fyi: https://github.com/apache/arrow/issues/44995

Closing this out as the checklist is complete. Thanks all for the help with the release tasks.


Performance in cygwin

Resty is 2 orders of magnitude slower then POCurl on cygwin. A request that takes 7 millis to get back using curl takes 700 millis with resty (same exact options).
(Will provide traces tomorrow...)

Hey @gv0tch0, I know it's a bit late, but could you provide us your traces?

closing so far because no details provided.


Provide this Wbsite Blocker in windows application using c++ driver

please Provide this Wbsite Blocker in windows application using c++ driver without host and windows source using using only personal firewall rules

please Provide this Wbsite Blocker in windows application using c++ driver without host and windows source using using only personal firewall rules

@AkshayUsha
it is so difficult... :(
sorry


Add alg support for EdDSA

Describe the bug
While implementing an application using these new Kotlin sdks, I was trying to integrate with an issuer, and ran into a cryptographic algorithm incompatibility. The issuer supports EdDSA but the call here does not:
val did = DidKey.create(InMemoryKeyManager(), CreateDidKeyOptions(algorithm = Algorithm("EdDSA")))

To Reproduce:
Steps to reproduce the behavior:
Try to create a did:key using EdDSA and you'll see the exception:
Exception in thread "main" java.lang.IllegalArgumentException: Algorithm EdDSA not supported
	at web5.sdk.crypto.Crypto.getKeyGenerator(Crypto.kt:187)
	at web5.sdk.crypto.Crypto.generatePrivateKey(Crypto.kt:77)
	at web5.sdk.crypto.InMemoryKeyManager.generatePrivateKey(InMemoryKeyManager.kt:42)
	at web5.sdk.crypto.KeyManager$DefaultImpls.generatePrivateKey$default(KeyManager.kt:31)
	at web5.sdk.dids.DidKey$Companion.create(DidKey.kt:89)

Expected behavior:
A clear and concise description of what you expected to happen.
I would expect the algorithm to be supported

Welp, this is a non-issue. Closing.
I needed to include a supporting Curve, such as this...
val did = DidKey.create(InMemoryKeyManager(), CreateDidKeyOptions(algorithm = Algorithm("EdDSA"), curve = Curve.Ed25519))

you're running into this issue because EdDSA isn't specific enough. algorithms can have many curves and in this case EdDSA doesn't indicate 1 specific curve. doing the following should work:
val manager = InMemoryKeyManager()
val opts = CreateDidKeyOptions(algorithm = JWSAlgorithm.EdDSA, curve = Curve.Ed25519)
val did = DidKey.create(manager)


Warning message: UserWarning: Group indices computation defaulting to pandas implementation. on simple single column groupby

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): AmazonLinux
Modin version (modin.__version__): 0.14.1
Python version: 3.8
Code we can use to reproduce:

partition_cols = ["year"]
for keys, subgroup in df.groupby(partition_cols, observed=True):
        subgroup = subgroup.drop(partition_cols, axis="columns")
        keys = (keys,) if not isinstance(keys, tuple) else keys
        ...

Describe the problem

I have a modin dataframe with 10 columns and 18M rows. One of the columns contains "year" values ranging from 1900 to 2020. Running the above code where I do a simple groupby on the year column throws this warning:
UserWarning: Group indices computation defaulting to pandas implementation.

Why would a simply groupby default to pandas? Note that removing the observed=True parameter does not change the warning message.
Perhaps this is similar to this issue (https://github.com/modin-project/modin/issues/1114) where the message is incorrectly displayed?
Source code / logs

Hi @jaidisido , thanks for posting!
In this case, the groupby itself is not the problem, but the for loop. Looping over groups destroys the parallelism because Modin is unable to parallelism regular Python for loops. Is there a way to avoid looping in this case? It is difficult to see from the code you've provided, but happy to help!

Thanks for the clarification @devin-petersohn, the full code that I am trying to parallelise is here. I am mostly interested in looping over the keys, not so much the subgroups. Is there a way to obtain those via the groupby without resorting to a for loop? Thanks!

Hi @jaidisido! Thank you for opening up this issue! I went ahead and looked at the code you linked above, and it seems pretty optimal as far as this computation pattern goes.
Given that the code seems to only require the groupby keys and the rows that correspond to them, I tried another approach - computing the unique values over the partition column (since you mention there's only one groupby key in the issue description) and then looping over those, and computing the dataframe "subgroups" using indexing.
For both this approach, and the approach in your code, with the following benchmarking code, I found that Modin matched pandas performance. This is on a 10 x 100,000 and 10 x 1,000,000 dataframe on my 2019 Intel Mac! I included a 2 second sleep in each loop to simulate the work being performed in your code.
Performance Benchmark
100K Rows:

Modin
pandas

Unique Column Values Loop
20.62s
20.09s

Groupby Groups Loop
20.21s
20.08s

1M Rows:

Modin
pandas

Unique Column Values Loop
21.57s
20.54s

Groupby Groups Loop
21.41s
20.55s

100,000 row df on Modin Code:
import modin.pandas as pd
import numpy as np
from time import sleep
shape = (9, 100_000) # We use 9 here since we will add the groupby column (years) in the below line
arr = np.vstack((np.random.randint(1, 1000, shape), np.random.choice(range(1990, 2040, 20), shape[1])))
df = pd.DataFrame(arr)
def col_unique_loop():
    for k in df[9].unique():
        subgroup = df[df[9] == k].drop(9, axis="columns")
        k = (k,)
        sleep(2) # Simulate work
def groupby_loop():
    for k, subgroup in df.groupby(9, observed=True):
        subgroup = subgroup.drop(9, axis="columns")
        k = (k,) if not isinstance(k, tuple) else k
        sleep(2) # Simulate work
from datetime import datetime, timedelta
start_unique = datetime.now()
col_unique_loop()
end_unique = datetime.now()
start_groupby = datetime.now()
groupby_loop()
end_groupby = datetime.now()
print(f"Looping over unique column values took {(end_unique - start_unique) / timedelta(seconds=1)} s")
print(f"Looping over groupby groups took {(end_groupby - start_groupby) / timedelta(seconds=1)} s")

To test on 1,000,000 million rows, or on pandas, simple change the import statement or the statement that defines shape!

Thank you so much @RehanSD, I was able to reproduce this. A couple of follow-up questions if I may before I close the issue:

Above tests were limited to a single groupby column. Would the performance significantly change if I was to groupby multiple columns?
I am trying to understand the impact on memory of running a groupby in Modin (Ray engine). Are new Ray blocks created as a result of a groupby? If yes, wouldn't that be duplicating memory on the cluster? If not, I am curious to hear what kind of magic Modin does behind the scene to avoid that :)

Hi @jaidisido

I don't believe performance should be changed.
In general, Modin doesn't do operations in place, but looping over a groupby as you're doing above is actually implemented as a series of filter, which is zero copy!


Unexpected token ...

Whatever I am doing, I always get the following error message (using node 4.4.5):
/Users/xxx/.nvm/versions/node/v4.4.5/lib/node_modules/generate/lib/utils.js:31
      .apply(console, [utils.log[color](msg), ...rest]);
                                              ^^^

SyntaxError: Unexpected token ...
    at exports.runInThisContext (vm.js:53:16)
    at Module._compile (module.js:373:25)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)

I installed generate globally, did the same for generate-node, then did a $ gen node:mit ...
Any ideas, am I missing something?
Regards
Stefan

you need to update node.js to a more recent version. 5 or greater. It's surprising you're only seeing this error now, since a lot of libs in the dependency tree use es2015 syntax

OK; will give it a try ... then you should update the package.json, since it is saying:
"engines": {
    "node": ">=4.0"
  },

k, thx

closing, created https://github.com/generate/generate/issues/12


allow scratch space to be an aribtirary directory

It would be useful to be able to run PerfKitBenchmarker on VMs/containers/physical-servers where there isn't a dedicated mount point for the scratch space, or where creating a mountpoint per service is not supported.
Currenlty trying to run fio with "scratch_disk_mountpoints": "/tmp/google-pkb" results in```
Traceback (most recent call last):
File "./pkb.py", line 21, in
sys.exit(Main())
File "/home/csb/prog/github/PerfKitBenchmarker/perfkitbenchmarker/pkb.py", line 358, in Main
return RunBenchmarks()
File "/home/csb/prog/github/PerfKitBenchmarker/perfkitbenchmarker/pkb.py", line 324, in RunBenchmarks
RunBenchmark(benchmark, collector)
File "/home/csb/prog/github/PerfKitBenchmarker/perfkitbenchmarker/pkb.py", line 229, in RunBenchmark
benchmark.Prepare(benchmark_specification)
File "/home/csb/prog/github/PerfKitBenchmarker/perfkitbenchmarker/benchmarks/fio_benchmark.py", line 65, in Prepare
disk_size_kb = vm.GetDeviceSizeFromPath(vm.GetScratchDir())
File "/home/csb/prog/github/PerfKitBenchmarker/perfkitbenchmarker/virtual_machine.py", line 405, in GetDeviceSizeFromPath
return int(stdout)
ValueError: invalid literal for int() with base 10: '210411904 16419404 '

While omitting results in the only marginally more helpful error message:

Traceback (most recent call last):
File "./pkb.py", line 21, in
sys.exit(Main())
File "/home/csb/prog/github/PerfKitBenchmarker/perfkitbenchmarker/pkb.py", line 358, in Main
return RunBenchmarks()
File "/home/csb/prog/github/PerfKitBenchmarker/perfkitbenchmarker/pkb.py", line 324, in RunBenchmarks
RunBenchmark(benchmark, collector)
File "/home/csb/prog/github/PerfKitBenchmarker/perfkitbenchmarker/pkb.py", line 229, in RunBenchmark
benchmark.Prepare(benchmark_specification)
File "/home/csb/prog/github/PerfKitBenchmarker/perfkitbenchmarker/benchmarks/fio_benchmark.py", line 65, in Prepare
disk_size_kb = vm.GetDeviceSizeFromPath(vm.GetScratchDir())
File "/home/csb/prog/github/PerfKitBenchmarker/perfkitbenchmarker/static_virtual_machine.py", line 91, in GetScratchDir
scratch_dir = self.scratch_disk_mountpoints[disk_num]
IndexError: list index out of range

Sounds like a good flag option to have.  Marking it as a feature and needing help.  If someone wants to pick it up we can assign them and proceed on how best to do it.

Connor - Did I see you fix this?

No, still open.  I fixed a hard-coded path in the fio benchmark.

This is done, but the documentation was not clear and no error was thrown if the user provides a string to static_disk_mountpoints. #166 addresses the documentation error and adds an error on an invalid static VM  file.

Documentation updated by #166. Closing.


WIP (do not merge) - add a FAQ page and update the docs to remove ixmp as submodule

This PR adds an FAQ and cleans up the documentation pages to reflect that ixmp is not included as a submodule any more.

lgtm, still a WIP?

Ok, also see your email now..

please review whenever you have a minute, @gidden

lgtm!


Parsing error

Hi Willi,
You will find below the traceback i met with the last release.
Scenario : Windows 2008
traceback evtxtract :
evtxtract raw_image.001 > evtxcarving.xml
INFO:evtxtract.carvers:Unknown exception processing record at 0xC74052200
Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/evtxtract/carvers.py", line 174, in extract_chunk_records
    record_xml = Evtx.Views.evtx_record_xml_view(record, cache=cache)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 186, in evtx_record_xml_view
    return _build_record_xml(record, cache=cache)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 169, in _build_record_xml
    xml = rec(record.root())
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 160, in rec
    subs_strs.append(rec(sub))
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 156, in rec
    for sub in root_node.fast_substitutions():
  File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 925, in fast_substitutions
    ofs = self.tag_and_children_length()
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 900, in tag_and_children_length
    for child in self.children():
  File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 890, in children
    return self._children(end_tokens=[SYSTEM_TOKENS.EndOfStreamToken])
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 130, in _children
    ofs += child.length()
  File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 148, in length
    for child in self.children():
  File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 305, in children
    SYSTEM_TOKENS.CloseEmptyElementToken])
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 130, in _children
    ofs += child.length()
  File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 148, in length
    for child in self.children():
  File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 305, in children
    SYSTEM_TOKENS.CloseEmptyElementToken])
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 124, in _children
    self._chunk, self)
TypeError: 'NoneType' object is not callable
INFO:evtxtract.carvers:Unicode decoding issue processing record at 0xC740533E0
INFO:evtxtract.carvers:Unicode decoding issue processing record at 0xC74053808
INFO:evtxtract.carvers:Unknown exception processing record at 0xC74053C80
Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/evtxtract/carvers.py", line 174, in extract_chunk_records
    record_xml = Evtx.Views.evtx_record_xml_view(record, cache=cache)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 186, in evtx_record_xml_view
    return _build_record_xml(record, cache=cache)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 169, in _build_record_xml
    xml = rec(record.root())
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 160, in rec
    subs_strs.append(rec(sub))
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 154, in rec
    f = _make_template_xml_view(root_node, cache=cache)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 129, in _make_template_xml_view
    for c in node.children():
  File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 139, in children
    return self._children()
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 130, in _children
    ofs += child.length()
  File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 148, in length
    for child in self.children():
  File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 305, in children
    SYSTEM_TOKENS.CloseEmptyElementToken])
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 130, in _children
    ofs += child.length()
  File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 148, in length
    for child in self.children():
  File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 305, in children
    SYSTEM_TOKENS.CloseEmptyElementToken])
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 130, in _children
    ofs += child.length()
  File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 148, in length
    for child in self.children():
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 477, in children
    self._chunk, self, self.type())
  File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 439, in get_variant_value
    TypeClass = types[type_]
KeyError: 194

Thanks

hi @nwf9
thanks for reporting these tracebacks. would you confirm that these were not fatal errors that killed the carver? i think these are dangerous-looking messages, yet are simply there to let you know some records were unrecoverable (corrupt, etc). i should probably make this more clear.

@nwf9
thanks for the quick reply.
evtxtract does at least two passes of the entire image while carving event log entries. you can probably expect to see a bunch of complete entries extracted during the first run, and a slower rate of incomplete entries during the second run. it often takes quite a while to scan a large image (imagine the time it takes to copy 500GB, twice!).

Thanks to you for all your discovery. In my case i test only in 60GB. It's currently underway.

Unfortunately the process kill itself
evtxtract raw_disk.001 > evtxcarve.xml
INFO:evtxtract.carvers:Unknown exception processing record at 0xC74052200
Traceback (most recent call last):
File "build/bdist.linux-x86_64/egg/evtxtract/carvers.py", line 174, in extract_chunk_records
record_xml = Evtx.Views.evtx_record_xml_view(record, cache=cache)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 186, in evtx_record_xml_view
return _build_record_xml(record, cache=cache)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 169, in _build_record_xml
xml = rec(record.root())
File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 160, in rec
subs_strs.append(rec(sub))
File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 156, in rec
for sub in root_node.fast_substitutions():
File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 925, in fast_substitutions
ofs = self.tag_and_children_length()
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 900, in tag_and_children_length
for child in self.children():
File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 890, in children
return self._children(end_tokens=[SYSTEM_TOKENS.EndOfStreamToken])
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 130, in _children
ofs += child.length()
File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 148, in length
for child in self.children():
File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 305, in children
SYSTEM_TOKENS.CloseEmptyElementToken])
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 130, in _children
ofs += child.length()
File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 148, in length
for child in self.children():
File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 305, in children
SYSTEM_TOKENS.CloseEmptyElementToken])
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 124, in _children
self._chunk, self)
TypeError: 'NoneType' object is not callable
INFO:evtxtract.carvers:Unicode decoding issue processing record at 0xC740533E0
INFO:evtxtract.carvers:Unicode decoding issue processing record at 0xC74053808
INFO:evtxtract.carvers:Unknown exception processing record at 0xC74053C80
Traceback (most recent call last):
File "build/bdist.linux-x86_64/egg/evtxtract/carvers.py", line 174, in extract_chunk_records
record_xml = Evtx.Views.evtx_record_xml_view(record, cache=cache)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 186, in evtx_record_xml_view
return _build_record_xml(record, cache=cache)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 169, in _build_record_xml
xml = rec(record.root())
File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 160, in rec
subs_strs.append(rec(sub))
File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 154, in rec
f = _make_template_xml_view(root_node, cache=cache)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Views.py", line 129, in _make_template_xml_view
for c in node.children():
File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 139, in children
return self._children()
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 130, in _children
ofs += child.length()
File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 148, in length
for child in self.children():
File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 305, in children
SYSTEM_TOKENS.CloseEmptyElementToken])
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 130, in _children
ofs += child.length()
File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 148, in length
for child in self.children():
File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 305, in children
SYSTEM_TOKENS.CloseEmptyElementToken])
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 130, in _children
ofs += child.length()
File "/usr/local/lib/python2.7/dist-packages/Evtx/BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 148, in length
for child in self.children():
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 477, in children
self.chunk, self, self.type())
File "/usr/local/lib/python2.7/dist-packages/Evtx/Nodes.py", line 439, in get_variant_value
TypeClass = types[type]
KeyError: 194
Killed

@nwf9
when you see killed, this typically means that the operating system has killed the process, often because it is using too many resources. this stackoverflow post is related: http://stackoverflow.com/questions/19189522/what-does-killed-mean
do you have a limited amount of memory available on your analysis system?

I have 32 Gigagbytes and more than 24 CPU core

ok, thats probably not the issue :-)
this is going to be a little tricky to debug without lots of roundtrips. i will brainstorm on that front, and also go through the code with this issue in mind.
in the meantime, a solution that may work for you is to split the image into smaller fragments, and run evtxtract against each fragment. since evtxtract is a carver, the results should be similar as running a single time against the entire image.
thanks for you patience with this issue, and for responding quickly to my questions!

you can emulate the fragmentation technique without copying the data by using losetup. the flags --offset and --sizelimit can be used to expose a device that points to a sub-section of an existing file.
something like:
losetup --offset=$((0 * 1024 * 1024 * 1024 )) --sizelimit=$((1024 * 1024 * 1024 )) /dev/loop0 raw_disk.001
losetup --offset=$((1 * 1024 * 1024 * 1024 )) --sizelimit=$((1024 * 1024 * 1024 )) /dev/loop1 raw_disk.001
...

I will test only the evtx into the unallocated cluster

I can confirm that it works better on the unallocated cluster image because i was recover more than 110MB of event while i recover only 510K with a disk image.

Willi do you need some other log to debug ?


UndefinedFunctionError) function Poison.encode!/1 is undefined (module Poison is not available)

GenServer Mixpanel.Client terminating
2022-08-11T22:41:32.879434747Z ** (UndefinedFunctionError) function Poison.encode!/1 is undefined (module Poison is not available)

Seeing this in prod environments but not dev. The mix.lock file lists poison:
  "poison": {:hex, :poison, "3.1.0", "d9eb636610e096f86f25d9a46f35a9facac35609a7591b3be3326e99a0484665", [:mix], [], "hexpm", "fec8660eb7733ee4117b85f55799fd3833eb769a6df71ccf8903e8dc5447cfce"},

I'm using application inference and there's nothing explicitly set for applications in mix.exs.
Any ideas?

I don't know for sure why this is happening but it seems bumping all dependencies to their modern versions solves the issue. @markquezada stay tuned, will release a new version of the package soon.


Add delimiters for TOML

Fork notice: With the upstream project sitting stale after nearly 8 months since the last activity, 71 outstanding pull requests, no comments from the original developer, and dozens of developers duplicating efforts while scratching their own itches, I've decided to step in and run a fork for a while. Hopefully the original developer will eventually consider picking this plugin up again or adding more maintainers so that the user base doesn't end up fragmented, but for now I would invite anyone interested in the latest updates to consider installing from and contributing pull requests to my fork.
About this PR: I have merged this PR as is. Please consider installing the plugin from alerque/nerdcommenter for this and (so far) over two dozen other updates.

👏

Change of plans. Thanks to scrooloose who has added me as a maintainer to this repo, I'll be going through and taking care of business here instead of in a fork. This PR has been merged directly ustream.
Thanks for your contribution @kballard.

Even better!


Add Dockerfile for TAS installer

This pr adds a Dockerfile for the tas-installer binary,
Usage

Build the image
In order for the installer to work without any extras it needs two things right now, your kubeconfig and the pull secret, to run the image you need to mount both inside the container like so:

podman run -it -v <path-to-kube-config>:/root/.kube/config -v <path-to-pull-secret>:/opt/app-root/src/pull-secret.json <image> install

Then during the install when prompted for the path to the pull secret put: pull-secret.json
If you want to specify any other things like a different values file a similar approach must be followed.
Once a more declarative approach is applied to the helm charts, we can mitigate some of this but we will always need to specify the kubeconfig to communicate with the cluster.

Closing this as the installer has been deprioritized


Feat/auth system

Add the authentication system back using streamlit auth

PR update:

Initial tests commit 41d55f8933e245186af62dc2b1e065fd1cc08421
Setup with this PR + docker compose (BE and broker) using development image, ok
Setup with this PR + BE source development branch + docker compose broker ok
By default, AUTH_SYSTEM_ENABLED is false. Can only be modified via source
Bot orchestration is hidden when user is not logged in ✅
User can still run backtesting ok and controller saved on ./backend-api/bots/conf/controller/
After admin login:

Check deploy, all controllers created when not login yet are available

Refresh page or press c

Automatically logout from the dashboard portal
When on bot orchestration page:

User still able to access all pages, create and deploy config and instances , recording here
User will not be able to access this page when going back to hummingbot dashboard , recording here

Pending:

run test on new commit bcac0366eb4b43a6649f7d40b7883400311356a3

256e193

Cloned PRdashboard157
By default, AUTH_SYSTEM_ENABLED is false. Changed to True
Manually built docker image with  AUTH_SYSTEM_ENABLED True
used deploy to start dashboard (docker)
go to localhost:8501

Page refresh will lead to user Logged out

After you added connector you need to refresh the page to reflect the changes, however it will lead to user logged out

Minor

Actual: Logout button available only for Humingbot Dashboard tab on side pane (if you are on other tab you could not log out)
Expected:
Logout on other tabs


HTML

HTML
User Story
As a bootcamp student
I want the prework notes to be structured on a webpage
so that I can easily find and read information
Acceptance Criteria
GIVEN a prework Study Guide website
WHEN I visit the webpage in my browser
THEN I see four boxes titled HTML, CSS, Git, and Javascript with associated notes listed

Successfully merged the pull request and updated the main branch with feature/add-HTML


suggestion: move to unocss

I have no idea why we're using twind. It's abandoned and outdated. We should move to unocss instead (which will provide the same utility classes anyways!)

done


[MB-7547] add EDI_ERROR to support API

Description
Adding EDI_ERROR status to the support API. It was introduced in PR https://github.com/transcom/mymove/pull/6089 but adding to the support API was missed.
Reviewer Notes
n/a
Setup
This is the effect support API. Run locally or in staging. There is nothing really see happening right now. The connections to GEX are not working.
filename: sendtosyncada.json
{
  "body": {
    "sendToSyncada": true,
    "readFromSyncada": false,
    "deleteFromSyncada": false
  }
}

prime-api-client --insecure support-reviewed-payment-requests --filename sendtosyncada.json

Code Review Verification Steps

[ ] If the change is risky, it has been tested in experimental before merging.
[ ] Code follows the guidelines for Logging
[ ] The requirements listed in Querying the Database Safely have been satisfied.
Any new migrations/schema changes:

[ ] Follow our guidelines for zero-downtime deploys (see Zero-Downtime Deploys)
[ ] Have been communicated to #g-database
[ ] Secure migrations have been tested following the instructions in our docs

[ ] There are no aXe warnings for UI.
[ ] This works in Supported Browsers and their phone views (Chrome, Firefox, IE, Edge).
[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] User facing changes have been reviewed by design.
[ ] Request review from a member of a different team.
[ ] Have the Jira acceptance criteria been met for this change?

References

Jira story for this change

Screenshots
n/a

Messages

:book:
:link: MB-7547

  Generated by :no_entry_sign: dangerJS against 98771f53addd4715a887e0265cfa54ac920d3c6f

File decode failed: json: unknown field "deleteFromSyncada"

Did you rebuild your cli? I have an alias, whenever I pull a branch I rebuild the cmd tools

File decode failed: json: unknown field "deleteFromSyncada"

Did you rebuild your cli? I have an alias, whenever I pull a branch I rebuild the cmd tools

I used the alias you sent over which resolved the initial issue but then started seeing this error (which also appears on master) so I think there's something up with my local set up
{"detail":"function ProcessReviewedPaymentRequest failed call to generator.Generate: id: 85c5ae40-b915-41dd-a970-722bd70c7c19 not found for paymentServiceItemParam","instance":"5eca5e47-c4e3-4cc1-b2be-02ff4ffd9b4c","title":"Internal Server Error"}


If the send side has closed via quicly_sendstate_shutdown, the assert will trigger. Handle that case.

Fixed by #426.

Fixed by #426.


Graphlient::Errors::TimeoutError does not inherit from Graphlient::Errors::ServerError

Describe the bug
Graphlient::Errors::TimeoutError does not inherit from Graphlient::Errors::ServerError currently, hence inner_exception, status_code and response can't be called.
To Reproduce
N/A
Expected behavior
As per description, Graphlient::Errors::TimeoutError should inherit from Graphlient::Errors::ServerError.
Screenshots

According to source code, TimeoutError inherits from base Error currently

This is by design I believe. A timeout is a client-side timeout, and you never got a response from the server. I'll close, but feel free to convince me I'm wrong and we can reopen.

This is by design I believe. A timeout is a client-side timeout, and you never got a response from the server (so no response or status code to be had). I'll close, but feel free to convince me I'm wrong and we can reopen.

Hi Daniel @dblock ,
If this is by design, can we please update the description? It is confusing to have the doc says otherwise.

For sure @PhucNguyen1501! Care to PR?


Remove Lovelace term

Maybe some explanation would be nice?

Just a random out of the blue PR without explanation will not be accepted


"unknown command wsserver" when trying to connect to websocket

Hey Eric! Thanks for this, its really cool!
What version of Bedrock are you working with?
I deployed bedrock to k8s using latest https://github.com/itzg/docker-minecraft-bedrock-server and am able to connect and have cheats enabled and my user is an operator, but when I try to run the connect command I get "unknown command wsserver, please check command exists and that you have permissions to access it"
kubecraftadmin pod is deployed and listening fine...
Found similar stories: https://bugs.mojang.com/browse/BDS-1206
Any chance you ran into this and are using a particular version of bedrock server?

Hey Matthew, I have to admit I haven't tested it with any dedicated server. Would be interesting to see if it works with Realms. The only way I've run it so far is by creating a world locally. Not that this (should) still allow you to do multiplayer, something I haven't tested yet either but that could be fun 👍
In the end if the server has no support for /wsserver or /connect commands, my project cannot work :(

Just to ensure I mimic your set up, when you say locally, what are you running on?

Ah, I progressed a bit further. I logged into my windows 10 minecraft client and created a world locally, with cheats enabled.
It now allows me to use the connect command ( conect <IP_ADDRESS>:30123/ws) and I am able to connect to the kubecradftadmin server, but I don't receive the kubecraftadmin splash...I just see that I connected and have been given diamond gear and tnt.
I try to run the init command and the connection dumps:
kubectl -n minecraft logs -f kubecraftadmin-5577ff8f78-jdjgj
Listening
Player has entered!
init
Player has disconnected
2020/10/09 12:12:36 http: panic serving <IP_ADDRESS>:50679: runtime error: invalid memory address or nil pointer dereference
goroutine 61 [running]:
net/http.(*conn).serve.func1(0xc000140280)
	/usr/local/go/src/net/http/server.go:1801 +0x147
panic(0x14607c0, 0x202f020)
	/usr/local/go/src/runtime/panic.go:975 +0x3e9
github.com/sandertv/mcwss.(*Player).Position.func1(0xc00029a040)
	/go/src/github.com/sandertv/mcwss/player.go:99 +0x2b
reflect.Value.call(0x13f2dc0, 0xc0000ce770, 0x13, 0x15e6cfb, 0x4, 0xc0006e97e8, 0x1, 0x1, 0xc00013c200, 0xc0002f4010, ...)
	/usr/local/go/src/reflect/value.go:475 +0x8c7
reflect.Value.Call(0x13f2dc0, 0xc0000ce770, 0x13, 0xc0006e97e8, 0x1, 0x1, 0x0, 0xc000342001, 0x34)
	/usr/local/go/src/reflect/value.go:336 +0xb9
github.com/sandertv/mcwss.(*Player).handleIncomingPacket(0xc00000a1e0, 0xc0002c4000, 0x24, 0xc000280c40, 0xf, 0x1, 0x13ba120, 0xc00012e200, 0x0, 0x0)
	/go/src/github.com/sandertv/mcwss/player.go:511 +0x3a5
github.com/sandertv/mcwss.(*Server).handleResponse(0xc0001562c0, 0x179f520, 0xc0000261c0, 0xc000294000)
	/go/src/github.com/sandertv/mcwss/server.go:153 +0x405
net/http.HandlerFunc.ServeHTTP(0xc0001f77e0, 0x179f520, 0xc0000261c0, 0xc000294000)
	/usr/local/go/src/net/http/server.go:2042 +0x44
net/http.(*ServeMux).ServeHTTP(0x204c540, 0x179f520, 0xc0000261c0, 0xc000294000)
	/usr/local/go/src/net/http/server.go:2417 +0x1ad
net/http.serverHandler.ServeHTTP(0xc0000260e0, 0x179f520, 0xc0000261c0, 0xc000294000)
	/usr/local/go/src/net/http/server.go:2843 +0xa3
net/http.(*conn).serve(0xc000140280, 0x17a2020, 0xc0002da000)
	/usr/local/go/src/net/http/server.go:1925 +0x8ad
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2969 +0x36c

will go back and see if I missed a step in deploying the kubecraftadmin container? any debug steps I can take?
I noticed in the youtube video you typed /connect <IP_ADDRESS>/kubecraftadmin to connect, but that doesn't work, for me.

Hi Matthew, sorry to hear you're having so many issues. Yes a couple of points:

In the video I'm connecting to /kubecraftadmin, but that has been indeed changed to /ws
My test setup is running kubecraftadmin in a docker under WSL2 on Win10
The Kubernetes cluster is an AKS cluster running on Azure
Minecraft runs locally on my laptop and connects to the WSL2 network through the bridge NIC

I am honestly not sure what is going wrong in your case, it does seem to be related to the Minecraft -> KubeCraftAdmin connection level. Could you try running it in plain Docker and see how that goes?

Thanks so much for the prompt replies!
No worries, I went 0-100 real quick in trying this on my microk8s cluster (which is containerd based) so that may be some of the issue.
I'll get my windows box set up with WSL and get docker rocking!
I have an azure trial too! will see how it goes!

no luck, unfortunately!
Now running Windows 10 Pro, WSL2 Ubuntu, docker. AKS.
kubecraftadmin ontainer comes up and is reachable,
Create a world locally, use /connect
I connect to the container over WSL IP but don't get a splash and init does the same crash above.
Will grab the minecraft versions and such. will keep poking...

Just to double check, did you turn on cheats in the world? I'm trying to understand what else could be the issue....

Here's a more a thourough rundown in case i missed something:
Minecraft 1.16.40 - signed in with my xbox account - are you signed in to a live account?
Ubuntu 20.04.1 LTS - I installed docker manually and ran:
docker run -p 8000:8000 -v /home/mattymo/.kube:/.kube -e namespaces=default,kube-system erjadi/kubecraftadmin

I missed the note on 4 namespace max the first time around, made sure to add this time....
NAMES
6945e4a3934d        erjadi/kubecraftadmin   "/app/main"         4 seconds ago       Up 3 seconds        <IP_ADDRESS>:8000->8000/tcp   elated_satoshi

container comes up fine...
docker logs -f 6945e4a3934d
Listening on port 8000

Anything I can do to exec into the container and check that its all good? Here's the docker inpect:
    {
        "Id": "6945e4a3934d0a11d29fc1d385e52a8d7fcf09f804088f6554c3509e6e78f37e",
        "Created": "2020-10-15T11:58:41.8286897Z",
        "Path": "/app/main",
        "Args": [],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 495,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2020-10-15T11:58:42.07885Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:d22605a56c64ce052374c33558941f50f4388451210d71d8b1f47bb9693afc7e",
        "ResolvConfPath": "/var/lib/docker/containers/6945e4a3934d0a11d29fc1d385e52a8d7fcf09f804088f6554c3509e6e78f37e/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/6945e4a3934d0a11d29fc1d385e52a8d7fcf09f804088f6554c3509e6e78f37e/hostname",
        "HostsPath": "/var/lib/docker/containers/6945e4a3934d0a11d29fc1d385e52a8d7fcf09f804088f6554c3509e6e78f37e/hosts",
        "LogPath": "/var/lib/docker/containers/6945e4a3934d0a11d29fc1d385e52a8d7fcf09f804088f6554c3509e6e78f37e/6945e4a3934d0a11d29fc1d385e52a8d7fcf09f804088f6554c3509e6e78f37e-json.log",
        "Name": "/elated_satoshi",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/home/mattymo/.kube:/.kube"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {
                "8000/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "8000"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Capabilities": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/888871b66089c4abc5ded5e82b5f5733d8c4e611faf303c5ebf12deb74e61fed-init/diff:/var/lib/docker/overlay2/cdd8918ed192d99ebfd300a6c683629171ea2450501da1cabcc35826e0ca7d33/diff:/var/lib/docker/overlay2/a07c9c859542deecb9c4a65376cbf1e3ce8b78b6975545d8d9b7158b51b4983a/diff:/var/lib/docker/overlay2/916df636318a2ed82eb746fae1004a1d98bd16f54e54411582e551b21accefce/diff:/var/lib/docker/overlay2/b00b467c49305c6f9f3a85170c0236b4e6ac81171388c60c6eb662bf85b8b334/diff",
                "MergedDir": "/var/lib/docker/overlay2/888871b66089c4abc5ded5e82b5f5733d8c4e611faf303c5ebf12deb74e61fed/merged",
                "UpperDir": "/var/lib/docker/overlay2/888871b66089c4abc5ded5e82b5f5733d8c4e611faf303c5ebf12deb74e61fed/diff",
                "WorkDir": "/var/lib/docker/overlay2/888871b66089c4abc5ded5e82b5f5733d8c4e611faf303c5ebf12deb74e61fed/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/home/mattymo/.kube",
                "Destination": "/.kube",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            "Hostname": "6945e4a3934d",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "8000/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "namespaces=default,kube-system",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/app/main"
            ],
            "Image": "erjadi/kubecraftadmin",
            "Volumes": null,
            "WorkingDir": "/app",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "com.azure.dev.image.build.buildnumber": "20201011.1",
                "com.azure.dev.image.build.builduri": "vstfs:///Build/Build/332",
                "com.azure.dev.image.build.definitionname": "erjadi.kubecraftadmin",
                "com.azure.dev.image.build.repository.name": "erjadi/kubecraftadmin",
                "com.azure.dev.image.build.repository.uri": "https://github.com/erjadi/kubecraftadmin",
                "com.azure.dev.image.build.sourcebranchname": "main",
                "com.azure.dev.image.build.sourceversion": "2fe9f37f91f8ed4f872f99b922fa86f3408d9449",
                "com.azure.dev.image.system.teamfoundationcollectionuri": "https://dev.azure.com/ericjadi/",
                "com.azure.dev.image.system.teamproject": "KubeCraftAdmin - Pipelines"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "de4c3983817b201d3aaa28353c2e32f82baf116fb16415af4d310bd117f3ec9c",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "8000/tcp": [
                    {
                        "HostIp": "<IP_ADDRESS>",
                        "HostPort": "8000"
                    }
                ]
            },
            "SandboxKey": "/var/run/docker/netns/de4c3983817b",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "e835b01c4e9e697778c21ad6f1efd6786e0f4f99af4b2a4745910ec2d8a8fb0f",
            "Gateway": "<IP_ADDRESS>",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "<IP_ADDRESS>",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:02",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "9d5c51e8292415ce2c37367c351d79fc3cb6c82448d6eb4614098ea318567073",
                    "EndpointID": "e835b01c4e9e697778c21ad6f1efd6786e0f4f99af4b2a4745910ec2d8a8fb0f",
                    "Gateway": "<IP_ADDRESS>",
                    "IPAddress": "<IP_ADDRESS>",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02",
                    "DriverOpts": null
                }
            }
        }
    }
]

Cheats are activated, create a new world in survival....connect to websocket ok

Closed stale issue


how to configure vuejs in this project please

Question
Answer

Issue or Enhancement
Issue

Laravel Version
Used version of Laravel

Project Version
Used version of this project

Current Behavior

Expected Behavior

Steps to Reproduce

Possible Solutions

have you check the issue #423 ?

Closed, no new response.


Sparse state preparation via alias sampling

fixes #1059

@tanujkhattar PTAL

Symbolic costs for sparsity $d$ out of $L$ and precision $\epsilon$:

(from the _sparse_state_prep_alias_symb bloq example)

@tanujkhattar ptal


solver parameter set？

I just want to ask how to control the test interval and snapshot in the training process?

use callbacks in mx.callbacks

you can read the python code ‘train_model.py’ in example

from mx.callbacks, i can only use batch_end_callback or epoch_end_callback, but i want a paramter like iter_end_callback in the mxxnet document.

we renamed iter to batch (I think)

so i think i need to develop a function by myself that i can specify the iterval to test or snapshot

I think simple use checkpoint as batch_end_callback should work


[FR]: Nested subfolder grouping

Contact Details
No response
Present your request
Currently, the plugin allows grouping files into one level deep subfolders based on one attribute.
It would be great if we could use multiple attributes that correspond to different levels of nested subfolders.
├─ books/
│  ├─ 2014/
│  │  ├─ to-review/
│  │  ├─ reviewed/
├─ papers/
│  ├─ 2015/
│  │  ├─ to-review/
│  │  ├─ reviewed/

For which platform do you request this request??
Desktop

currently (I think) you can generate multiple levels but the relative path is needed

2014/to-review
2014/reviewed
...

If not, we can include the obsidian native functions into the formulas so you can configure your own system

Maybe the config section could be too much big with those details
My idea its to include scripts just like the formulas to run whatever you want if a value of your column is modified

https://github.com/RafaelGB/obsidian-db-folder/issues/115

sure, I like the idea. We can discuss about the error handling.
I closed this issue in favour of automations.
The idea is to add "rules" to a column and one of then should be the creation of subfolders. Just like the formulas
I.E.:
db.rules.moveNote(`${row.a}/${row.b}`)

But i dont know yet where the rules should be: column config or table config

As UX improvement of this task I suggest to create a new settings component just like tags to multi select the inputs instead of use the comma split

It seems this feature is available only for certain Column Types. Please describe this in your documentation.
Alternatively I could do this if you point me to the File where I should document it.


[Proposal] Give the all property amiss in the hasManyThrough output and adding hasOneThrough, belongsToThrough and belongsToManyThrough aliases

This is just a proposal for better compliance of hasManyThrough with other relationship outputs and defining new aliases.
Give the all property amiss in the hasManyThrough output
The current output of hasManyThrough includes an all property which contains all returned records, eg.
    => App\Models\User {#806
         id: 1,
         name: "test0",
         email<EMAIL_ADDRESS>         status: 0,
         created_at: "2017-12-04 12:11:20",
         updated_at: "2017-12-04 12:11:20",
         deleted_at: null,
         state: Illuminate\Database\Eloquent\Collection {#809
           all: [
             App\Models\Generic\State {#821
               id: 1,
               district_id: null,
               name: "TestState",
               additional_data: null,
               status: 0,
               created_at: "2017-12-04 12:04:00",
               updated_at: "2017-12-04 12:04:00",
               deleted_at: null,
               state_id: 1,
             },
           ],
         },
       }

This particular property is not conforming the norm and the state in this case can be simply an array (or a Collection) of States.
Adding hasOneThrough, belongsToThrough and belongsToManyThrough aliases
Also, since there is no (need for) hasOneThrough or belongsToThough, it also breaks the semantics of code and output. The above suggest the dropping of all property in output to fix this inconsistency; while this section proposes the addition of hasOneThrough, belongsToThrough and belongsToManyThrough aliases with shuffled parameter orders to comply the semantics.

This repo is for bug tracking. This isn't a bug so you can open a proposal in the laravel/internals repo

For suggestions and feature requests please use the https://github.com/laravel/internals repo.


Tooltip is empty

I'm seeing that "Example 17" on the Fuller Demo shows a tooltip that doesn't have any content.

"Example 17" is not meant to have a tooltip. When you open it in the Profile Editor, that field is blank. I'm not sure why it has "remark":"" in the JSON. I could fix the profile JSON on that.