Update Dependencies for system uses Python 3.12 or latter

Which package/s are you suggesting this feature for?
grid_client
Is your feature request related to a problem? Please describe
I had this error during running yarn install command
[4/4] Building fresh packages...
[15/20] ⠠ canvas
[-/20] ⠠ waiting...
[13/20] ⠠ vue-demi
[8/20] ⠠ cpu-features
warning Error running install script for optional dependency: "/home/sameh/Projects/tfgrid-sdk-ts/node_modules/cpu-features: Command failed.
Exit code: 1
Command: node buildcheck.js > buildcheck.gypi && node-gyp rebuild
Arguments:
Directory: /home/sameh/Projects/tfgrid-sdk-ts/node_modules/cpu-features
Output:
gyp info it worked if it ends with ok
gyp info using<EMAIL_ADDRESS>gyp info using<EMAIL_ADDRESS>| linux | x64
gyp info find Python using Python version 3.12.2 found at \"/home/sameh/.asdf/installs/python/3.12.2/bin/python3\"
gyp info spawn /home/sameh/.asdf/installs/python/3.12.2/bin/python3
gyp info spawn args [
gyp info spawn args   '/home/sameh/Projects/tfgrid-sdk-ts/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/home/sameh/Projects/tfgrid-sdk-ts/node_modules/cpu-features/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/sameh/Projects/tfgrid-sdk-ts/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/sameh/.cache/node-gyp/20.12.2/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/home/sameh/.cache/node-gyp/20.12.2',
gyp info spawn args   '-Dnode_gyp_dir=/home/sameh/Projects/tfgrid-sdk-ts/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/home/sameh/.cache/node-gyp/20.12.2/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/home/sameh/Projects/tfgrid-sdk-ts/node_modules/cpu-features',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
Traceback (most recent call last):
  File \"/home/sameh/Projects/tfgrid-sdk-ts/node_modules/node-gyp/gyp/gyp_main.py\", line 42, in <module>
    import gyp  # noqa: E402
    ^^^^^^^^^^
  File \"/home/sameh/Projects/tfgrid-sdk-ts/node_modules/node-gyp/gyp/pylib/gyp/__init__.py\", line 9, in <module>
    import gyp.input
  File \"/home/sameh/Projects/tfgrid-sdk-ts/node_modules/node-gyp/gyp/pylib/gyp/input.py\", line 19, in <module>
    from distutils.version import StrictVersion
ModuleNotFoundError: No module named 'distutils'
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/home/sameh/Projects/tfgrid-sdk-ts/node_modules/node-gyp/lib/configure.js:325:16)
gyp ERR! stack     at ChildProcess.emit (node:events:518:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:294:12)
gyp ERR! System Linux 6.5.0-44-generic
gyp ERR! command \"/home/sameh/.asdf/installs/nodejs/20.12.2/bin/node\" \"/home/sameh/Projects/tfgrid-sdk-ts/node_modules/.bin/node-gyp\" \"rebuild\"
gyp ERR! cwd /home/sameh/Projects/tfgrid-sdk-ts/node_modules/cpu-features

distutils module is used to install Python packages for node-gyp. This module is usually installed as part of Python installation.
In Python v3.12, distutils module was removed. This means your local doesn't have any tool in place to support Python package installation for node-gyp.
So when node-gyp tries to install a Python package using distutils that are not installed as part of Python, the above issue happens.
Describe the solution you'd like
for Python 3.12 or latter, to fix this you need to run this command:
python3 -m pip install setuptools

Add a note in the documentation stating that if the Python version is 3.12 or later used, users may encounter this specific error and provide instructions for resolving it using the mentioned command.

Suggestion: add hint in docs to install setuptools before install deps

Work completed:
Added a note for setuptools installation.

verified
docs edited and added installing python to the installtion of the gridclient docs


Linked YT Music icon to Revanced version

Linked YT Music to YT Music Revanced package
Description
Linked
Type of change

:x: Bug fix (non-breaking change which fixes an issue)
✅ : Icon addition (non-breaking change that adds/modifies Lawnicons's icons)
:x: General change (non-breaking change that doesn't fit the above categories like copyediting)

Icons linked

YT Music (linked app.android.apps.youtube.music to<EMAIL_ADDRESS>

youtube_music_revanced icon isn't present. You can make a new icon or link it to
@ drawable/youtube_revanced

Thanks mate, silly mistake 🤦🏼


Will adapter for Android 9 ?

Not work on Android 9
ROM: MIUI 10 9.9.3
I have rooted the phone.

App was developed on 9.


Is the clustering result stable?

For the same input, is the clustering result given by this code stable? That is, whether the output result of multiple runs is the same clustering result.

I set the parameter random_state, ran it twice, and finally confirmed that the results were consistent.


multipass: 1.11.0 -> 1.11.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/canonical/multipass/releases.
meta.description for multipass is: Ubuntu VMs on demand for any workstation.
meta.homepage for multipass is: https://multipass.run
Updates performed

Version update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.11.1 with grep in /nix/store/x1419488jxsg604pb7328h06f3wp13ha-multipass-1.11.1
found 1.11.1 in filename of file in /nix/store/x1419488jxsg604pb7328h06f3wp13ha-multipass-1.11.1

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
multipass

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/x1419488jxsg604pb7328h06f3wp13ha-multipass-1.11.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A multipass https://github.com/r-ryantm/nixpkgs/archive/0e13156ae35702f1720b28b75b68449263ddc4ef.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/x1419488jxsg604pb7328h06f3wp13ha-multipass-1.11.1
ls -la /nix/store/x1419488jxsg604pb7328h06f3wp13ha-multipass-1.11.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

multipass

Maintainer pings
cc @jnsgruk for testing.

I just tested this and it worked just fine!


Update opentracing-go to v1.2.0

A library I imported in my code depends on<EMAIL_ADDRESS>which results in the v1.1.0 (which baseplate depends on) being overwritten.
The problem is they have changed the error log key in v1.2.0 which is a breaking change since it results in the Span.FinishWithOptions function ignoring errors.
Can we update the version to v1.2.0 (release notes)?
Or, otherwise, we might want to respect both error and error.object keys.

The CI failure is unrelated to this change, is it?

The CI failure is unrelated to this change, is it?

it's unrelated.

Can someone merge this, please?


Fixing dw query builder to handle Unicode characters

Why make this change?
Dw pipeline test case was failing due to incorrect parsing for chinese characters.
In addition, dw pipeline was not enabled for linux.
What is this change?
Use the N' prefix that handles unicode characters correctly when building the json in dw.
Added the db type in the config-generator.sh
Tests
Unique characters test case passes along with other test cases.

and a test to validate

and a test to validate


There is no auth basic feature in horaedb-client-go

There is no auth basic feature in horaedb-client-go

Yeah, welcome anyone interested to contribute.


BIP32-based wallet refactor

fixes #292
fixes #295
fixes #296

Refactored the internal life cycle to handle the new "account" concept. As a result, the command line needed some big changes. Now it's sub command based:
Usage:
  textile [OPTIONS] <command>

Application Options:
  -r, --repo-dir=     specify a custom repository path

Log Options:
  -l, --log-level=    set the logging level [debug, info, notice, warning, error, critical] (default: debug)
  -n, --no-log-files  do not save logs on disk

Help Options:
  -h, --help          Show this help message

Available commands:
  daemon   Start a node daemon
  init     Init the node repo
  shell    Start a node shell
  version  Print version and exit

Adding --help to sub commands prints detailed options.
To get started, you'd init a repo (as w/ ipfs) like so:
textile init -s STqdutoFiPV9Nzh96L7bvSZNzrHkQiB2phDW1ySG9Na1kkGE

Then run the daemon:
textile daemon

Or the interactive shell:
textile shell

One big benefit here is that you are no longer able to overwrite your repo 💯

One more example... textile shell --help:
Usage:
  textile [OPTIONS] shell [shell-OPTIONS]

Start an interactive node shell session.

Application Options:
  -r, --repo-dir=              specify a custom repository path

Log Options:
  -l, --log-level=             set the logging level [debug, info, notice, warning, error, critical] (default: debug)
  -n, --no-log-files           do not save logs on disk

Help Options:
  -h, --help                   Show this help message

[shell command options]

    Gateway Options:
      -g, --gateway-bind-addr= set the gateway address (default: <IP_ADDRESS>:random)

    Cafe Options:
      -c, --cafe=              cafe host address
          --cafe-bind-addr=    set the cafe address
          --cafe-db-hosts=     set the cafe mongo db hosts uri
          --cafe-db-name=      set the cafe mongo db name
          --cafe-db-user=      set the cafe mongo db user
          --cafe-db-password=  set the cafe mongo db user password
          --cafe-db-tls        use TLS for the cafe mongo db connection
          --cafe-token-secret= set the cafe token secret
          --cafe-referral-key= set the cafe referral key

Further cleanup for the wallet command:
$ textile --help
Usage:
  textile [OPTIONS] <command>

Help Options:
  -h, --help  Show this help message

Available commands:
  daemon   Start a node daemon
  init     Init the node repo and exit
  shell    Start a node shell
  version  Print version and exit
  wallet   Manage a wallet of accounts

$ textile wallet --help
Usage:
  textile [OPTIONS] wallet <accounts | init>

Initialize a new wallet, or view accounts from an existing wallet.

Help Options:
  -h, --help      Show this help message

Available commands:
  accounts
  init

Tests passing on this, finally. Plenty more to do, but this PR is cooked. Can now move to smaller changes. Can I haz review approval, anyone?


Floating action button for quick report entry adding implemented

As I have already written on the Google+ group it would be good to have a fab for quick time adding. So, here is my variant on implementing it. You can include it into your codebase if you want.

Hi, thanks for participating.
I don't think FAB at the main menu screen is the appropriate way to do this, it's kind of confusing UI. In the new version, main menu will be replaced by navigation drawer at the left side. Until then, you can use Chronometer for quick time adding.


mesheryctl status on windows looks bad

Description
mesheryctl status command output on linux or Mac seems to presented better. On windows it looks pretty bad. The formatting is off but a big factor.
Expected Behavior
mesheryctl status prints the status in a manner identical to that of on linux or Mac.

@nunix can you please add a snapshot from your terminal?

here you have

Got it! im on it will update!

Fixed by #460.


Update metrics scripts

Purpose
Make it so the right metrics scripts run properly during their celery beat.
Changes

Ensure the script has access to remote base folder regardless of where it's called from
Import the management commands to celery
Disable running keen snapshots by default (because they are slow)

QA Notes
No QA
Side Effects
Anything that might have relied on being able to pass in the folder rather than using website settings will no longer work.
 Should be free of this testing mode soon. Continue to ignore as you have been.

test

test

Last? test

Welllll, one more.

test


[HOLD for payment 2024-08-13] [$250] Delay reconnect callbacks to prevent thundering herd DDOS

Coming from https://expensify.slack.com/archives/C05CBC62HGW/p1721844511149789
Problem
When the site is fully offline and comes back up, App web clients DDOS the API. This "thundering herd" :zap::bison: problem occurs because (on web) App considers itself offline if Auth is not reachable. When Auth comes back online, all NewDot clients call ReconnectApp at close to the same time, leading to an artificial surge in traffic right when Auth comes back.
Note: this does not affect iOS or Android because on those platforms there are native platform APIs for network connectivity information, and we just trust those without considering whether the server is reachable.
Solution
Update the App network layer to separately track whether internet is reachable, and if the Expensify servers are down. If the server was down and comes back online, add a randomized delay of between 0 and 20 seconds before executing reconnect callbacks to space out the reconnect callbacks across many devices.
To do this, we'll make the following changes:

Start separately tracking isInternetReachable and isServerReachable. We can do this by updating the reachabilityTest config of the @react-native-community/netinfo package such that:

if http code is != 200, internet is down and isInternetReachable should be set to false
if jsonCode != 200 then auth is down, and isServerReachable should be set to false

isOffline (the Onyx field) will be updated to be !isInternetReachable || !isServerReachable. This is essentially equivalent to what we have today
Then, we'll address the thundering herd. To do this, we'll update triggerReconnectionCallbacks to add some special handling for a new reason which we can simply call serversStandingUp. If the serversStandingUp reason is provided, then we'll add a random delay of between 0 and 20 seconds (inclusive) before executing reconnection callbacks. This means that in aggregate we'll space out the reconnection callbacks and not all clients will attempt to reconnect at the same time.

Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~012f66d59aecc97ac2
Upwork Job ID:<PHONE_NUMBER>060407566
Last Price Increase: 2024-07-25
Automatic offers:

ShridharGoel | Contributor | 103272034

Issue OwnerCurrent Issue Owner: @sonialiap

Definitely don't need design support on this one

@ShridharGoel the problem with making this external is that it's not clear how you'd test the server being unreachable (i.e: Ping responding with a non-zero jsonCode)

Can you mock responses in the chrome dev tools network tab?

Random delay of clients' reconnection should not be the perfect solution, imagine that the amount of clients surge in future - then increase the 20 seconds to 40 seconds? Need to update server side code, e.g. add a message queue(simple FIFO should be ok) to cache the callback reconnections, even you are not using any load balancing it's still ok for the server to just store the reconnection meta data in RAM then process them in asynchronous way(e.g. use a thread pool to traverse and handle these callback reconnections later).

Thanks for your feedback @wfdong. We agree that this solution won't scale forever, and are constantly working to improve the performance and reliability of our back-end. That said, we still think this change will be beneficial to our systems overall and serve us well for the foreseeable future

@roryabraham I'm not sure on how we can write a regression test for this. Do we need it? If so, can you suggest something? Thanks!

Payment summary:

@allroundexperts $250 - please request in ND
@ShridharGoel $250 - paid in upwork ✔

@roryabraham bumping Sibtain's question about whether we need a regression test and if yes, how it should be written for this issue https://github.com/Expensify/App/issues/46143#issuecomment-2282886844

I don't think so, what would it be anyways?
On Fri, Aug 16, 2024 at 11:54 AM Sonia Liapounova @.***>
wrote:

@roryabraham https://github.com/roryabraham bumping Sibtain's question
about whether we need a regression test and if yes, how it should be
written for this issue #46143 (comment)
https://github.com/Expensify/App/issues/46143#issuecomment-2282886844
—
Reply to this email directly, view it on GitHub
https://github.com/Expensify/App/issues/46143#issuecomment-2293111042,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA7Y2OLQ2SVNRX2PALOSRZDZRW44ZAVCNFSM6AAAAABLNHMBDOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTGEYTCMBUGI
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

That's the question 😂 Since Flo doesn't think we need one, closing out

Ah, I forgot to mention that in the most recent downtime, I've observed that this problem is fixed for now 👍

$250 approved for @allroundexperts


Can't generate metadata for GoogleMaps pod

When GoogleMaps is installed as pod, no metadata is generated for GoogleMaps APIs, because GoogleMaps Pod includes 3 folders in header search paths(${PODS_ROOT}/Headers/Public, ${PODS_ROOT}/Headers/Public/GoogleMaps, ${PODS_ROOT}/Headers/Public/GoogleMaps/GoogleMaps). These paths break the module recognition logic of clang and no module is found for GoogleMaps headers. This is why the GoogleMaps Pod adds additional compiler flag (-isystem flag) to make the compiler to treat them as system headers:
-isystem "${PODS_ROOT}/Headers/Public"
-isystem "${PODS_ROOT}/Headers/Public/GoogleMaps"
-isystem "${PODS_ROOT}/Headers/Public/GoogleMaps/GoogleMaps".
These compiler flags don't propagate to metadata generator and it fails finding module for GoogleMaps headers.

Fixed https://github.com/NativeScript/ios-runtime/pull/314


Tracking issue for supporting larger x86_64/CUDA benchmarks

We are planning to onboard more benchmarks including different batch sizes on x86_64 and CUDA benchmarks. However currently the x86_64 benchmarks already take ~50mins to finish (CUDA only takes ~7mins). We need to take some actions before adding more benchmarks.

https://github.com/openxla/iree/pull/12890 adds ResNet50 TF models at various batch sizes to CUDA. Increases benchmark time from 10 mins to 3.5 hours.

If I run T5-Large for both CUDA and x86_CPU, I get an out of disk space error. The saved model is 1.4 GB zipped and 3 GB unzipped.

I'm unable to run Bert-Large at different batch sizes due to lack of disk space. This is now blocking adding new benchmarks. @pzread is it possible to increase the disk space size?
Bert-Large consists of 9 benchmarks, each with a zipped saved model of 1.2GB. I'm guessing we'll need roughly 30GB for Bert-Large.
T5-Large is also 1.4GB zipped + 3 GB unzipped. We also plan on benchmarking this model at different batch sizes.

Sure, I think we can enlarge the disk size of the build machines as required (currently it's a 100GB ram disk and we have about 360GB ram in total). Before doing that, I want to make sure how much we need.
Can you use https://github.com/openxla/iree/pull/13000 as the merge base and post the output of "After disk size" in the build step build_e2e_test_artifacts (https://github.com/openxla/iree/actions/runs/4660055553/jobs/8247995730#step:11:1) ? You can do that by change your PR target base from main to jerry-enlarge-cpu-disk.
Also are these large models all part of the "long" benchmarks? It will probably make sense to put them into a different build target (currently all models are built under iree-benchmark-suites) and only built in the postsubmit or nightly job. I can start working on this

200GB looks good for now:

Since we have created a separate workflow to run long benchmarks and the disk size has been increased. I think we can close this issue for now.


Fix scope for the addon variable

“let” isn’t as forgiving as “var”, so this didn’t work right. Now it does.

Connects to heroku/herokuconnect#2612

Connects to heroku/herokuconnect#2612

:+1:


Updated README

Included TouchableHighlight component in example.
Clarified example setup.

Prop is a pretty widely accepted term in the React community but I'll merge this because I think the improved getting started instructions and the correction to the example code are valuable. Thanks.


Add a color option on revealer

Feedback appreciated #newbie

Hey, this one is a little bit too specific. Because in a real use case scenario you'll use css to specify the color you want, even specify a different color for each letter !
Maybe that's what this PR need more, an example showing how to specify a differ color for each letter, don't you think ?
Cheers man

Looking at the diff, the color picker seems a bit overkill.
Actually you would use a custom tokenizer and insert  around each letter no?
I add to solve a related problem recently (guess where), a 5 digits counter, starting from one, with leading 0 with less opacity.
function getDisplay() {
    var counterLength = 5,
        numItem = getCount(),
        prefix = '' + Array(counterLength + 1 - numItem.toString().length).join('0') + '';
    return prefix + numItem.toString();
}

Hey! interesting problem for your 5 digits, actually I would have used css to solve this issue. In addition you'll get nice animations out of the box by using css styles.
I've tried this approach in a little jsfiddle demo.
For the posterity I've attached the core concept code :
html

    0
    0
    0
    0
    0

    <button id="down">-1</button>
    <button id="up">+1</button>

css
span {
    display: inline-block;
    opacity: 0.15;
    transition: opacity 800ms ease-out;
}
.counter[data-digits="1"] span:nth-child(n+5), .counter[data-digits="2"] span:nth-child(n+4), .counter[data-digits="3"] span:nth-child(n+3), .counter[data-digits="4"] span:nth-child(n+2), .counter[data-digits="5"] span:nth-child(n+1) {
    opacity: 1.0;
    animation: pop-upwards 500ms;
}

js
var counter = document.querySelector('.counter');

function updateDisplay(count) {
    var digits = count.toString().split('');
    counter.dataset.digits = digits.length;
    // counter.classList.add('counter--' + digits.length);

    // reset to 0 every value
    Array.prototype.forEach.call(counter.querySelectorAll('span'), function(s) {
        s.innerHTML = '0';
    });

    // fill the digits
    var dl = counter.dataset.digitLength;
    digits.forEach(function(d, idx) {
        counter.querySelector('span:nth-child('+(dl - digits.length + idx + 1)+')').innerHTML = d;
    });
};


jenkins: retry build_devel

I don't like polluting individual scripts like this. Can we just retry the call to build_devel?

retried up to 3 times when manually forcing a failure: https://jenkins.comma.life/blue/organizations/jenkins/openpilot/detail/retry-clone/8/pipeline/21/

happened again in CI but it was retried and it worked:
https://jenkins.comma.life/blue/organizations/jenkins/openpilot/detail/master/15/pipeline/20#step-290-log-133


Fixed TextureButton behavior when using shortcut

This change allows TextureButton to show expected behavior when a shortcut is set.
BaseButton used to only send "pressed" signal when a shortcut is pressed, without changing any of its internal state.
Now all signals and button state changes in BaseButton are sharing the same logic as "ui_accept" action when a shortcut event is triggered.
Note: the diff shows a lot of changes, but I've really just moved the logic from "ui_accept" events into a private method to re-use it for the shortcut event case.
Fixes #21261, fixes #18715.

Sorry for the late review.
@volzhs refactored the BaseButton event handling in #29060, so this PR would need to be rebased (or redone, given the overlap) to take this into account.

@akien-mga @volzhs Please let me know when is a good time to rebase again. Some recent changes have created new conflicts and I'd like to fix it again only if it gets merged to master within a reasonable delay.

Sorry about that. I had forgotten about this PR and I did #29717 myself to fix a related issue, as Shortcuts were indeed subpar in BaseButton.
I'm not sure if the changes in this PR are still relevant now, but if they are, we should merge them. Please leave a comment after rebasing as GitHub doesn't send a notification on force pushes.

@akien-mga No worries. I've pushed my change again to make sure shortcuts work exactly like mouse click or ui_accept events. It was needed to fix #21261, whereas your change already fixed #18715.

Thanks!


Extend README introduction

Hi Stefan,
thanks for publishing this! This pull request proposes an extended README introduction and some formatting modifications.
Cheers,
Thomas

Wrong target


rust_version doesn't display anything

I added rust_version to my POWERLEVEL9K_LEFT_PROMPT_ELEMENTS list but nothing appeared.
Looking at https://github.com/bhilburn/powerlevel9k/blob/master/powerlevel9k.zsh-theme#L1107 I see the command used but this doesn't seem to do anything.
Similar commands:
$ rustc --version
rustc 1.24.0-nightly (b98fd524e 2018-01-05)
$ rustc --version 2>&1 | grep -oe "^rustc\s*[^ ]*"
rustc 1.24.0-nightly
$ rustc --version 2>&1 | grep -oe "^rustc\s*[^ ]*" | grep -o '[0-9.a-z\\\-]*$'

Seems like the regex is off in the final grep?

Ha! Good catch @TheDan64 . When I implemented this segment I had some noise at the end of the version string. That is why the last regex exists.
Would you mind creating a PR for that?

Sure; should the inital "rustc" be trimmed out as well? Isn't there a rust icon that should make that obvious?

On second thought, this is a bit odd, as on my machine it works as expected:

There is also a test that ensures that the version string gets trimmed correctly.
Which version of grep do you use?

Oh. This is probably my fault. I'm not at my computer at the moment, but I bet I have grep aliased to ripgrep lol
I can verify later and close if that's the case

Ok, so it's not the case that my grep is aliased to ripgrep. It is, however, aliased to use --color=always like so: alias grep="/bin/grep --color=always"
It seems as though the additional coloring messes up the final grep (but not the first one) for some reason

I would avoid using ripgrep here, not everybody has it installed. And I wouldn't fix using the grep colors in that regex.
The way to go would be to get rid of grep completely, and use command:
prompt_rust_version() {
  local rust_version
  rust_version=$(command rustc --version 2>&1)
  # Remove "rustc " (inclusive the whitespace) from the beginning
  # of the version string and remove everything after the next
  # whitespace. This way we'll end up with only the version.
  rust_version=${${rust_version/rustc /}%% *}

  if [[ -n "$rust_version" ]]; then
    serializeSegment "$0" "" $1 "$2" $3 "darkorange" "$DEFAULT_COLOR" "Rust $rust_version" 'RUST_ICON'
  fi
}

Example:

My Bad. The parameters for serializeSegment and $1_prompt_segment differ.
This should be correct:
prompt_rust_version() {
  local rust_version
  rust_version=$(command rustc --version 2>&1)
  # Remove "rustc " (inclusive the whitespace) from the beginning
  # of the version string and remove everything after the next
  # whitespace. This way we'll end up with only the version.
  rust_version=${${rust_version/rustc /}%% *}

  if [[ -n "$rust_version" ]]; then
    "$1_prompt_segment" "$0" "$2" "darkorange" "$DEFAULT_COLOR" "$rust_version" 'RUST_ICON'
  fi
}

If this works, could you open a PR for that?

Meh. Need to apologize again. I wrote that piece of code based on the next branch. There we changed the colors to named ones:
"$1_prompt_segment" "$0" "$2" "darkorange" "$DEFAULT_COLOR" "$rust_version" 'RUST_ICON'

On master that line should be:
"$1_prompt_segment" "$0" "$2" "208" "$DEFAULT_COLOR" "Rust $rust_version" 'RUST_ICON'


Slow _find query when no documents match selector

_find returns results within 400-800ms when documents match the selector, but takes > 25000ms when no documents match the selector.
Expected Behavior

Expect result response to be at least as responsive when no documents match selector.

Current Behavior

Response takes >25000ms to return if no documents match the selector.

Possible Solution

Nil

Steps to Reproduce (for bugs)

Database contains 15k+ documents of athlete news.
Sample document:
{
  "_id": "athlete.news:news.mlb.sdi.662908",
  "_rev": "1-e0e6c8cef543dcc3b3773c91b225b0a2",
  "meta": {
    "type": "news.athlete",
    "source": "xxxx",
    "xxxx.source": {
      "player-news": "player-news:mlb.20180607"
    },
    "updatedTime": "2018-06-22T00:58:38+00:00",
    "updatedMicroseconds":<PHONE_NUMBER>000000
  },
  "league": "MLB",
  "athlete": {
    "id": "athl.mlb.sdi.114027",
    "name": "Wei-Yin Chen",
    "firstName": "Wei-Yin",
    "lastName": "Chen",
    "images": {
      "small": "https://www.xxxxxxxxx.com/usatsimg/image/thumb/250-250/9944051.jpg",
      "large": "https://www.xxxxxxxxx.com/usatsimg/image/thumb/650-650nw/9944051.jpg"
    }
  },
  "team": {
    "id": "team.mlb.sdi.2963",
    "name": "Miami",
    "nickname": "Marlins",
    "abbreviation": "MIA",
    "displayName": "Miami Marlins"
  },
  "dateTime": "2018-06-07T00:22:33-07:00",
  "priority": 3,
  "headline": "Falters in fifth in no-decision",
  "update": "Chen allowed two runs on six hits and two walks with two strikeouts over 4.1 innings in a no-decision Wednesday against St. Louis.",
  "analysis": "Chen has completed the fifth inning just three times in eight starts this season. When a walk and a single left Chen at 91 pitches with one out in the fifth, the Marlins knew his night needed to end before the four-run lead they had built dissipated into nothing. Chen induced seven groundballs, but three of the six he gave up went for extra bases, and he has reliever Drew Rucinski to thank for cleaning up his mess in the fifth. Chen owns a 5.86 ERA and will look to pick up his first win since his first start of the season when he takes the mound again Monday against the Giants.",
  "injured": false,
  "publishTime":<PHONE_NUMBER>
}

This is the index:
POST /{db}/_index
{
	"index": {
        "fields": [{
        	"publishTime": "desc"
        }]
    },
    "name": "news-by-dateTime-index",
    "type": "json"
}

The query selector:
POST  /{db}/_find
{
	"selector":{
		"league":{
			"$in":["MLS","EPL"]
		},
		"priority":{
			"$lte":3
		}
	},
	"fields":["_id","dateTime","priority","league","athlete","team","headline","update","analysis","injured","publishTime"],
	"sort":[{"publishTime":"desc"}],
	"limit":100,
	"bookmark": null,
	"execution_stats": true,
	"use_index":"_design/441fec348e12ff72dbb2563b15af4391ff665c9d"
}

Use the league selector to find all "MLB" news. If documents match, results are returned within 400-800ms.

{
    "docs": [
        { ... }, {... }, ...
    ],
    "bookmark": "g1AAAABWeJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYorJJZk5KSWpOrlpZYXW4EIvdycJL3ilEw9M3NTS0sjkD4OmL4coA7GpOhspvNZWQBdEBpM",
    "execution_stats": {
        "total_keys_examined": 0,
        "total_docs_examined": 152,
        "total_quorum_docs_examined": 0,
        "results_returned": 100,
        "execution_time_ms": 350.547
    }
}

Use the league selector to find all "EPL" news. If no documents match, an empty result is returned after more than 25000ms.

{
    "docs": [],
    "bookmark": "nil",
    "execution_stats": {
        "total_keys_examined": 0,
        "total_docs_examined": 24856,
        "total_quorum_docs_examined": 0,
        "results_returned": 0,
        "execution_time_ms": 34390.737
    }
}

Makes no difference if selector is changed to $eq

{
    "selector":{
		"league":{
			"$eq":"EPL"
		},
...

Context

This data is returned to a frontend client directly via a websocket request and affects the app's responsiveness.

Your Environment

Version used: CouchDB 2.1.1
Browser Name and version:: Chrome 68.0.3440.84
Operating System and version (desktop or mobile): Linux 4.17.11-arch1
Link to your project: N/A

Hi @CWX-iggy, thanks for your issue.
Unfortunately, you need to know that the $in operator has to do a full scan on every query on every document in the index. This cannot be pre-determined given the way Mango builds indexes. Mango will index the full contents of the field you are searching on (here, league) but has to at query time do a manual comparison of the contents of that index, per-document, with your parameters at query time. You will experience similar poor performance with other inexact operators, such as the $regex operator. @tonysun83 can you confirm my understanding of Mango indexing here?
I agree this is not clearly called out in the documentation, and for that I apologise. Would you agree with me converting your issue to one that would let us improve our documentation on this point?
Your choices to improve performance are to use Mango partial indexing to reduce the number of documents in your index (best option), use $eq or another exact operator instead of $in (next best, but you may not be able to capture your query, or try something like Clouseau/Dreyfus for full text search capability instead (hardest to set up, requires JDK6 and Apache Lucene, but is more flexible in terms of partial term searching). This last option lets you use text-based Mango indexes if you want to stay with Mango.
I recommend starting with partial indexing. You can use this to make multiple sub-indexes that reduce the likelihood of few documents being found. Partial indexes are especially good to filter documents by type, for which it looks like your meta.type or league fields might be good candidates. You may even find that running 2 mango queries (for instance, across 2 leagues) across 2 partial indexes and then merging the results in your client / app layers will be faster than running the single query across the entire content database.

One other thing I noticed - your query may not be using the right index. Can you check with /db/_explain that the index is indeed being used? I don't see you indexing the field you're querying against here, i.e. league.

@wohali I suspect that may the issue - no index on league (or priority). How do I specify multiple use_index? This doesn't seem to work:
{..."use_index": ["_design/filter_by_league","_design/sort_by_date"]}
This is the response from /db/_explain
{
    "error": "error",
    "reason": "{invalid_object,{<<\"\ufffd\ufffd\ufffd\ufffd\">>}}"
}

Fauxton simply reports error fetching query plan.

You can't. Mango underneath is using the same index file format as MapReduce - you're still querying only a single index at once. You'll need to index all the fields you need in a single index.
That's weird on your /db/_explain response, you may need to upgrade to 2.2.0 for some bugfixes.
Closing as this isn't strictly a Mango issue.

Happy to keep discussing it with you in this ticket if you like.

@wohali is right that the query that takes long is doing a full index scan on _all_docs as you can see from the stats:
"execution_stats": {
        "total_keys_examined": 0,
        "total_docs_examined": 24856,
        "total_quorum_docs_examined": 0,
        "results_returned": 0,
        "execution_time_ms": 34390.737
    }

The one that is faster is because it uses the "publishTime" index. Mango needs a way to know to select that index on your queries. The best way to do that is to add a publishTime into your selector even if its something like publishTime: {$gt: null}. That way it should always select that index. You can also add in the use_index to specify that index.
But running _explain on your queries will give you some information around what is happening internally and which index is being selected.
Since you using league in your query it might be worth adding that into the index as well even if you only use it with the $in selector. That way only the docs with a league field and a publishTime field will be in your index.

just to clarify why the index is being used in the first instance, "sort":[{"publishTime":"desc"}] adds an implicit "publishTime": { "$exists": true } to the "selector". If you're not specifying the sort in subsequent queries, you could add the selector clause explicitly to allow the index to be used (as @garrensmith suggested above).
It sounds like the most performant option would be to create an index on ["league", "publishTime"] and issue multiple requests - one for each league. You'd then need to merge/sort the results on the client.


Usolved Metas

I get the following error message.
/Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category/Wiring.agda:19,1-58
Unsolved metas at the following locations:
  /Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category/Tensor.agda:69,27-35
  /Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category/Tensor.agda:70,12-20
  /Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category/Tensor.agda:73,27-35
  /Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category/Tensor.agda:74,12-20
when scope checking the declaration
  open import Web.Semantic.DL.Category.Tensor using (_⊗_)

Am looking to fix it. I will create a commit referencing this issue when I do.

After working on a few files that error message disappeared but I still get it another file
src/Web/Semantic/DL/Category.agda

Error
/Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category.agda:4,1-44
Unsolved metas at the following locations:
  /Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category/Composition.agda:99,9-17
  /Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category/Composition.agda:99,35-43
  /Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category/Composition.agda:105,9-17
  /Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category/Composition.agda:105,36-44
when scope checking the declaration
  import Web.Semantic.DL.Category.Composition

After working on a few files that error message disappeared but I still get it another file
src/Web/Semantic/DL/Category.agda

Error
/Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category.agda:4,1-44
Unsolved metas at the following locations:
  /Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category/Composition.agda:99,9-17
  /Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category/Composition.agda:99,35-43
  /Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category/Composition.agda:105,9-17
  /Volumes/Dev/Programming/Proof/Agda/agda-web-semantic/src/Web/Semantic/DL/Category/Composition.agda:105,36-44
when scope checking the declaration
  import Web.Semantic.DL.Category.Composition

This commit deals with the "Unsolved Metas" problem https://github.com/bblfish/agda-web-semantic/commit/064ed896b3559d5d369ac807216a6e324911d5b4
There seem to be a few more problems to get a full clean compile.

This commit deals with the "Unsolved Metas" problem https://github.com/bblfish/agda-web-semantic/commit/064ed896b3559d5d369ac807216a6e324911d5b4
There seem to be a few more problems to get a full clean compile.


RMS value Calculator

Issue Id you have worked upon -
#336
Briefly explain your program logic -
RMS Value Calculator that allows users to input values separated by spaces and calculate the RMS value for those values.

Screenshots(Attach 2 screenshots of your own input and output) -
Attach here

By raising this PR I affirm that -

[x] My code follows the guidelines of this project.

[x] I have performed a self-review of my own code.

[x] I have commented my code.

[x] My code gives the correct output.

[x] I affirm that I strictly follow contributing guidelines and code of conduct.

Congratulations 🎉 @ravishrivas on your contribution.
Thanks for adding value to this repository. 😊
Let's celebrate this fest 😃 by providing more developers 👨💻👩💻 an opportunity to contribute. Help us make this repository reach far and more by sharing this to your friends and colleagues.
Your efforts can make difference.

Hey 👋, may you also encourage us by clicking the star icon on the repository main page. ⭐


New feature: add option to define attributes for playable characters

Hello !
I've been playing a bit with eastworld. Its amazing and it works great !
For my use case I'd like to have attributes for playable characters ( e.g. constitution, intelligence ) and to be able to define behaviours for agents based on those attributes.
For example if you have low constitution the agent will be less likely to be intimidated by you. ( similar to attribute checks in d&d )
The attributes could be defined for playable characters in the "eastworlds agent" react client and there could also be an endpoint available so they could be updated in real time from the actual game ( e.g. based on a player action we increase his constitution temporarily and then it will be more likely for the agents to be intimidated by him ).
For non playable agents it might suffice to just add the attribute behaviours to "agent instructions", the data that is send to the llm would also need to be updated in order for it to include player attributes.
Is this something planned for implementation ?
If not I could try to implement it.
Thanks

For non playable agents it might suffice to just add the attribute behaviours to "agent instructions", the data that is send to the llm would also need to be updated in order for it to include player attributes.

How would you expect to define what changes in the agents? If "Constitution" was an attribute from 1-10, how would the instructions to the agent change depending on the value? Would you define it yourself?

Yes, for example if player constitution is below 5 act hostile towards the player.
If player constitution is above 5 and player asks for something give it to him ( e.g . call GiveAnswer() )

Hmm, that seems like something more specific to a particular game. It's possible to do this already with Conversation instructions and custom actions. Not sure that it should be a part of the framework.


Broken on Django 1.9: render_bundle returns escaped HTML.

Release notes for 1.9: https://docs.djangoproject.com/en/1.9/releases/1.9/#simple-tag-now-wraps-tag-output-in-conditional-escape
Not entirely sure what the issue is, since it seems you're wrapping the output with mark_safe here: https://github.com/owais/django-webpack-loader/blob/master/webpack_loader/templatetags/webpack_loader.py#L25

+1

It doesn't solve the root problem, but turning autoescape off in the template where your bundle is rendered seems to be working for me. i.e.:
{% autoescape off %}
{% render_bundle 'bundle_name' %}
{% endautoescape %}
I'll try to take a closer look in the next few days unless someone beats me to it.

Yeah that's how I fixed it temporarily, too. I attempted to fix the core issue via fork but ran into a ton of test failures in NPM related to babel-core unable to find module '../traversal'.

mark_safe change is not yet released to PyPI. Tested wtih current master and it works fine with Django 1.9 in my setup.
So only thing needed for this issue is to release a new version to PyPI.

Released 0.2.3


Pin header scan for SEG-Y to big endian due to segyio limitation

Closes #258

Codecov Report

Merging #281 (7d90679) into main (a0d388a) will not change coverage.
Report is 2 commits behind head on main.
The diff coverage is 0.00%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
@@           Coverage Diff           @@
##             main     #281   +/-   ##
=======================================
  Coverage   84.44%   84.44%
=======================================
  Files          46       46
  Lines        2186     2186
  Branches      348      348
=======================================
  Hits         1846     1846
  Misses        297      297
  Partials       43       43

Files Changed
Coverage Δ

src/mdio/segy/_workers.py
17.72% <0.00%> (ø)


Restore not working with Veeam integrated domino.

HCL Product Version
Which version of the relevant HCL product is in us:-Domino 12.0.2FP1
Describe the bug
A clear and concise description of what the bug is.
We are in the process of migrating the backup solution to veeam. We are able to configure backup and it's working fine.
While restoring the database we are getting an admin account not found error. But required access has been granted to the AD account on veeam server.
Raised the case with Veeam and hcl but seems it's an issue with the integration script. Does anyone have any idea about this backup?
Error Message
If an error message is displayed, include details here. Quickly identifying the code causing the error will help to more quickly resolve it.
NO ADMIN ACCOUNT FOUND
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. Windows]

Additional context
Add any other context about the problem here.

Hi,
thanks for raising this issue. Your issue does not have sufficient information to help you.
This does not sound like a defect, but a configuration issue.
When using the Veeam integration. please make sure you are using the native VSS Writer support in Domino 12.0.2 as described for the Domino 12.0.2 integration. This provides you with full snapshnot support in Windows using the VSS Writer.
Only for restore you will need an integration. For backup all steps are handled by the Backup writer.
Can you please first make sure you have the new modern integration. You should not run the earlier integration for Windows any more, now that there is a native VSS Writer integration.
For this integration we wrote a simplified instruction just covering the exact steps for this type of configuration.
There are a couple of steps you have to perform to integrate the restore script.
Can you please verify first you have performed all the steps listed here -> https://opensource.hcltechsw.com/domino-backup/backup-providers/veeam/install_vss_writer/
Then please make sure where you see the error message.
The restore script invokes ssh, which then invokes a power shell script on the Veeam server.
Since Domino 12.0.2 there is end to end logging, which provides the log document directly in restore document.
If you can't share the logs here, please make sure escalates your ticket. Since you don't provide your customer name here, I don't know if you are the case I am right now involved in, where I am waiting for logs and details.

Here is the output.
get-vbrcredentials
Name                           Description

seint3615\notes                seint3615\notes
Hi HCL Team,
After changing the username in hierarchical format in config file ("notes" to "seint3615\notes"), Error "no administrator account found" has been resolved. However now we are facing the below error. Can you please check ?
2023-06-30 13:07:44 Mounting backup restore point: e1c431bc-f050-46d2-895b-a54a7a6b518a Backup Time Local: 2023-05-05 13:19:31 Backup Time UTC: 2023-05-05 11:19:31
Publish-VBRBackupContent : Failed to publish backup content. The user name or password is incorrect.
Failed to connect to share '\<IP_ADDRESS>\ADMIN$'
.
At C:\dominobackup\DominoRestore.ps1:251 char:23

... reSession = Publish-VBRBackupContent -RestorePoint $RestorePoint -Tar ...

CategoryInfo : InvalidResult: (:) [Publish-VBRBackupContent], CAppException
FullyQualifiedErrorId : PublishVBRBackupContentErrorId,Veeam.Backup.PowerShell.Cmdlets.PublishVBRBackupContent

Thanks & Reg:
Prabhat

There is no problem with the Notes account and password on the Veeam server. It is working at OS level. However when we are trying to restore the database through script it is showing username & password issue.
Looks like the process is stuck on the code below. Can you tell us from where it is getting the password for the “seint3615\notes”  account. Even we are not using “seint3615\notes” password to connect to Veeam server. It is using only SSL connectivity.
Is local window “notes” account also required on Domino server (<IP_ADDRESS>) ?
{
$RestoreSession = Publish-VBRBackupContent -RestorePoint $RestorePoint -TargetServerName $RestoreTarget -TargetServerCredentials $TargetAdminCreds -Reason "Domino Restore operation"
}

Thanks & Reg:
Prabhat

I changed the label from bug to help wanted. this is not a bug. this is a configuration issue from what it looks like.
We keep it open. But you really need someone to help you on the Veeam side.

Hello Daniel,
Thank you for your update!
We are now able to backup our Domino database via Veeam and restoration is also working fine. Requesting you to please share documentation regarding how the Domino directory link works with veeam backup.
Also please suggest how archival logs backup logs via veeam.
Thanks, Prabhat
Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows
From: Daniel @.>
Sent: 21 July 2023 15:38
To: @.>
Cc: Prabhat @.>; @.>
Subject: Re: [HCL-TECH-SOFTWARE/domino-backup] Restore not working with Veeam integrated domino. (Issue #10)
You don't often get email from @.*** Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification
I changed the label from bug to help wanted. this is not a bug. this is a configuration issue from what it looks like.
We keep it open. But you really need someone to help you on the Veeam side.
—
Reply to this email directly, view it on GitHubhttps://github.com/HCL-TECH-SOFTWARE/domino-backup/issues/10#issuecomment-1645339082, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BATGUPTV6KVUE75G4UUDUX3XRJIKPANCNFSM6AAAAAAZIENZVM.
You are receiving this because you authored the thread.Message ID: @.***>
This e-mail is confidential and it is intended only for the addressees. Any review, dissemination, distribution, or copying of this message by persons or entities other than the intended recipient is prohibited. If you have received this e-mail in error, kindly notify us immediately by telephone or e-mail and delete the message from your system. The sender does not accept liability for any errors or omissions in the contents of this message which may arise as a result of the e-mail transmission.

Good to hear it now works for you.
Can you describe in more detail what you are trying to do with directory links?
All the NSF data must be a on single volume. Directory links are only supported on the same disk/volume.
Domino backup is explicitly checking if all the data is on the same device and returns errors for databases outside the same volume.
You can split

NSF -> This is what needs online backup
NIFSNSF/FT  does not need a backup
DAOS  can be backed up separately without an online backup)
TRANSLOG needs separate backup

But you can't split NSF when you use snapshot Domino backup.
And there are no plans currently to change this.
The background is the complexity this would add on backup and restore.

Archive style Translog is a separate functionality in Domino backup/restore and cannot be combined directly with a snapshot backup.
Transaction logs are written step by step and need to be copied step by step. this doesn't match up with snapshot backup.
You can still configure translog backup when using Veeam using a file integration. For example you can store those translog extends via file copy operations or other integrations for S3.
Snapshots would not really work for translog data, because you would need to restore multiple snapshots.
Still it can be combined.
But please keep in mind that this is a much more complex setup than just a snapshot backup.
I don't know what your use case for archive style translog is.
Archive style translog is usually a good way to reduce the backup time and volume by only backing up the deltas during the week. With snapshot backup you have a completely different approach for backup and archive style backup is not required for snapshot backup from storage point of view.
Still you might also want to use it for point in time recovery -- which is the second aspect of archive translog.
To keep your configuration simple, you could use linear transaction logging (circular translog just with more space) and can point in time restore for some days at least.
Snapshot backups are not really a good match for snapshot backup. You would need to mount too many volumes for the restore case.
I hope this helps
Daniel

Hello Daniel,
Thanks for detailed explanation.
I will give you background regarding Veeam implementation. We are planning to configure Veeam backup for more than 30 servers. Curranty we are using TSM(Tivoli storage manager) and we are trying to replace TSM with Veeam.
We take two types of backup on TAPE via TSM. One is file-level backup and another is database(TDP). TDP is using Archival style of TXN for incremental backup. Only recently modified document is backed-up.
Regarding directory links actual database is stored in another drive and a Domino link is created inside Domino data directory.
Please suggest.
Thanks , Prabhat
Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows
From: Daniel @.>
Sent: 25 July 2023 11:11
To: @.>
Cc: Prabhat @.>; @.>
Subject: Re: [HCL-TECH-SOFTWARE/domino-backup] Restore not working with Veeam integrated domino. (Issue #10)
You don't often get email from @.*** Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification
Good to hear it now works for you.
Can you describe in more detail what you are trying to do with directory links?
All the NSF data must be a on single volume. Directory links are only supported on the same disk/volume.
Domino backup is explicitly checking if all the data is on the same device and returns errors for databases outside the same volume.
You can split

NSF -> This is what needs online backup
NIFSNSF/FT does not need a backup
DAOS can be backed up separately without an online backup)
TRANSLOG needs separate backup

But you can't split NSF when you use snapshot Domino backup.
And there are no plans currently to change this.
The background is the complexity this would add on backup and restore.
Archive style Translog is a separate functionality in Domino backup/restore and cannot be combined directly with a snapshot backup.
Transaction logs are written step by step and need to be copied step by step. this doesn't match up with snapshot backup.
You can still configure translog backup when using Veeam using a file integration. For example you can store those translog extends via file copy operations or other integrations for S3.
Snapshots would not really work for translog data, because you would need to restore multiple snapshots.
Still it can be combined.
But please keep in mind that this is a much more complex setup than just a snapshot backup.
I don't know what your use case for archive style translog is.
Archive style translog is usually a good way to reduce the backup time and volume by only backing up the deltas during the week. With snapshot backup you have a completely different approach for backup and archive style backup is not required for snapshot backup from storage point of view.
Still you might also want to use it for point in time recovery -- which is the second aspect of archive translog.
To keep your configuration simple, you could use linear transaction logging (circular translog just with more space) and can point in time restore for some days at least.
Snapshot backups are not really a good match for snapshot backup. You would need to mount too many volumes for the restore case.
I hope this helps
Daniel
—
Reply to this email directly, view it on GitHubhttps://github.com/HCL-TECH-SOFTWARE/domino-backup/issues/10#issuecomment-1649157078, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BATGUPUTEJXOLEJK2NEDGYDXR5MBXANCNFSM6AAAAAAZIENZVM.
You are receiving this because you authored the thread.Message ID: @.***>
This e-mail is confidential and it is intended only for the addressees. Any review, dissemination, distribution, or copying of this message by persons or entities other than the intended recipient is prohibited. If you have received this e-mail in error, kindly notify us immediately by telephone or e-mail and delete the message from your system. The sender does not accept liability for any errors or omissions in the contents of this message which may arise as a result of the e-mail transmission.

I think I explained most of it.
You can't use a different disk in case of snapshot.
What size are the disks? why do you need separate disks?
Usually you have one NSF disk and move everything else to different disks (NIFNSF, FT, DAOS, TRANSLOG)
You did not explain why you need archive translog. If this is for reducing the size of backup, that's not needed with snapshots any more.
If you want point in time recovery you need to find a way to backup the archive style translogs. this could be for example S3 or copy to a different disk and from where you back it up.
Translog can be quite some size. And if you don't need point in time recovery outside the snapshot times or just for a couple of days, I would only use linear translog in combination with snapshot backups via Veeam.

Can you describe where you directory link is? Directory links are not supported for snapshots. You can only have one disk in snapshot and there directory links make not much sense. the integration for the restore is using a script. the script does not have any logic to find databases based on directory links. I don't see that with a snapshot directory links make much sense, when only one disk is supported for snapshots.

Hello Daniel,
Our Domino data directory are stored in another drive and database is stored in another drive.
We use to create directory link like below so that we can utilize both drives.
@.***
We have many servers where application database is stored outside Domino data directory and we are not able to restore these database.
I have attached error.
Kindly suggest.
Thanks , prabhat
Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows

From: Daniel Nashed @.>
Sent: Tuesday, October 17, 2023 7:53:41 PM
To: HCL-TECH-SOFTWARE/domino-backup @.>
Cc: Prabhat Kumar5_c @.>; Author @.>
Subject: Re: [HCL-TECH-SOFTWARE/domino-backup] Restore not working with Veeam integrated domino. (Issue #10)
Can you describe where you directory link is? Directory links are not supported for snapshots. You can only have one disk in snapshot and there directory links make not much sense. the integration for the restore is using a script. the script does not have any logic to find databases based on directory links. I don't see that with a snapshot directory links make much sense, when only one disk is supported for snapshots.
—
Reply to this email directly, view it on GitHubhttps://github.com/HCL-TECH-SOFTWARE/domino-backup/issues/10#issuecomment-1766526543, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BATGUPTGE3ZMSSYEJNRUR53X72IG3AVCNFSM6AAAAAAZIENZVOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRWGUZDMNJUGM.
You are receiving this because you authored the thread.Message ID: @.***>
This e-mail is confidential and it is intended only for the addressees. Any review, dissemination, distribution, or copying of this message by persons or entities other than the intended recipient is prohibited. If you have received this e-mail in error, kindly notify us immediately by telephone or e-mail and delete the message from your system. The sender does not accept liability for any errors or omissions in the contents of this message which may arise as a result of the e-mail transmission.

c:\dominobackup\veeam\restore_db.cmd "C:\Backup\VeeamTest\testveem.nsf" "VeeamTest\testveem.nsf" "" "SEDLNS98" "" "SNAP" "20230914101942" "c:\backup\nsf" "D:\Lotus\Domino\Data\restore\restore1\testveem.nsf" ""
Active code page: 65001
[2023-10-10  7:35:33,16] --- RESTORE DB from SNAPSHOT ---
PhysicalFileName  : "C:\Backup\VeeamTest\testveem.nsf"
FileName          : "VeeamTest\testveem.nsf"
BackupReference   : ""
BackupNode        : "SEDLNS98"
BackupName        : ""
BackupMode        : "SNAP"
BackupStartDT     : "20230914101942"
BackupTargetDir   : "c:\backup\nsf"
RestoreFileName   : "D:\Lotus\Domino\Data\restore\restore1\testveem.nsf"
All Paramters     : "C:\Backup\VeeamTest\testveem.nsf" "VeeamTest\testveem.nsf" "" "SEDLNS98" "" "SNAP" "20230914101942" "c:\backup\nsf" "D:\Lotus\Domino\Data\restore\restore1\testveem.nsf" ""
"Restore operations from snapshot"
Appending missing .DAD extension
Searching for backup tag before mounting [dominobackup_20230914101942.tag]
[2023-10-10  7:35:33,18] ----- Begin Mounting snaphot -----

Domino Backup for Veeam Restore
Command      : [mount]
Date         : [20230914101942]
Connect      : [<IP_ADDRESS>]
VM Host      : [sedlns98]
RestoreOS    : [Windows]
Account      : [SANDVIK\VeeamNotes]
MountFUSE    : [False]
ClockSkewMin : [5]
Backup Time String       :<PHONE_NUMBER>1942
Backup Time (local time) : 09/14/2023 12:19:42
Backup Time (UTC)        : 09/14/2023 10:19:42
Backup Search Time UTC   : 09/14/2023 10:14:42
RestoreVmHost            : sedlns98
Credentials used:        : SANDVIK\VeeamNotes
Backup search: 316,5 seconds )
2023-10-10 07:41:04 Mounting backup restore point:  2ba4a450-66cc-42e3-b942-2e4aef334da4  Backup Time Local:  2023-09-14 16:47:19  Backup Time UTC:  2023-09-14 14:47:19
2023-10-10 07:43:31 Mount operation finished ( 147,4 seconds )
Backup Job Name : SOL_1900_VM_DOM_SERVER
Restore Point   : 2023-09-14T16:47:19.7600000+02:00
VM Name         : SEDLNS98

Mounted Disk   : f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98-flat.vmdk f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98_4-flat.vmdk f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98_3-flat.vmdk f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98_2-flat.vmdk
Mounted At     : C:\VeeamFLR\SEDLNS98_3d6ac375\Volume0 C:\VeeamFLR\SEDLNS98_3d6ac375\Volume1 C:\VeeamFLR\SEDLNS98_3d6ac375\Volume2 C:\VeeamFLR\SEDLNS98_3d6ac375\Volume3 C:\VeeamFLR\SEDLNS98_3d6ac375\Volume4
Mounted As     :
Available From : <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> (Port: 3260 )
Available Via  : iqn.2006-06.com.veeam:sedlns98.52a062e0ab254bf9a44e3ad0651d4658:LUN(0) iqn.2006-06.com.veeam:sedlns98.52a062e0ab254bf9a44e3ad0651d4658:LUN(1) iqn.2006-06.com.veeam:sedlns98.52a062e0ab254bf9a44e3ad0651d4658:LUN(2) iqn.2006-06.com.veeam:sedlns98.52a062e0ab254bf9a44e3ad0651d4658:LUN(3)
OK: Mount operations successfully completed
0
[2023-10-10  7:43:32,21] ----- End Mounting snaphot -----
Searching for backup tag after mounting [dominobackup_20230914101942.tag]
-----------------------------------------------------------Search Results-----------------------------------------------------
TARGET            : D:\Lotus\Domino\Data\restore\restore1\testveem.nsf.DAD
FOUND_FILE_PATH   : []
FOUND_NOTESDATA   : []
-------------------------------------------------------------Search End-------------------------------------------------------

c:\dominobackup\veeam\restore_db.cmd "D:\Lotus\Domino\VeeamTest2\ktest2.nsf" "VeeamTest2\ktest2.nsf" "" "SEDLNS98" "" "SNAP" "20231016172307" "c:\backup\nsf" "D:\Lotus\Domino\Data\restore\ktest2.nsf" ""
Active code page: 65001
[2023-10-17 14:43:46,65] --- RESTORE DB from SNAPSHOT ---
PhysicalFileName  : "D:\Lotus\Domino\VeeamTest2\ktest2.nsf"
FileName          : "VeeamTest2\ktest2.nsf"
BackupReference   : ""
BackupNode        : "SEDLNS98"
BackupName        : ""
BackupMode        : "SNAP"
BackupStartDT     : "20231016172307"
BackupTargetDir   : "c:\backup\nsf"
RestoreFileName   : "D:\Lotus\Domino\Data\restore\ktest2.nsf"
All Paramters     : "D:\Lotus\Domino\VeeamTest2\ktest2.nsf" "VeeamTest2\ktest2.nsf" "" "SEDLNS98" "" "SNAP" "20231016172307" "c:\backup\nsf" "D:\Lotus\Domino\Data\restore\ktest2.nsf" ""
"Restore operations from snapshot"
Appending missing .DAD extension
Searching for backup tag before mounting [dominobackup_20231016172307.tag]
[2023-10-17 14:43:46,66] ----- Begin Mounting snaphot -----

Domino Backup for Veeam Restore
Command      : [mount]
Date         : [20231016172307]
Connect      : [<IP_ADDRESS>]
VM Host      : [sedlns98]
RestoreOS    : [Windows]
Account      : [SANDVIK\VeeamNotes]
MountFUSE    : [False]
ClockSkewMin : [5]
Backup Time String       :<PHONE_NUMBER>2307
Backup Time (local time) : 10/16/2023 19:23:07
Backup Time (UTC)        : 10/16/2023 17:23:07
Backup Search Time UTC   : 10/16/2023 17:18:07
RestoreVmHost            : sedlns98
Credentials used:        : SANDVIK\VeeamNotes
Backup search: 333,3 seconds )
2023-10-17 14:49:30 Mounting backup restore point:  ffaafb24-263b-4868-a5bf-f71e537f0536  Backup Time Local:  2023-10-16 19:23:20  Backup Time UTC:  2023-10-16 17:23:20
2023-10-17 14:51:51 Mount operation finished ( 141,3 seconds )
Backup Job Name : SOL_1900_VM_DOM_SERVER
Restore Point   : 2023-10-16T19:23:20.0370000+02:00
VM Name         : SEDLNS98

Mounted Disk   : f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98_4-flat.vmdk f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98_2-flat.vmdk f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98_3-flat.vmdk f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98-flat.vmdk
Mounted At     : C:\VeeamFLR\SEDLNS98_0c8eaa85\Volume0 C:\VeeamFLR\SEDLNS98_0c8eaa85\Volume1 C:\VeeamFLR\SEDLNS98_0c8eaa85\Volume2 C:\VeeamFLR\SEDLNS98_0c8eaa85\Volume3 C:\VeeamFLR\SEDLNS98_0c8eaa85\Volume4
Mounted As     :
Available From : <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> (Port: 3260 )
Available Via  : iqn.2006-06.com.veeam:sedlns98.189c1102f2534b038bcf6e665653ca4f:LUN(0) iqn.2006-06.com.veeam:sedlns98.189c1102f2534b038bcf6e665653ca4f:LUN(1) iqn.2006-06.com.veeam:sedlns98.189c1102f2534b038bcf6e665653ca4f:LUN(2) iqn.2006-06.com.veeam:sedlns98.189c1102f2534b038bcf6e665653ca4f:LUN(3)
OK: Mount operations successfully completed
0
[2023-10-17 14:51:52,36] ----- End Mounting snaphot -----
Searching for backup tag after mounting [dominobackup_20231016172307.tag]
-----------------------------------------------------------Search Results-----------------------------------------------------
TARGET            : D:\Lotus\Domino\Data\restore\ktest2.nsf.DAD
FOUND_FILE_PATH   : [c:\VeeamFLR\SEDLNS98_0c8eaa85\Volume3\Lotus\Domino\data\dominobackup_20231016172307.tag]
FOUND_NOTESDATA   : [c:\VeeamFLR\SEDLNS98_0c8eaa85\Volume3\Lotus\Domino\data]
-------------------------------------------------------------Search End-------------------------------------------------------
xcopy /Y "c:\VeeamFLR\SEDLNS98_0c8eaa85\Volume3\Lotus\Domino\data\VeeamTest2\ktest2.nsf" "D:\Lotus\Domino\Data\restore\ktest2.nsf.DAD*"
File not found - ktest2.nsf
0 File(s) copied

Only one disk is supported for snapshot backup.
You should actually get an error during backup already depending on you configure multiple disks.
It checks during backup if the database is on the same volume then the data directory.
Because you are not explaining how exactly your directory structure looks like, I can not tell you why you are not getting an error here.
If more than one disk is involved in the NSF storage, you are running and unsupported configuration. this is part of the documented limitations of the snapshot backup.
The background for this limitation is that with multiple disks it is almost impossible to build a standardized interface to backup and restore data.
If this is one disk and you are just outside the data directory with Notes links only the restore script would need some customization.
But because this is a complex, none standard scenario for the Veeam backup integration, this is outside anything the backup project can do for you.
You are outside the supported configuration and you are not providing even sufficient details.

If this should be just one disk, there is no need for directory links and you could resolve it moving the data into the data directory
If you have more than one disk with NSF data it's unsupported and I can't help you go get it working

If you are having details in your message, they are not showing up. It looks like you are replying to the issue response by mail and that's not showing up.
Please log into GitHub and reply with details I can read.
Thanks
Daniel

I replied via email and my information was never complete. I am extremely sorry.
Paste recent email
Hello Daniel,
Our Domino data directory is stored in another drive and the database is stored in another drive.
We used to create a directory link like below so that we could utilize both drives.
backup error dir link.txt
backup error.txt
We have many servers where the application database is stored outside the Domino data directory and we are not able to restore these databases.

can you share the content of the directory links?

I just want to understand where you store your data.
is the main data directory below the binary directory?
how big are you disks each?
did you move out NIF via NIFNSF?
did you move out FT index to different disks?
are those both significant size?
are you using DAOS? where is DAOS located?
where is translog located?

I can't help you with getting around with multiple directory links. if you want to use Veeam you can only have one volume for NSF.
you only want to snapshot the data directory with the NSF data. I think with the right features enabled you can and should optimize your Domino data structure. This does not impact the application layer above, which does not see those directory links.
also directory links can effect performance. you can get away with 2-3 but with more directory links there can be issues as well.
I would avoid directory links in general and they are usually an indicator of a different problem.
We can try to help you to move to a supported setup, which also optimizes your snapshots.
You don't want NIF and FT nor DAOS in the same snapshot. The snapshot should contain the data directory on a separate drive.
Example setup:

d:\binary directory
e:\notesdata
f:\ft
g:\nif
t:\translog
h:\daos

If would usually at least split notesdata and daos.
The other parts can be on one volume.
What you want to snapshot for Domino data backup is disk d:\
You can combine different parts on one volume. But notesdata is what you snapshot and what needs to be separated.
If you use DAOS, it should be a separate snapshot.
Using snapshots for backup changes the way you have to look into your infrastructure.
This type of change would make your life easier in future also for other reasons.
I am not saying you need separate disks for everything. but everything should be moved out of the data directory and disk.

Hello Daniel,
Thank for your email and assistance. We have already configured Veeam on all servers now working to remove TSM.
Backup on servers are working fine and even we are able to restore database. But restore of database is not working if database is stored outside Domino binary.
I have reproduced this one of our test server for your suggestion but we are having many servers where data in TB are stored outside domino data directory.
Our domino directory is in D:\Lotus\Domino\data and program directory is in C:\Lotus\Domino.
We created a directory link and physically stored databases outside Domino directory(D:\Lotus\Domino\data). There is a link and we can open database outside directory from admin client.
Please check
@.***
@.***
These database are showing on backup database
[image]
But store is giving error
Searching for backup tag after mounting [dominobackup_20230914101942.tag]
-----------------------------------------------------------Search Results-----------------------------------------------------
TARGET            : D:\Lotus\Domino\Data\restore\restore1\testveem.nsf.DAD
FOUND_FILE_PATH   : []
FOUND_NOTESDATA   : []
-------------------------------------------------------------Search End-------------------------------------------------------

All required logs are attached and it contains scripts which we are using.

I just want to understand where you store your data:-  All Domino data is stored under D lotus domino data. And we are trying to restore database from C drive
is the main data directory below the binary directory? No , its in C drive and Data drive is in D
how big are you disks each?:- Production servers are having 40Gbs databases
did you move out NIF via NIFNSF?:- No
did you move out FT index to different disks?:- No
are those both significant size?:- On some servers
are you using DAOS? where is DAOS located?:- No
where is translog located?:- Its in E drive. Circular style .
HCL Case# CS0426621

Kindly suggest.
Thanks & Regards,
Prabhat Kumar
Technical Services Professional- Sandvik
Mobile: +919901113603 | E-mail<EMAIL_ADDRESS>@.***
Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows
From: Daniel Nashed @.>
Sent: Wednesday, October 18, 2023 1:14:05 AM
To: HCL-TECH-SOFTWARE/domino-backup @.>
Cc: Prabhat Kumar5_c @.>; Author @.>
Subject: Re: [HCL-TECH-SOFTWARE/domino-backup] Restore not working with Veeam integrated domino. (Issue #10)
can you share the content of the directory links?

I just want to understand where you store your data.
is the main data directory below the binary directory?
how big are you disks each?
did you move out NIF via NIFNSF?
did you move out FT index to different disks?
are those both significant size?
are you using DAOS? where is DAOS located?
where is translog located?

I can't help you with getting around with multiple directory links. if you want to use Veeam you can only have one volume for NSF.
you only want to snapshot the data directory with the NSF data. I think with the right features enabled you can and should optimize your Domino data structure. This does not impact the application layer above, which does not see those directory links.
also directory links can effect performance. you can get away with 2-3 but with more directory links there can be issues as well.
I would avoid directory links in general and they are usually an indicator of a different problem.
We can try to help you to move to a supported setup, which also optimizes your snapshots.
You don't want NIF and FT nor DAOS in the same snapshot. The snapshot should contain the data directory on a separate drive.
Example setup:

d:\binary directory
e:\notesdata
f:\ft
g:\nif
t:\translog
h:\daos

If would usually at least split notesdata and daos.
The other parts can be on one volume.
What you want to snapshot for Domino data backup is disk d:\
You can combine different parts on one volume. But notesdata is what you snapshot and what needs to be separated.
If you use DAOS, it should be a separate snapshot.
Using snapshots for backup changes the way you have to look into your infrastructure.
This type of change would make your life easier in future also for other reasons.
I am not saying you need separate disks for everything. but everything should be moved out of the data directory and disk.
—
Reply to this email directly, view it on GitHubhttps://github.com/HCL-TECH-SOFTWARE/domino-backup/issues/10#issuecomment-1767057056, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BATGUPWJDFC6NIBLAZBCIRTX73NYLAVCNFSM6AAAAAAZIENZVOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRXGA2TOMBVGY.
You are receiving this because you authored the thread.Message ID: @.***>
This e-mail is confidential and it is intended only for the addressees. Any review, dissemination, distribution, or copying of this message by persons or entities other than the intended recipient is prohibited. If you have received this e-mail in error, kindly notify us immediately by telephone or e-mail and delete the message from your system. The sender does not accept liability for any errors or omissions in the contents of this message which may arise as a result of the e-mail transmission.

c:\dominobackup\veeam\restore_db.cmd "D:\Lotus\Domino\VeeamTest2\ktest2.nsf" "VeeamTest2\ktest2.nsf" "" "SEDLNS98" "" "SNAP" "20231016172307" "c:\backup\nsf" "D:\Lotus\Domino\Data\restore\ktest2.nsf" ""
Active code page: 65001
[2023-10-17 14:43:46,65] --- RESTORE DB from SNAPSHOT ---
PhysicalFileName  : "D:\Lotus\Domino\VeeamTest2\ktest2.nsf"
FileName          : "VeeamTest2\ktest2.nsf"
BackupReference   : ""
BackupNode        : "SEDLNS98"
BackupName        : ""
BackupMode        : "SNAP"
BackupStartDT     : "20231016172307"
BackupTargetDir   : "c:\backup\nsf"
RestoreFileName   : "D:\Lotus\Domino\Data\restore\ktest2.nsf"
All Paramters     : "D:\Lotus\Domino\VeeamTest2\ktest2.nsf" "VeeamTest2\ktest2.nsf" "" "SEDLNS98" "" "SNAP" "20231016172307" "c:\backup\nsf" "D:\Lotus\Domino\Data\restore\ktest2.nsf" ""
"Restore operations from snapshot"
Appending missing .DAD extension
Searching for backup tag before mounting [dominobackup_20231016172307.tag]
[2023-10-17 14:43:46,66] ----- Begin Mounting snaphot -----

Domino Backup for Veeam Restore
Command      : [mount]
Date         : [20231016172307]
Connect      : [<IP_ADDRESS>]
VM Host      : [sedlns98]
RestoreOS    : [Windows]
Account      : [SANDVIK\VeeamNotes]
MountFUSE    : [False]
ClockSkewMin : [5]
Backup Time String       :<PHONE_NUMBER>2307
Backup Time (local time) : 10/16/2023 19:23:07
Backup Time (UTC)        : 10/16/2023 17:23:07
Backup Search Time UTC   : 10/16/2023 17:18:07
RestoreVmHost            : sedlns98
Credentials used:        : SANDVIK\VeeamNotes
Backup search: 333,3 seconds )
2023-10-17 14:49:30 Mounting backup restore point:  ffaafb24-263b-4868-a5bf-f71e537f0536  Backup Time Local:  2023-10-16 19:23:20  Backup Time UTC:  2023-10-16 17:23:20
2023-10-17 14:51:51 Mount operation finished ( 141,3 seconds )
Backup Job Name : SOL_1900_VM_DOM_SERVER
Restore Point   : 2023-10-16T19:23:20.0370000+02:00
VM Name         : SEDLNS98

Mounted Disk   : f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98_4-flat.vmdk f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98_2-flat.vmdk f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98_3-flat.vmdk f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98-flat.vmdk
Mounted At     : C:\VeeamFLR\SEDLNS98_0c8eaa85\Volume0 C:\VeeamFLR\SEDLNS98_0c8eaa85\Volume1 C:\VeeamFLR\SEDLNS98_0c8eaa85\Volume2 C:\VeeamFLR\SEDLNS98_0c8eaa85\Volume3 C:\VeeamFLR\SEDLNS98_0c8eaa85\Volume4
Mounted As     :
Available From : <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> (Port: 3260 )
Available Via  : iqn.2006-06.com.veeam:sedlns98.189c1102f2534b038bcf6e665653ca4f:LUN(0) iqn.2006-06.com.veeam:sedlns98.189c1102f2534b038bcf6e665653ca4f:LUN(1) iqn.2006-06.com.veeam:sedlns98.189c1102f2534b038bcf6e665653ca4f:LUN(2) iqn.2006-06.com.veeam:sedlns98.189c1102f2534b038bcf6e665653ca4f:LUN(3)
OK: Mount operations successfully completed
0
[2023-10-17 14:51:52,36] ----- End Mounting snaphot -----
Searching for backup tag after mounting [dominobackup_20231016172307.tag]
-----------------------------------------------------------Search Results-----------------------------------------------------
TARGET            : D:\Lotus\Domino\Data\restore\ktest2.nsf.DAD
FOUND_FILE_PATH   : [c:\VeeamFLR\SEDLNS98_0c8eaa85\Volume3\Lotus\Domino\data\dominobackup_20231016172307.tag]
FOUND_NOTESDATA   : [c:\VeeamFLR\SEDLNS98_0c8eaa85\Volume3\Lotus\Domino\data]
-------------------------------------------------------------Search End-------------------------------------------------------
xcopy /Y "c:\VeeamFLR\SEDLNS98_0c8eaa85\Volume3\Lotus\Domino\data\VeeamTest2\ktest2.nsf" "D:\Lotus\Domino\Data\restore\ktest2.nsf.DAD*"
File not found - ktest2.nsf
0 File(s) copied

c:\dominobackup\veeam\restore_db.cmd "C:\Backup\VeeamTest\testveem.nsf" "VeeamTest\testveem.nsf" "" "SEDLNS98" "" "SNAP" "20230914101942" "c:\backup\nsf" "D:\Lotus\Domino\Data\restore\restore1\testveem.nsf" ""
Active code page: 65001
[2023-10-10  7:35:33,16] --- RESTORE DB from SNAPSHOT ---
PhysicalFileName  : "C:\Backup\VeeamTest\testveem.nsf"
FileName          : "VeeamTest\testveem.nsf"
BackupReference   : ""
BackupNode        : "SEDLNS98"
BackupName        : ""
BackupMode        : "SNAP"
BackupStartDT     : "20230914101942"
BackupTargetDir   : "c:\backup\nsf"
RestoreFileName   : "D:\Lotus\Domino\Data\restore\restore1\testveem.nsf"
All Paramters     : "C:\Backup\VeeamTest\testveem.nsf" "VeeamTest\testveem.nsf" "" "SEDLNS98" "" "SNAP" "20230914101942" "c:\backup\nsf" "D:\Lotus\Domino\Data\restore\restore1\testveem.nsf" ""
"Restore operations from snapshot"
Appending missing .DAD extension
Searching for backup tag before mounting [dominobackup_20230914101942.tag]
[2023-10-10  7:35:33,18] ----- Begin Mounting snaphot -----

Domino Backup for Veeam Restore
Command      : [mount]
Date         : [20230914101942]
Connect      : [<IP_ADDRESS>]
VM Host      : [sedlns98]
RestoreOS    : [Windows]
Account      : [SANDVIK\VeeamNotes]
MountFUSE    : [False]
ClockSkewMin : [5]
Backup Time String       :<PHONE_NUMBER>1942
Backup Time (local time) : 09/14/2023 12:19:42
Backup Time (UTC)        : 09/14/2023 10:19:42
Backup Search Time UTC   : 09/14/2023 10:14:42
RestoreVmHost            : sedlns98
Credentials used:        : SANDVIK\VeeamNotes
Backup search: 316,5 seconds )
2023-10-10 07:41:04 Mounting backup restore point:  2ba4a450-66cc-42e3-b942-2e4aef334da4  Backup Time Local:  2023-09-14 16:47:19  Backup Time UTC:  2023-09-14 14:47:19
2023-10-10 07:43:31 Mount operation finished ( 147,4 seconds )
Backup Job Name : SOL_1900_VM_DOM_SERVER
Restore Point   : 2023-09-14T16:47:19.7600000+02:00
VM Name         : SEDLNS98

Mounted Disk   : f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98-flat.vmdk f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98_4-flat.vmdk f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98_3-flat.vmdk f628704f-e964-4537-89c7-1348fe93aa5a (vm-37490)\SEDLNS98_2-flat.vmdk
Mounted At     : C:\VeeamFLR\SEDLNS98_3d6ac375\Volume0 C:\VeeamFLR\SEDLNS98_3d6ac375\Volume1 C:\VeeamFLR\SEDLNS98_3d6ac375\Volume2 C:\VeeamFLR\SEDLNS98_3d6ac375\Volume3 C:\VeeamFLR\SEDLNS98_3d6ac375\Volume4
Mounted As     :
Available From : <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS> (Port: 3260 )
Available Via  : iqn.2006-06.com.veeam:sedlns98.52a062e0ab254bf9a44e3ad0651d4658:LUN(0) iqn.2006-06.com.veeam:sedlns98.52a062e0ab254bf9a44e3ad0651d4658:LUN(1) iqn.2006-06.com.veeam:sedlns98.52a062e0ab254bf9a44e3ad0651d4658:LUN(2) iqn.2006-06.com.veeam:sedlns98.52a062e0ab254bf9a44e3ad0651d4658:LUN(3)
OK: Mount operations successfully completed
0
[2023-10-10  7:43:32,21] ----- End Mounting snaphot -----
Searching for backup tag after mounting [dominobackup_20230914101942.tag]
-----------------------------------------------------------Search Results-----------------------------------------------------
TARGET            : D:\Lotus\Domino\Data\restore\restore1\testveem.nsf.DAD
FOUND_FILE_PATH   : []
FOUND_NOTESDATA   : []
-------------------------------------------------------------Search End-------------------------------------------------------

not sure why you are starting again with the same thread. please read my last reply.
there is no other supported way. Snapshots and classical backup like IBM Spectrum Protect are completely different approaches to backup Domino. You can't expect a 1:1 transition. You need to adopt to the nature of snapshot backup and apply today's best practices.
There is nothing more I can do for you in context of this issue. I am closing the issue.
There is no bug nor enhancement that can be added from the project side.
Please re-read my last detailed response.


Jacdac extension

This pull requests add support for Jacdac for this accessory which allows users to use simulators in MakeCode. Jacdac support will be released in the next major release of MakeCode for micro:bit (summer 2022).

This change adds a new nested extension (jacdac folder)
and does not modify the existing extension.
No hardware modification is required for existing accessories, this feature
is backward compatible. However, it requires a micro:bit V2 to run.

The benefits for the users and you will be:

Simulator Jacdac enables simulations of all sensors and actuators
Digital twins Jacdac surfaces the hardware state directly into the MakeCode editor
Standardized blocks and lessons the programming will be done through
Jacdac blocks maintained by the Microsoft team

We recommend reading the Jacdac software only accessory documentation page to learn more
about the details of this approach. You can also review a list of similar software only extensions.
Please do not hesitate to contact us through this pull request or at<EMAIL_ADDRESS>if you have any question or want to schedule a call.
How to test this extension as a user?
This features requires to beta editor of MakeCode at https://makecode.microbit.org/beta.

Click on https://microsoft.github.io/jacdac-docs/ddk/microbit/extension-samples/
Find the extension for this accessory
Click on Try MakeCode to open a project that as a user
Use the blocks from the Modules toolbox

You can follow the micro:bit Jacdac guide to learn how Jacdac integrates into MakeCode
TODOs

[ ] merge this pull request
[ ] create a new release for the repository
[ ] review the accessory page in the Jacdac device catalog to make sure we got all the details right

Once the pull request is merged, we will update the catalog to point to it rather than our temporary fork.
Future accessories TODOs

Review the micro:bit accessory Jacdac integration guide to learn how you can integrate Jacdac into your future accessories for a better user experience. We provide various options to integrate Jacdac into your hardware at minimal cost.
Review the Jacdac Device Development Kit for more details about hardware integration of Jacdac in general.
Contact us if you have any question about adding Jacdac to your next accessory through Discussions or at<EMAIL_ADDRESS>

Needs a bit of cleaning before filing.


Postman download link pointing at google play store??

The link for PostMan is going to the google play store???? surely it should be going here, I thought it was rather strange
I would suggest this:
https://www.postman.com/downloads/

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 12c8e934-6639-88c5-08c1-31c90531854e
Version Independent ID: 440b6cf2-692c-9a2c-558b-22326b3bdfb3
Content: Call the Text Analytics API - Azure Cognitive Services
Content Source: articles/cognitive-services/text-analytics/how-tos/text-analytics-how-to-call-api.md
Service: cognitive-services
Sub-service: text-analytics
GitHub Login: @aahill
Microsoft Alias: aahi

@CelticCoder we will review your feedback and get back to you shortly. Thanks.

@CelticCoder thanks for you feedback. We will update link accordingly.


Log correct list of nodes

/assign @nikhiljindal

This change is

Coverage decreased (-0.03%) to 44.053% when pulling bf3739d8524494d8dabf3ca701631c656eeb4ac7 on nicksardo:node-log-msg into 2a0cf435cb6b867bc7bc24aa198a34909ffe3929 on kubernetes:master.

haha, lgtm, thanks :)


[Feature] Batched writes during subgraph indexing

Description
Currently Graph Node writes entities to the store on every block. This leads to a large number of database interactions during historical syncing. This can be a significant performance bottleneck.
Graph Node could instead store entity changes in memory, writing those changes to the database every X blocks, or when the diff to be written is above a certain size.
This should combine with https://github.com/graphprotocol/graph-node/issues/4477 to reduce the number and size of database updates during syncing.
Are you aware of any blockers that must be resolved before implementing this feature? If so, which? Link to any relevant GitHub issues.

Impact on proof of indexing (ensuring that POI is not impacted by this change)

Some information to help us out

[ ] Tick this box if you plan on implementing this feature yourself.
[X] I have searched the issue tracker to make sure this issue is not a duplicate.

Closed by #4606.


Update documentation: no need to include play-json in own build.sbt

You will need to add play-json as a dependency to your project when using private fields. Do this by adding the below line to your build.sbt file.

libraryDependencies ++= Seq("com.typesafe.play" %% "play-json" % "2.4.0")

It seems this library is already included in your build.sbt:
libraryDependencies ++= Seq(
  "com.typesafe.play" %% "play-json" % "2.4.0",
  "commons-codec" % "commons-codec" % "1.10",
  "org.scalatest" % "scalatest_2.11" % "2.2.4" % "test"
)

Nice library btw :-)

Nice catch, commit d3d0363f7512b30492b33934a3b4371f6f0a8c7f updates the README accordingly.

Nice library btw :-)
Thank you!


InfoGAN: Q_loss question

Hi, great work and repo. I have one question about the way the Qloss is computed in infoGAN here
ent = tf.reduce_mean(-tf.reduce_sum(tf.log(c + 1e-8) * c, 1))
To me, log(c) is not correct there.
I guess it should be log(P(c)), so log(0.1) in this implementation.
With that, the Qloss value than reach the 2.3 of H(c) mentioned in the paper.
Does it make sense? Thanks

Just re-check the paper. It seems the entropy is over the prior. So it's a constant, and we can remove that.

I haven't tried that. But the paper indicates that we use different network for different c_i, then compute the MI lower bound for each, and add (or average) them together.


DO NOT MERGE: Add InitalDelayMs to function spec and use in sidecar

Addresses #572

Closing in favor of #577


Fix Directory Structure

Its kinda weird to have a repo with all the real files one level down.

@say25 you don't even go here

:(

Wait Steven, don't change the file structure yet. Otherwise, Matt's going to have more trouble merging his stuff. I was planning on fixing file structure after he merged his stuff

Yea I figured you guys should do it. I don't want to mess you up thats why I deleted my branch and just made an issue


Remove all Systems with PooledEngine

Hey everybody.Is possible remove all systems with PooledEngine?

I miss this possibility as well, but at this moment I don't think it's possible with a single call.
I use this:
    final ImmutableArray<EntitySystem> systems = engine.getSystems();
    while(systems.size() > 0) {
        engine.removeSystem(systems.first());
    }


Some minor simplifications & optimizations

Thanks @MaxGraey - really appreciate this. I'll take a look over these shortly, make sure i full digest them

Thanks @MaxGraey - really appreciate this. I'll take a look over these shortly, make sure i full digest them


Authentication using oidc

Feature Request
Is your feature request related to a problem? Please describe.
When developping my operator, I need to connect using my local kube/config file.
To authenticate on our kubernetes cluster we are using Dex so the oidc authentication, but it panic :
panic: No Auth Provider found for name "oidc"

Describe the solution you'd like
I found a workaround to my problem by adding this line in the vendor/operator-framework/operator-sdk/pkg/k8sclient/client.go :
_ "k8s.io/client-go/plugin/pkg/client/auth/oidc"

Is it possible to add this line in the official release ? or is there a way I can configure this elsewhere ?
thanks

@allamand  I'm not positive this will work, but have you tried adding that import line to your main.go?
Here's an example of a similar import (albeit with the latest SDK version that uses controller-runtime): https://github.com/operator-framework/operator-sdk-samples/blob/4c6934448684a6953ece4d3d9f3f77494b1c125e/memcached-operator/cmd/manager/main.go#L12

Thanks, @joelanford for replying. Unfortunately, I already try to put this in my main.go but it doesn't work.
I've not yet migrated on the latest SDK, maybe the behavior is different then, since the file I referenced above seams to not exists in new version : https://github.com/operator-framework/operator-sdk-samples/tree/4c6934448684a6953ece4d3d9f3f77494b1c125e/memcached-operator/vendor/github.com/operator-framework/operator-sdk/pkg

@hasbro17 Any ideas on this?

@allamand The SDK has no longer has pkg/k8sclient. It uses the controller-runtime's client now.
The No Auth Provider is a common issue since the plugins aren't included by default and you have to import them like you did. We had to do the same for the gcp plugin a while back #376 (although I just realized that needs to be added to the main.go scaffold again after the refactoring).
However I'm surprised it isn't working when you import _ "k8s.io/client-go/plugin/pkg/client/auth/oidc" in main.go. That SDK version shouldn't really matter.
I'll need to test out this out to see why this isn't working because if we support these plugins by default I'm not sure where else we would import them for v0.1.0.

Yeah the gcp auth plugin works fine when included in main.go.
And it panics if I remove _ "k8s.io/client-go/plugin/pkg/client/auth/gcp" from main.go run the operator out of cluster against GKE:
$ operator-sdk up local
2018/11/07 11:45:37 Go Version: go1.10.2
2018/11/07 11:45:37 Go OS/Arch: darwin/amd64
2018/11/07 11:45:37 operator-sdk Version: 0.1.1+git
panic: No Auth Provider found for name "gcp"
...

Not sure how to test out the oidc case though.

@hasbro17 Should we add this to documentation (about adding an auth plugin) rather than adding them to the scaffold?

@shawn-hurley Yes we should add a note somewhere(probably in the up local section) to import any necessary auth plugins in main.go in order to run out of cluster.
https://github.com/kubernetes/client-go/tree/master/plugin/pkg/client/auth

Hello,
I made new try and found that it works in the main.go, but the order import is important :
this don't works :
        "github.com/operator-framework/operator-sdk/pkg/sdk"
	_ "k8s.io/client-go/plugin/pkg/client/auth/oidc"

this order works :
	_ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
	"github.com/operator-framework/operator-sdk/pkg/sdk"

The bad thing is that my goland IDE plugin automatically change the order of my imports and put it in the order where it don't works :(

@allamand We ended up adding all of the client-go auth plugins in #1199, which will be included in our next release, so this should be fixed.


changed the link in overview.md line 28 from See the [ documentation](…

…../api/classes/movie.html) to See the  documentation.
any suggestions or edits  i should make ?

this is a demo PR    to confirm that this way  is the right way of fixing, what do u think ?

Thanks again for contributing! Looks good.
I'm going to rebase and update the commit message (I like to keep the first line a brief summary, followed by two newlines, followed by an optional longer description).

It looks like next time I need to press merge from the web UI instead of the command line for it register haha


[buddy] remove redundant case

The stripmining.mlir case is an old handwritten case as a workaround for the lack of the lower-rvv pass. But now we have stripmining-huge.mlir, which serves the same purpose but has better implementation than the former case, so we don't need the stripminig.mlir case anymore.

LGTM


[Travis] Don't allow php7 to fail anymore

Would be nice to get new tag with this and at least #266 to be able to do the same on StashBundle in https://github.com/tedious/TedivmStashBundle/pull/88.

I was hoping to push out v0.14 before I left on vacation but didn't manage. I got back yesterday. My plan is to have another release this week, potentially in the next couple of days.

Ok
But FYI master + this PR as 0.13.2 here will unblock PR to have php7 green tests in the bundle to be able to do a new patch release there.

Good point.


add stale action

Pull Request Test Coverage Report for Build 1861

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 1852:
0.0%

Covered Lines:
1102

Relevant Lines:
1102

💛  - Coveralls


Write Access

Hi @simoncozens. I was trying to make a branch to the project test out some things and load language profiles. Could I have write access to the repo?
Thanks

You know you can always make a fork of any public GitHub repository without anybody's authorization..

Yep, just hit "fork" and then you can send me pull requests!


chore(ci): Configure sccache to use local caches

Description
Because we use self-hosted runners we do not want the sccache action
to query the github action cache service for each compilation request.
Instead it should be using the local cache.
This means the first time something is compiled on a runner it will
not yet be in the cache, even if another runner already did compile
this.  However because the cache size is set reasonably large over
time all the common dependencies will end up cached on each
self-hosted runner.
Maybe it will be possible to run a self-hosted caching service
sometime, but for now relying on a large local cache is expected to be
reasonable.
Notes & open questions

This will not start showing results right away, as caches on the
self-hosted runners need to be filled.  They are full by now though
and show improvements across the board.

Please check that all local runners have 100G space for a cache in
~/.cache/sccache or %LOCALAPPDATA%\Mozilla\sccache.

Change checklist

[x] Self-review.
Documentation updates if relevant.
Tests if relevant.

we could probably easily half the size of the cache based on current numbers

Can I take you up on the offer to drop the cache to something like 60G, the arm runner has a 100G disk in total. All the others are fine