[d3d11] Zenless Zone Zero Zooting Ze VRAM

Zenless Zone Zero (not a Steam-published game but hey) seems to be suffering from a VRAM leak. Whether it's from upstream or a DXVK quirk, I can't be sure, but Windows players seem to be unaffected, or at least affected far more slowly. Comparatively, there have been numerous reports of both AMD/ATI and nVidia users getting game crashes 30 minutes to an hour into a gameplay session. I myself have been suffering these slowdowns, so much so that at the mere HINT of frames being dropped, I close the game and restart it to be able to play another relatively stable bout of 30 minutes.
Before anyone wonders about the issue title, I was trying to go for maximum alliteration. Basically, ZZZ is "oversubscribing" the VRAM and it seems as if it's assuming more memory is available than there is. Could be due to some DX11 APIs related to resource loading that have implementation quirks, i dunno.
Software information
Game: Zenless Zone Zero
Type: ARPG (Gacha, F2P - can be installed without paying)
Core Engine: Unity 2019.4.40 (ultra-customized, most likely)
Settings: High on everything. (No ultra available)
Resolution: 3840x1600 (ultrawide 21:9)
System information

GPU: RTX 3080 10gb
Driver: 555.58.02
Wine version: GE-Proton 9-10
DXVK version: 2.3.1-120-gfe9c---
Desktop: KDE5 on Wayland

Apitrace file(s)

None for now

Log files

Proton logfile of a run that had a slowdown at the end: steam-18233669865720250368.log

Unity Engine games are kinda known to load all resources into VRAM regardless of the GPU in use, this is generally not a leak. DXVK's memory management also cannot be improved due to some fundamental design flaws.

Fair point, but usually DXVK itself doesn't come close to the VRAM limit. For that matter, the game looks like it's averaging at 5.5~gb (for now), with occasional peaks that then go down after a bit (when they're not crashing the system). I'm guessing that everything else running in the background (composition, accelerated desktop, etc) takes precious resources away from DXVK, but still.
Is there an obvious way to limit the total allowed vidmem heap that I'm missing, or is this another one of these "can't be solved unless/until the dev stops being rude to memory" issues (which, per your statement, it certainly might be)?

Me neither, blows up to 60% of my vram on my 3070ti xd so 8gig gpu's are out of the question.

By setting DXVK_CONFIG="dxgi.emulateUMA = True" can fix this issue.
Seems that Windows version game uses UMA as default.

That option did little for me, @kamikaze211.
However, update 1.1 actually stabilized the game, by and large. Seems that DXVK may have "merely" exacerbated issues present for Windows players.
Closing for now.

This is very interesting.
I have a theory after reading the statement that the game tend to use a lot of video memory than usual. I do think that the game may just use this tactic to "optimise" the game instead of manually managing memory manually. So tried every single way to "lie" to the game that I'm starving VRAM with every DXVK configs I could think of
DXVK_HUD=memory DXVK_CONFIG="dxgi.emulateUMA = True; dxgi.maxDeviceMemory = 2; dxgi.maxSharedMemory = 0; d3d11.zeroWorkgroupMemory = True; dxvk.halveNvidiaHVVHeap = True;" %command%

I don't know which options do make a difference, but after this, the game now tries to be very, very conservative on VRAM usage instead lol. It always kicked off to full VRAM usage before.

Be warned, these launch options cause significantly more stutter than usual, but it no longer runs out of video memory.

This is very interesting.
[...]
DXVK_HUD=memory DXVK_CONFIG="dxgi.emulateUMA = True; dxgi.maxDeviceMemory = 2; dxgi.maxSharedMemory = 0; d3d11.zeroWorkgroupMemory = True; dxvk.halveNvidiaHVVHeap = True;" %command%

Very interesting indeed. With that config - and the fact that 1.1 also drove down the OOM issues quite a bit on my RTX3080 - the memory overuse SEEMS to be getting mitigated QUITE A BIT. I've not broken 8gb total in my validation, whereas booting into the game, warping to Scott Outpost and merely walking into the tent gets me to 8gb+ total VRAM (5.ish GB to ZZZ alone).
VERY nice find, @SysError99. Much appreciated.

Heads up: we're gonna remove that option in the next version.

Heads up: we're gonna remove that option in the next version.

which one? nVidia HVV heap or ZeroWorkgroup?

I got a conclusion, there's two options that need to be set in order to lie the game (at least, on my machine), dxgi.enableUMA, and dxgi.maxDeviceMemory. The dxgi.maxDeviceMemory can be set to any low value but zero, and the game now behaves properly with low memory graphics cards.
DXVK_HUD=memory DXVK_CONFIG="dxgi.emulateUMA = True; dxgi.maxDeviceMemory = 1;" %command%

Yes..., the option that happened to be removed in further versions (https://github.com/doitsujin/dxvk/commit/eb8d1885bde916b3476229a9220952ad0f639ff9) happens to be the option that's crucial for this workaround to work...

I did surf the dxgi_adapter.cpp a little bit, it seems like this is all that the option dxgi.emulateUMA does.
if ((m_adapter->isLinkedToDGPU() && deviceProp.deviceType == VK_PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU) ||
    options->emulateUMA) {
    sharedMemory = std::max(sharedMemory, deviceMemory);
    deviceMemory = 512ull << 20;
}

The deviceMemory must be set to anything in absurd value in order to get it working (in this case, 512 << 20 or 536870912). So I did try to set the maxDeviceMemory to an absurd value, but this code section blocked me away to do it...
// Some games are silly and need their memory limited
if (options->maxDeviceMemory > 0
 && options->maxDeviceMemory < deviceMemory)
    deviceMemory = options->maxDeviceMemory;

if (options->maxSharedMemory > 0
 && options->maxSharedMemory < sharedMemory)
    sharedMemory = options->maxSharedMemory;

I think we just need to remove true max memory limit to allow its users to freely set any values into it. We just need to warn them that it can be risky and may crash their machine, like below.
// Some games are silly and need their memory limited
if (options->maxDeviceMemory > 0)
    deviceMemory = options->maxDeviceMemory;

if (options->maxSharedMemory > 0)
    sharedMemory = options->maxSharedMemory;

Never freaking mind, I read the code thoroughly again, and all I need to do is to set both maxDeviceMemory and maxSharedMemory to the lowest possible lol.
DXVK_HUD=memory DXVK_CONFIG="dxgi.maxSharedMemory = 1; dxgi.maxDeviceMemory = 1;" %command%

Case closed!

The deviceMemory must be set to anything in absurd value in order to get it working (in this case, 512 << 20 or 536870912), and the sharedMemory must be very low value.

The in code value is in bytes which why the number is so big. deviceMemory = 512ull << 20;  is just 512MB.
In the 2.4 release the only thing the dxgi.enableUMA option does is the equivilant of setting dxgi.maxDeviceMemory to 128 and dxgi.maxSharedMemory to the amount of vram you GPU has.
Now as you showed above the exact numbers changed a but on master before the option got pulled but the concept is more or less the same.
While i was typing this i see you figured out it wasn't a high number anyway.
The reason setting something like dxgi.maxDeviceMemory to a low number is that you are telling the game that you have less vram than you actually have (1 being 1MB) and if the game cares about that it might try and be more careful with it's usage.
When i tested it ingame it seems like it might be more aggressive in freeing vram again instead of keeping it around. Also maybe pulling less in to it to begin with? Not completely sure.


Offline installation and use request

Please include deployment instructions and usage for installation on systems that do not have access to the commercial internet. i.e. closed network systems with air gap file transfer capabilities.

Great idea F5Rob, thanks for raising that, we will do it.

Filled internal SOLUTIONS-6783

resolved with version 1.0.1


PP-265: Update decisions index

To test:

Checkout branch
You might have to edit the search cluster config and settings.php to get the local index to work:

In conf/cmi/elasticsearch_connector.cluster.paatokset.yml change line 7 to url: 'http://elastic:9200'
In public/sites/default/settings.php comment out all settings that start with $config['elasticsearch_connector.cluster.paatokset']

Run make new (create database dump beforehand to store your current situation)
SSH into the container and run: drush ap:fs;drush mim ahjo_cases:latest;drush mim ahjo_decisions:latest
Check that the decisions list has values: https://helsinki-paatokset.docker.so/fi/admin/content/decisions

Filter by the "Decision date" heading and make sure all decisions have some value. Everything should atl east have the default date 2001-01-01

Check the Search API settings and make sure the cluster works: https://helsinki-paatokset.docker.so/fi/admin/config/search/search-api
Check that the new indexing options are in use: https://helsinki-paatokset.docker.so/fi/admin/config/search/search-api/index/decisions/edit

This should now index decisions nodes and not the custom agenda_item entities
Go through the field settings and see if anything relevant is missing

Index the content: https://helsinki-paatokset.docker.so/fi/admin/config/search/search-api/index/decisions

Delete all indexed content, rebuild tracking information and then queue everything for reindexing
You might have to delete and rebuild tracking information again before the indexing works

Run the react search app locally (follow instructions here: https://github.com/City-of-Helsinki/paatokset-search/)

Make sure all the data is valid and that the search works as intended with the new index

Functionally this seems great. Some notes:

We should probably index the 'content' field of decisions as well, so we can get matches from this text content as well. This field should have less boost than the titles - 1 should be fine for now.
Field_meeting_date is better off as date field rather than string

Latter of the changes will cause issues with the current version of the app, but I have a fix for it ready in a branch.

Suggestions in comment above ☝

Merging this now, adding the suggestions in https://github.com/City-of-Helsinki/helsinki-paatokset/pull/123


Implement quantum circuits in frontend

We could use https://quantumjavascript.app/Q-Circuit.html

The textual output format seems like something sufficient to construct the corresponding quantum circuit in the Python code.
There is still a question on detectors and how they should be implemented, but this package seems to be sufficient to handle at least the quantum circuit part.

I think this is part of tasks 4+.

Could we look at qni instead? It's based on Quirk and at least has a npm package available.
I was interested in doing this task, but I would need some pointers on how to include any of the alternatives.
As I see it q.js is pure javascript and renders div elements; qni uses typescript to render svg elements.
For a quick solution it would be fine to use one of those, but wouldn't it make more sense to use a pixy for this?
Especially since we don't need the simulation capabilities, and we could also define the output based on our needs

Could we look at qni instead? It's based on Quirk and at least has a npm package available. I was interested in doing this task, but I would need some pointers on how to include any of the alternatives. As I see it q.js is pure javascript and renders div elements; qni uses typescript to render svg elements. For a quick solution it would be fine to use one of those, but wouldn't it make more sense to use a pixy for this? Especially since we don't need the simulation capabilities, and we could also define the output based on our needs

@Gistbatch my intuition is that we could treat qni as a sub-module within the frontend code and invoke whichever components we need on-demand. React and Pixi already provide the HTML components, so we'd only need to invoke qni's logic through JavaScript.

Marking as redundant


Dependency update

Needed for gulp tree deduping

Done and published as 2.0.0

It's time again (for micromatch to be updated to v4)


Confirm organic referrers work correctly

Seems like only Yandex and Google are being classed as organic correctly right now (from https://sbjs.rocks, after searching for sourcebuster on Bing and Yahoo):

AFAIK they should both be organic:
https://github.com/woocommerce/sourcebuster-js/blob/411d8dffaa37d7602d678c2ec02baacee93c12d5/src/js/params.js#L82-L83

Problem also originally reported here: https://github.com/alex35mil/sourcebuster-js/issues/41


Add FAQ as separate document. Removed from RST docs.

Dear reviewers: Most of the content is moved the the RST documents, with RST-> MD edits, and other light copy-editing. Hopefully, there shouldn't be technical errors in that (apart from obsolete info).
However, there are also sections that are new (or substantially new) that you should check are factually correct as well as style, etc:

Supported Python versions (removed Python 2 versus 3 discussion, and rewrote)
Relationship with kivy3, kivy4, kivy5, kivyMD
Internationalization section

Future work:

Add references to contact us.
Some of the historical answers (Why Python? Relationship with PyMT) are way too long for an FAQ.

Maintainer merge checklist

[ ] Title is descriptive/clear for inclusion in release notes.
[ ] Applied a Component: xxx label.
[ ] Applied the api-deprecation or api-break label.
[ ] Applied the release-highlight label to be highlighted in release notes.
[ ] Added to the milestone version it was merged into.
[ ] Unittests are included in PR.
[ ] Properly documented, including versionadded, versionchanged as needed.

Okay, I didn't actually mean to update the Contribution Guide in the same branch, and hence into the same PR and same review as the FAQ, but... well, it is done now.

Thanks for the eagle eyes. Fixes made.

Now bigger than ever before, with #8450 included.

@Julian-O we have a conflict 😬, can you take a look?

Fixed readme error. Rebased.

PS: maybe we can add links to CONTACT.md, CONTRIBUTING.md, FAQ.md here?

I would love to have done that, but could not work out how.
I tried reading up on the ReadTheDocs theme, but couldn't see any parameters. Sphinx isn't my speciality, so open to suggestions

@misl6 All done. Fingers crossed.

Sorry, there were some problems with checking the code. I closed the old reviews and opened the new ones.

7th time is the charm!


I want to add a project called "Snakes and Ladder", implemented in Java.

 How different classes work together to form a game.

Try to include it. You won't regret it.

Sure

Sorry you closed it..before I submitted. Please try to open it again


--window-borderless not working on macOS, scrcpy version 1.17

[x] I have read the FAQ.
[x] I have searched in existing issues.

Environment

OS: macOS 10.15.7 (Catalina) and 10.14.6 (Mojave)
scrcpy version: 1.17_1
installation method: brew
device model: Galaxy Note 9, others
Android version: 10

Describe the bug
Passing --window-borderless to version 1.16 correctly showed a window with no title bar.
Since update to 1.17 the title bar appears despite --window-borderless.
As an experiment, I tried building with an additional line in screen_init_rendering:
    SDL_SetWindowBordered(screen->window, !window_borderless);

This change resulted in a restored control over the window border.

Please post screenshots of both v1.16 and v1.17. I guess it's due to some SDL change.

This is v1.17_1 showing the command to launch adjacent to the problematic window on macOS Catalina.

I don't have 1.17 or 1.16_4 installed anymore and cannot easily roll back the version (looks like I'd have to create a tap to extract into...dunno). Basically, the screenshot would have no title bar showing.

Is reproducible
OS: macOS Big Sur 11.2.3
scrcpy version: 1.17
Installation method: brew
Device: Essential Products PH-1 (Android 10)

Thank you for your screenshots.

As an experiment, I tried building with an additional line in […]

Could you please try building v1.16 or even earlier versions and test?
You mentioned that it worked in v1.16 but not in v1.17, but I suspect that it may be related to some independent SDL upgrade. This would help to determine the cause.
Thank you.

Ok, got the screenshot. And the error remains in 1.16.

As suspected, the issue is in SDL. Which makes sense, because nothing about how scrcpy requests a borderless window has changed between 1.16 and 1.17.

👍 Could you report to SDL, please?


DDPG implementation fails to learn well on at least five MuJoCo-v2 envs for all three noise types. I report steps to reproduce and learning curve plots [and show that PPO2 seems to work fine].

Dear @pzhokhov  @matthiasplappert @christopherhesse  et al.,
Thank you for providing an implementation of DDPG. However, I have been unable to get it to learn well on the standard MuJoCo environments by running the provided command in the README (and with related commands). Here are the steps to reproduce. I apologize for the length of the post, but I want to show what I tried to reduce ambiguity and to potentially counter the potential argument that it might be due to bad hyperparameters.
First, here's the machine I am using with relevant versions of software:

Ubuntu 18.04
MuJoCo 2.0
Create a clean Python 3.6.7 virtualenv and install all the required stuff with pip install commands. I'm using TensorFlow 1.13, gym 0.12.1, and mujoco-py <IP_ADDRESS>. All appear to be installed correctly and show no signs of error.
 I'm splitting these into three groups based on the three types of noise we can inject into our policy.
Group 1: Parameter Noise
I first decided to take the default command provided in the README because I assumed that hyperparameters here have been tuned to save users the time and compute needed for expensive hyperparameter sweeps.
python -m baselines.run --alg=ddpg --env=Ant-v2 --num_timesteps=1e6
python -m baselines.run --alg=ddpg --env=HalfCheetah-v2 --num_timesteps=1e6
python -m baselines.run --alg=ddpg --env=Hopper-v2 --num_timesteps=1e6
python -m baselines.run --alg=ddpg --env=Swimmer-v2 --num_timesteps=1e6
python -m baselines.run --alg=ddpg --env=Walker2d-v2 --num_timesteps=1e6

I use my plotting code to get plots. Here it is:
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
plt.style.use('seaborn-darkgrid')
import argparse
import csv
import pandas
import os
import sys
import pickle
import numpy as np
from os.path import join

# matplotlib
titlesize = 33
xsize = 30
ysize = 30
ticksize = 25
legendsize = 25
error_region_alpha = 0.25

def smoothed(x, w):
    """Smooth x by averaging over sliding windows of w, assuming sufficient length.
    """
    if len(x) <= w:
        return x
    smooth = []
    for i in range(1, w):
        smooth.append( np.mean(x[0:i]) )
    for i in range(w, len(x)+1):
        smooth.append( np.mean(x[i-w:i]) )
    assert len(x) == len(smooth), "lengths: {}, {}".format(len(x), len(smooth))
    return np.array(smooth)

def _get_stuff_from_monitor(mon):
    """Get stuff from `monitor` log files.

    Monitor files are named `0.envidx.monitor.csv` and have one line for each
    episode that finished in that CPU 'core', with the reward, length (number
    of steps) and the time (in seconds). The lengths are not cumulative, but
    time is cumulative.
    """
    scores = []
    steps  = []
    times  = []
    with open(mon, 'r') as csv_file:
        csv_reader = csv.reader(csv_file, delimiter=',')
        line_count = 0
        for csv_row in csv_reader:
            # First two lines don't contain interesting stuff.
            if line_count == 0 or line_count == 1:
                line_count += 1
                continue
            scores.append(float(csv_row[0]))
            steps.append(int(csv_row[1]))
            times.append(float(csv_row[2]))
            line_count += 1
    print("finished: {}".format(mon))
    return scores, steps, times

def plot(args):
    """Load monitor curves and the progress csv file. And plot from those.
    """
    nrows, ncols = 1, 2
    fig, ax = plt.subplots(nrows, ncols, squeeze=False, sharey=True, figsize=(11*ncols,7*nrows))
    title = args.title

    # Global statistics across all monitors
    scores_all = []
    steps_all = []
    times_all = []
    total_train_steps = 0
    train_hours = 0

    monitors = sorted(
        [x for x in os.listdir(args.path) if 'monitor.csv' in x and '.swp' not in x]
    )
    progfile = join(args.path,'progress.csv')

    # First row, info from all the monitors, i.e., number of CPUs.
    for env_idx,mon in enumerate(monitors):
        monitor_path = join(args.path, mon)
        scores, steps, times = _get_stuff_from_monitor(monitor_path)

        # Now process to see as a function of episodes and training steps, etc.
        num_episodes = len(scores)
        tr_episodes = np.arange(num_episodes)
        tr_steps = np.cumsum(steps)
        tr_times = np.array(times) / 60.0 # get it in minutes

        # Plot for individual monitors.
        envlabel = 'env {}'.format(env_idx)
        sm_10 = smoothed(scores, w=10)
        ax[0,0].plot(tr_steps, sm_10, label=envlabel+'; avg {:.1f} last {:.1f}'.format(
                np.mean(sm_10), sm_10[-1]))
        sm_100 = smoothed(scores, w=100)
        ax[0,1].plot(tr_times, sm_100, label=envlabel+'; avg {:.1f} last {:.1f}'.format(
                np.mean(sm_100), sm_100[-1]))

        # Handle global stuff.
        total_train_steps += tr_steps[-1]
        train_hours = max(train_hours, tr_times[-1] / 60.0)

    # Bells and whistles
    for row in range(nrows):
        for col in range(ncols):
            ax[row,col].set_ylabel("Scores", fontsize=30)
            ax[row,col].tick_params(axis='x', labelsize=25)
            ax[row,col].tick_params(axis='y', labelsize=25)
            leg = ax[row,col].legend(loc="best", ncol=1, prop={'size':25})
            for legobj in leg.legendHandles:
                legobj.set_linewidth(5.0)
    ax[0,0].set_title(title+', Smoothed (w=10)', fontsize=titlesize)
    ax[0,0].set_xlabel("Train Steps (total {})".format(total_train_steps), fontsize=xsize)
    ax[0,1].set_title(title+', Smoothed (w=100)', fontsize=titlesize)
    ax[0,1].set_xlabel("Train Time (in Hours {:.2f})".format(train_hours), fontsize=xsize)
    plt.tight_layout()
    figname = '{}.png'.format(title)
    plt.savefig(figname)
    print("\nJust saved: {}".format(figname))

if __name__ == "__main__":
    pp = argparse.ArgumentParser()
    pp.add_argument('--path', type=str)
    pp.add_argument('--title', type=str)
    args = pp.parse_args()
    plot(args)

To use this code, just run python [script].py --path [PATH] --title [TITLE]. Feed in the path to the 0.0.monitor.csv file (i.e., that's the /tmp/openai-[DATE] directory) and some title. I did this for all five environments above and got these results:

None of these curves appear to be getting better than random performance. Maaaaaybe Ant-v2 is getting better than random performance, but it seems to be stuck at 0 and many, many papers report values far above 0. Perhaps it has something to do with the number of environments? I briefly tried increasing the number of parallel environments to 8 but that did not seem to work:
python -m baselines.run --alg=ddpg --env=Ant-v2 --num_timesteps=1e6 --num_env=8
python -m baselines.run --alg=ddpg --env=HalfCheetah-v2 --num_timesteps=1e6 --num_env=8

Incidentally, it seems like having N environments means that the actual number of steps increases by a factor of N. This is different behavior from PPO2 where increasing N does not change the number of actual time steps total at the end; increasing N for PPO2 means each individual environment can execute fewer steps.
PS: for some of the above plots, I did not run to exactly 1M steps, i.e., I terminated it near the end if it was clear that the algorithm was not learning well.
Group 2: Gaussian Noise
All right, next I decided to avoid parameter space noise. In the TD3 paper which used DDPG, the authors used Gaussian noise with standard deviation 0.1. I decided to try that, keeping all other settings fixed:
python -m baselines.run --alg=ddpg --env=Ant-v2 --num_timesteps=1e6 --noise_type=normal_0.1
python -m baselines.run --alg=ddpg --env=HalfCheetah-v2 --num_timesteps=1e6 --noise_type=normal_0.1
python -m baselines.run --alg=ddpg --env=Hopper-v2 --num_timesteps=1e6 --noise_type=normal_0.1
python -m baselines.run --alg=ddpg --env=Swimmer-v2 --num_timesteps=1e6 --noise_type=normal_0.1
python -m baselines.run --alg=ddpg --env=Walker2d-v2 --num_timesteps=1e6 --noise_type=normal_0.1

Here are the results:

Once again, it seems like there is no learning happening. The performance appears to be similar to the parameter space noise case.
Group 3: OU Noise (along with tau=0.001)
I decided to run one last batch of commands, this time with the original OU noise. After carefully checking the TD3 paper, and the DDPG directory from the July 27, 2017 commit when DDPG was first released, I saw that the tau parameter back then was set at 0.001. Now for some reason it is 0.01. DeepMind used 0.001 so I decided to try OU noise with tau 0.001. This appears to be the only hyperparameter difference that I can see from this code base and the values used by DeepMind.
python -m baselines.run --alg=ddpg --env=Ant-v2 --num_timesteps=1e6  --noise_type=ou_0.2 --tau=0.001
python -m baselines.run --alg=ddpg --env=HalfCheetah-v2 --num_timesteps=1e6 --noise_type=ou_0.2 --tau=0.001
python -m baselines.run --alg=ddpg --env=Hopper-v2 --num_timesteps=1e6 --noise_type=ou_0.2 --tau=0.001
python -m baselines.run --alg=ddpg --env=Swimmer-v2 --num_timesteps=1e6 --noise_type=ou_0.2 --tau=0.001
python -m baselines.run --alg=ddpg --env=Walker2d-v2 --num_timesteps=1e6 --noise_type=ou_0.2 --tau=0.001

Results:

(The swimmer curve looks like it's going up, but the reward is lower as compared to the other two plots.)
The results I am getting seem to differ from the blog post here which shows HalfCheetah rewards of at least +1500, and much larger depending on the parameter noise setting, and for 2M steps. It might be a hyperparameter issue, but I'm not sure. In particular, notice that the hyperparameters here (for the most part) match those from the DDPG or TD3 papers.
The TD3 paper reports these results:

The TD3 paper says it used DDPG (presumably from OpenAI baselines as of late 2017?) and then "Our DDPG" above is when the author tuned hyperparameters. Both get final rewards that are far higher than what I am seeing, and we are all using 1M training steps here. Unfortunately, from reading the TD3 code base, it is not clear which commit from baselines was used for the results.
The paper above does not report results for Swimmer, so I looked at the "Benchmarking DeepRL" paper, which says DDPG on Swimmer should get 85 +/- 1.8, and this is far higher than the Swimmer results I am getting above.
I suspect that there must be have been some change to the code that caused it to somehow either stop working well or be exorbitantly sensitive to hyperparameters? For example, maybe the process of removing MPI caused some unexpected results? Or it could be due to MuJoCo environments v1 to v2, since the TD3 paper used MuJoCo v1 environments, but as this report suggests, RL performance should be similar. Notice that all the reward curves there for PPO show increasing reward, whereas I'm just seeing stagnation and noise for DDPG.
This is perhaps relevant to the following issue reports:

https://github.com/openai/baselines/issues/913
https://github.com/openai/baselines/issues/855
https://github.com/openai/baselines/issues/764
https://github.com/openai/baselines/issues/690
https://github.com/openai/baselines/issues/482

all of which have noticed issues with DDPG. If the fix is found, then the above can probably all be closed.
Hopefully in the spirit of my previous report on DQN here, we can resolve this issue together. Does anyone have any general suggestions or ideas about the potential causes? At this point I am unable to confidently use the DDPG code because it does not pass standard benchmarks. My previous issue report about DQN suggests that it could be an environment processing issue. Is the code processing the MuJoCo environments in a simliar way as in July 2017? Do the PPO2 results apper to be fine, but the DDPG results off? Is there a difference with how the two algorithms process observations and normalize data?
I'm happy to help investigate this if you have ideas on what might be the root cause. I only report this issue because having highly tuned algorithms and hyper-parameters ready to go "off-the-shelf" greatly helps the entire research community by accelerating research cycles and reducing the need to write our own error-prone implementations of algorithms.
Thanks!

PPO2 Results as a Sanity Check
To confirm that something related to DDPG is the issue (which might include DDPG-specific processing steps) I ran these commands for PPO2, using the same master branch commit as above. These commands:
python -m baselines.run --alg=ppo2 --env=Ant-v2 --num_timesteps=1e6
python -m baselines.run --alg=ppo2 --env=HalfCheetah-v2 --num_timesteps=1e6
python -m baselines.run --alg=ppo2 --env=Hopper-v2 --num_timesteps=1e6
python -m baselines.run --alg=ppo2 --env=Swimmer-v2 --num_timesteps=1e6
python -m baselines.run --alg=ppo2 --env=Walker2d-v2 --num_timesteps=1e6

Resulted in the following learning curves:

Which were generated via  a similar plotting script as I used in my post above.
These look much better!

Ant is exceeding 1000, actually it seems better than TD3 paper results
HalfCheetah is getting around 1500, roughly matching TD3 paper results
Hopper is getting around 2000 as expected.
Swimmer is exceeding 100 points.
Walker2d is getting around 1500, slightly lower than reported results from TD3 but still way better than earlier.

It therefore seems like something is wrong with DDPG.

Hello,
was the normalize_obsset  to True? (seems like the default, but I would rather double check)
Otherwise, the observation get clipped  And how many random seeds did you try?
You can find hyperparams (and trained agents) for pybullet envs here (in the rl zoo)
NOTE: it uses stable baselines version, but should be the same underlying algorithm.

@araffin
The normalize observation seems to be True here:
https://github.com/openai/baselines/blob/ba2b01782034bcbdb73a2e744cff4cb1c99ab612/baselines/ddpg/ddpg.py#L32

A question about your experiments with OU Noise only:
The OU noise does not use tau. It uses the another parameter called dt. Where do you read that dt and tau are the same? tau is the soft update parameter of the target networks, which is 0.001 in the Baseline code and also in the Deepmind paper. dt is another parameter only for the OU noise. In the Deepmind paper dt doesn't exist.
But this is only a side question. I don't think, that will solve your problem at all. But this clears, that there seems to be no difference in the hyperparameters of the Deepmind paper and the code from Baselines, when using OU noise only.

@MoritzTaylor
Just to be clear, the tau variable indeed is the soft update. I only changed it from 0.01 to 0.001 because that is what the DeepMind paper used and I wanted to pick some noise setting to go with it (OU, normal, or param noise). It does not have anything to do specifically with the OU noise. So perhaps I should have run a fourth trial above, with OU noise but with tau 0.01. It is not a big deal.
I agree that it will not solve DDPG's current performance issues.

As side note, in case you're interested in quickly trying something out before this issue gets resolved, I would highly recommend the TD3 author's official implementation (which is in pytorch though). Clean stand alone interface, and I benchmarked it a month or so back and it matched with the paper result.

@sritee  Good point. :) I am actually benchmarking with rlkit right now https://github.com/vitchyr/rlkit which seems to be similar to the original author's implementation.

Hello,
I have been doing a quick sanity check using the rl zoo.
To reproduce, add that to hyperparams/ddpg.yaml:
Gaussian noise:
HalfCheetah-v2:
  n_timesteps: !!float 1e6
  policy: 'MlpPolicy'
  gamma: 0.99
  memory_limit: 1000000
  noise_type: 'normal'
  noise_std: 0.2
  batch_size: 64
  normalize_observations: True
  normalize_returns: False

Param noise:
HalfCheetah-v2:
  n_timesteps: !!float 1e6
  policy: 'LnMlpPolicy'
  gamma: 0.99
  memory_limit: 1000000
  noise_type: 'adaptive-param'
  noise_std: 0.2
  batch_size: 64
  normalize_observations: True
  normalize_returns: False

Command to run (with tensorboard support):
python train.py --algo ddpg --env HalfCheetah-v2 -tb /tmp/ddpg/

With one random seed, 100k steps on HalfCheetah-v2, I am getting those results (gaussian noise is orange, param noise is blue):

So it seems something wrong happened with the original baselines (stable baselines is based on OpenAI Baselines of last year).

@MoritzTaylor
Just to be clear, the tau variable indeed is the soft update. I only changed it from 0.01 to 0.001 because that is what the DeepMind paper used and I wanted to pick some noise setting to go with it (OU, normal, or param noise). It does not have anything to do specifically with the OU noise. So perhaps I should have run a fourth trial above, with OU noise but with tau 0.01. It is not a big deal.
I agree that it will not solve DDPG's current performance issues.
Note that tau by default is 0.01:
https://github.com/openai/baselines/blob/ba2b01782034bcbdb73a2e744cff4cb1c99ab612/baselines/ddpg/ddpg.py#L42
The above will override the 0.001 from the DDPG code here:
https://github.com/openai/baselines/blob/ba2b01782034bcbdb73a2e744cff4cb1c99ab612/baselines/ddpg/ddpg_learner.py#L66-L71

Ok, unfortunately I did not see that tau is set to 0.01 in the learn function. Of course this overwrites all other tau params.
But since this tau occurs regardless of the type of noise, you should always set tau to 0.001 manually, or not? And neither in the parameter noise setup nor in the gaussian noise setup you set tau to 0.001 manually, or am I missing something again?

@MoritzTaylor Right, if this were a research paper, I would always keep tau = 0.001, or always keep it at 0.01 to keep experimental conditions the same while varying just one thing (the noise setting, if that was what I was investigating, but really I was just trying to see what setting would lead to any improvements in performance). I only changed it here because I thought I might as well at least try to see what happens.
@araffin  Thanks for the results. It does seem like something happened between the commit corresponding to stable-baselines and the commit corresponding to the most recent one on master, that I used to test. Out of curiosity do you have the exact commit for baselines here that corresponds to what stable-baselines uses?

@DanielTakeshi

Out of curiosity do you have the exact commit for baselines here that corresponds to what stable-baselines uses?

there was several changes/bug fixes afterward, but we forked it (apparently) from:
https://github.com/hill-a/stable-baselines/commit/a6b1bc70f156dc45c0da49be8e80941a88021700

Thanks, maybe something happened between then that caused changes in the environment processing code? From reading OpenAI's DDPG code, I can't find any obvious errors yet.
Hmm @araffin  just wondering, you report the "exploration policy" right? From looking at the DDPG code, there is an exploration environment by default, and there is a separate evaluation environment which will step in the environment with a deterministic policy (which is what we want all along with DDPG). I wonder if the evaluation policy can still do a good job even if the exploration policy is very bad. My tests with other software packages that have DDPG show that the exploration policy does nearly as well as the evaluation policy so I am not entirely optimistic, but it is something to think about.

you report the "exploration policy" right?

Exactly. This is the behavioral policy, so the deterministic policy + noise in that case.

there is a separate evaluation environment which will step in the environment with a deterministic policy

Yes, the eval env will only be used if you provide it. I did not provide any in my case.

I wonder if the evaluation policy can still do a good job even if the exploration policy is very bad.
My tests with other software packages that have DDPG show that the exploration policy does nearly as well as the evaluation policy so I am not entirely optimistic, but it is something to think about.

I think it would do only in the case of high amount of noise. Otherwise, the training performance (of the exploration policy) is usually a good proxy for the real performance.

@DanielTakeshi I think your guess was good:
https://github.com/openai/baselines/blob/master/baselines/run.py#L116
It seems the normalization is applied twice.
Can you check commenting this line?

@araffin @pzhokhov
I commented the line out above. Results look much better, though they're not as good as some published results that I see (e.g., from the TD3 paper figure above).
After avoiding the VecNormalize command, I ran the following:
python -m baselines.run --alg=ddpg --env=Ant-v2 --num_timesteps=1e6
python -m baselines.run --alg=ddpg --env=HalfCheetah-v2 --num_timesteps=1e6
python -m baselines.run --alg=ddpg --env=Hopper-v2 --num_timesteps=1e6
python -m baselines.run --alg=ddpg --env=Swimmer-v2 --num_timesteps=1e6
python -m baselines.run --alg=ddpg --env=Walker2d-v2 --num_timesteps=1e6
python -m baselines.run --alg=ddpg --env=InvertedPendulum-v2 --num_timesteps=1e6
python -m baselines.run --alg=ddpg --env=Reacher-v2 --num_timesteps=1e6

python -m baselines.run --alg=ddpg --env=Ant-v2 --num_timesteps=1e6 --noise_type=normal_0.1
python -m baselines.run --alg=ddpg --env=HalfCheetah-v2 --num_timesteps=1e6 --noise_type=normal_0.1
python -m baselines.run --alg=ddpg --env=Hopper-v2 --num_timesteps=1e6 --noise_type=normal_0.1
python -m baselines.run --alg=ddpg --env=Swimmer-v2 --num_timesteps=1e6 --noise_type=normal_0.1
python -m baselines.run --alg=ddpg --env=Walker2d-v2 --num_timesteps=1e6 --noise_type=normal_0.1
python -m baselines.run --alg=ddpg --env=InvertedPendulum-v2 --num_timesteps=1e6 --noise_type=normal_0.1
python -m baselines.run --alg=ddpg --env=Reacher-v2 --num_timesteps=1e6 --noise_type=normal_0.1

python -m baselines.run --alg=ddpg --env=Ant-v2 --num_timesteps=1e6  --noise_type=ou_0.2 --tau=0.001
python -m baselines.run --alg=ddpg --env=HalfCheetah-v2 --num_timesteps=1e6 --noise_type=ou_0.2 --tau=0.001
python -m baselines.run --alg=ddpg --env=Hopper-v2 --num_timesteps=1e6 --noise_type=ou_0.2 --tau=0.001
python -m baselines.run --alg=ddpg --env=Swimmer-v2 --num_timesteps=1e6 --noise_type=ou_0.2 --tau=0.001
python -m baselines.run --alg=ddpg --env=Walker2d-v2 --num_timesteps=1e6 --noise_type=ou_0.2 --tau=0.001
python -m baselines.run --alg=ddpg --env=InvertedPendulum-v2 --num_timesteps=1e6 --noise_type=ou_0.2 --tau=0.001
python -m baselines.run --alg=ddpg --env=Reacher-v2 --num_timesteps=1e6 --noise_type=ou_0.2 --tau=0.001

For each of the seven environments, I ran one training run with three noise settings. I then have this following plot, which overlays the three noise settings together for each game to make comparisons easier:

Note that these are the exploration environment rewards. I don't have an evaluation environment here. (It's easy to enforce an evaluation environment, and I'm actually testing that now, but there's no one single command we can add to the command line to get the evaluation environment set up.) Also, I plot these episode rewards from the 0.0.monitor.csv files that are stored in the log directory, and I smooth by a sliding window of size 20 across the episodes list.

Ant: parameter space noise doesn't seem to work well but the other two finally get above 0, but getting 1000+ would be ideal.
HalfCheetah: looks better, gets to +2000-ish for parameter/Gaussian noise, though the TD3 paper claims DDPG should get 3000+.
Hopper: parameter space noise does the best, with 1500, but the other two can get over 500. But we should be seeing about 2000+.
InvertedPendulum: Gaussian noise sometimes gets to the 1000 limit, the others don't, though this might be a function of how we're using an exploration policy. Ideally all methods get 1000 on this env.
Reacher: Gaussian and OU noise are better with around -10 as the reward, but getting -4 or -3 would be ideal.
Swimmer: the methods get around 14-43. It is not really much of an improvement over the prior results I was getting, unfortunately.
Walker2d: gets around 400-800 but ideally we'd get around 2000.

It looks like parameter space noise might have a slight advantage, but I am not sure I can tell from these limited results. Gaussian noise on the actions seems easier conceptually.
Overall these results are giving me much more confidence in the code base. :) And I think with evaluation rewards instead of exploration rewards, the above would be closer to published results for DDPG.

@DanielTakeshi Did you run any of these benchmarks on vision-based tasks, or know of any results?

@sritee  No, I'm not aware of any standard vision-based DDPG tasks. You'd actually need to change the network design in the DDPG class as well

there are some differences between this code and the implementation in the paper:

@GameHoo
Yes, because OpenAI published not the original DDPG but rather the implementation of the paper PARAMETER SPACE NOISE FOR EXPLORATION. I remember that the parameters matched with the paper.

@DanielTakeshi do you get the same issues if you downgrade mujoco as in https://github.com/openai/gym/issues/1541 ?

@christopherhesse Apologies for not responding, I never had time to follow-up on this, apologies.


Chrome freezes when I try to open the Admin page

Something is preventing me to open the admin page.
admin page

Should've been fixed by https://github.com/agile-alliance-brazil/submissions/commit/957747bfa09404e0010bae21b51db9de94fc0f64.
The problem was that the tagcloud couldn't render any words in the space given because the div was too small. Changed the body of the page to be:
div(tagcloud){width: 700px;height: 300px}.

This ensured the words can actually fit in the container and jqcloud finished running.
The commit just ensures that if we just have div(tagcloud) it won't infinite loop again.


CSV header fixes

What

Renamed columns with better wording
Added percentage symbol to 'User who searched from page` metric
Reordered reading time column to be consistent with single data page

Review Checklist

[x] Changes in scope.
[x] Added/updated unit tests.
[x] Added/updated feature tests.
[x] Added/updated relevant documentation.
[x] Added to Trello card.

 (3) Review data names in CSV download

 (2) CSV header fixes


Go-to-Definition on super() constructor call goes to class when there's no default constructor

In this code, I think I would expect Go-to-Definition on the super() constructor call in the C() constructor to jump to the A() constructor.
class A {
  A();
}

class B extends A {}

class C extends B {
  C() : super(); // invoke Go-to-Definition on "super"
}

However, it currently jumps to class ^B.
@bwilkerson any opinion on this? Technically the implied constructor in B is what's being called, but since it's synthetic I feel like we should skip it.
However, I'm not sure there aren't edge cases that might make this seem inconsistent (for example if B didn't have A as a subclass, then probably we would need to jump to class ^B).

I almost always have an opinion. :-)
I think it's a bit of a wrinkle, but,

As you noted, it would be inconsistent with the behavior when there's no explicit constructor in any of the superclasses.
The field initializers in B (if there are any) will be executed, and skipping B might obscure that fact, especially for newer Dart users. (Although we don't include field initializers in the normal navigation, so that's a fairly weak argument.)

The worst part, IMO, is that once the user navigates to B there's no way to continue along the navigation chain.
The real question is, which would best fit the user's expectations? I have two data points (yours and mine), but it's hard to generalize from there.

From a pedantic point of view, going to B is obviously correct.
The default constructor of the B class is the constructor that super denotes, not the constructor in A.
And as mentioned, if B had initializer fields, then that's where they would be executed.
Now, let's make things interesting.
class A {
  A();
}
mixin M {}
class C extends A with M {
  C() : super();
}

Now, which constructor does C's super() denote? ....
Answer: The implicitly added forwarding constructor of the anonymous mixin application class of A with M.
Nobody wants to go there. Ever. (If there even is a "there" to go. I don't know where I'd put the cursor. On the M in with M probably.)
The denoted super-constructor existing is not enough reason to navigate to it.
If I instead write the same program, just with the mixin application being named instead of anonymous:
class A {
  A();
}
mixin M {}
abstract class AM = A with M;
class C extends AM {
  C() : super();
}

then what?
Going to class AM is technically correct. Just as correct as going to B above. The class AM is class, it has an unnamed constructor that you can refer to in subclasses, like C does.
I'd do the same thing here as for B. (And this didn't bring me closer to figuring out what that had to be.)
I am leaning towards the current, pedantically correct, behavior.
(I don't think it's possible to display a synthetic default constructor, like B(): super(); // Synthetic default constructor, because there is no place to show it. Otherwise that would be neat.)

abstract class AM = A with M;

TIL you can do this 🤯

(I don't think it's possible to display a synthetic default constructor, like B(): super(); // Synthetic default constructor, because there is no place to show it. Otherwise that would be neat.)

We could potentially do this (it's basically a variation of "Closing Labels"), although I'd really like to move closing labels to something more standard like Inlay Hints - but currently we can't control the styling for that (requires https://github.com/microsoft/vscode/issues/151920).
I am less sure now that this should be changed. I do feel like not being able to continue the chain sucks, but you can always invoke Go to Super to get to the super class and then manually locate the constructor from there. It's not perfect, but it seems like there probably isn't any perfect solution here (besides maybe showing synthetic constructors in the editor, but I'm also not convinced that wouldn't be a little weird and cause things to jump around as you're adding constructors).
I'll close this on the assumption we probably don't want to change this then (we can always re-open if additional opinions change things, or if we think we should try implementing some other mechanism like showing the synthetic constructor, or a way to jump to super constructor from a class that doesn't have an explicit constructor).


Filters for max value

Min filter is great, but I regularly find myself wanting a max filter, too.  There are so many instance types that with just a min filter one still sees a ton of instances, so it's hard to narrow down to just the instance types you really want to see.
For example, we have an instance with 7.5 GB RAM and we want to look for types that have just a little more (but not 32GB, not 64GB, etc.).  I can enter 7.5 for min RAM, but that will show me everything above 7.5, which is still a lot of instance types.

:+1:
I just wanted to create an issue for the same thing, luckily I found this first :)

I tried to implement this and I created a patch(find it attached) but I'm completely new to jquery so soon I got stuck and I don't know how to make it work.
Feel free to pick it up from where I left it.
min-max-patch.txt


Fix forgotten yield

Bah, thanks. It looks like I need to spend some time cleaning up this code. There's no need for these nested gen.Returns and yields.


new version cannot close

when i close bluetuith, use q, or ctrl c is not work
so i just use kill command.
and i kill bluetuith.
it show look this

cannot run anything

The new keybinding is now Shift+q, I forgot to update the docs, I'll do it
soon. See #8.
On Thu, 22 Jun, 2023, 10:53 am Thinker, @.***> wrote:

when i close bluetuith, use q, or ctrl c is not work
so i just use kill command.
and i kill bluetuith.
it show look this
[image: image]
https://user-images.githubusercontent.com/87791643/247831147-62b6bafa-0578-411d-9c90-6ff606e0ba98.png
cannot run anything
—
Reply to this email directly, view it on GitHub
https://github.com/darkhz/bluetuith/issues/17, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKQERARGFL6RLM2INNLQKXLXMPJGLANCNFSM6AAAAAAZPVMUTU
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

The new keybinding is Shift+q, I forgot to update the docs, I'll do it soon. See #8.

@darkhz can this change be added to invidtui too please

@sabrehagen sure

@darkhz seeing as no other function is bound to q, why not make q quit? q is common throughout many tui utilities as the method of exit, I have never encountered Shift+q before.

It's very easy to accidentally press 'q', so I think Ctrl+C may be a more suitable choice.

I would disagree, for a competent user 'q' is not usually an issue. Writers of less, more, etc. i.e. unix core tools must not have felt it was an issue.

That's true, let me look into this


Question

How to get the diamond and the octagon in the new version

I haven't made them yet :D
Working on them this week


Bless failed.

I saw there was an issue with that, and solved, but recently my Boot Champ started to give me bless failed message. Tried to remove my NTFS driver, tried basically everything, but no solution.

Was there additional information in the error dialog?

Right I am sorry. This is what I got. If I can provide any log, I will, I'm just not familiar with Boot Champ, neither with GitHub (yet) Just tell me what to do, and what data you need. :)

Are you running version 1.7?

Yup, I downloaded it straight from Your website. :)

Can you email me the logs at ~/Library/Application Support/BootChamp/Logs ?

The only file there is BOHelper (checked the hidden ones too) Is it what you need? Or is logs turned off and I need to turn it on somehow?

I think that is the system-level library. The one I'm looking for is in your Users folder.  Paste in this exactly:
~/Library/Application Support/BootChamp/Logs

Then click Go. Finder should open this location and there should be text files in it.

emailed them through your website. :)

Did you do a major Windows update recently? Does your Windows volume show in Startup Disk in System Preferences?

I put up a test build. Can you run this command first and then try that build?
defaults write com.kainjow.BootChamp NoLegacy -bool YES

No I did not have any update, I dont update my Windows at all. Though I cannot see it in Sys Preferences. When I hold opt on start, I see it, but in Sys Pref I dont. I tried the test build, it restarts into OS X.

My Windows partition does appear in System Prefs.
I ran that command then tried your new Boot Champ test build but it errored out again:
--
BootChamp was unable to set your Windows volume as the temporary startup disk
Bless failed:
EFI found at IODeviceTree:/efi
Mount point for /Volumes/BOOTCAMP is /Volumes/BOOTCAMP
Mount point is '/Volumes/BOOTCAMP'
No BootX creation requested
No boot.efi creation requested
found ioreg "FirmwareFeaturesMask"; featureMaskValue=0xC003FF37
found ioreg "FirmwareFeatures"; featureFlagsValue=0xC001F537
isPreBootEnvironmentUEFIWindowsBootCapable=0
preboot environment is not UEFI boot capable
isDVDWithElToritoWithUEFIBootableOS=0
Checking if disk is complex (if it is associated with booter partitions)
GPT detected
Booter partition required at index 5
System partition found
Preferred system partition found: disk0s1
Returning booter information dictionary:
<CFBasicHash 0x7fedbbd00200 [0x7fff73104890]>{type = mutable dict, count = 3,
entries =>
0 : <CFString 0x103ba0990 [0x7fff73104890]>{contents = "System Partitions"} = (
disk0s1
)
1 : <CFString 0x103ba1170 [0x7fff73104890]>{contents = "Data Partitions"} = (
disk0s4
)
2 : <CFString 0x103ba1190 [0x7fff73104890]>{contents = "Auxiliary Partitions"} = (
)
}
Path to mountpoint given: /Volumes/BOOTCAMP
IOMedia disk0s4 has UUID 088795F8-5F77-431B-BE89-A3F6C56A74D8
Setting EFI NVRAM:
efi-boot-next='IOMatchIOProviderClassIOMediaIOPropertyMatchUUID088795F8-5F77-431B-BE89-A3F6C56A74D8BLLastBSDNamedisk0s4'
Could not set boot device property: 0xe00002bc

@Mudflapper this build tries to address a specific issue @danihungary is seeing. I think you might be having the issue with the latest El Capitan beta? See issue #19.

@danihungary interesting, so it doesn't give an error, but doesn't boot into Windows.
I think something is off with your Boot Camp setup. Usually if the Windows partition doesn't show up in Startup Disk, then there is a configuration problem. BootChamp expects Boot Camp to be working properly. So at this point I don't really know what is the issue, or what else to check.

Thank You for all the help! So the fault is at different place, I'll try to figure it out. :) Thank You though again for all the effort, I hope I can use BootChamp soon again. :+1:


AttributeError: 'ExponentialTrainer' object has no attribute 'deepspeed'

Traceback (most recent call last):
File "run.py", line 139, in
train(trainer, training_args.resume_from_checkpoint, last_checkpoint)
File "run.py", line 26, in train
train_result = trainer.train(resume_from_checkpoint=checkpoint)
File "/home/***/P-tuning-v2/training/trainer_exp.py", line 369, in train
if self.deepspeed:
AttributeError: 'ExponentialTrainer' object has no attribute 'deepspeed'

I encountered this error when running CoNLL12 on BERT.


Implement initial functionality

🗣 Description
This PR contains the code to create an AMI with the Egress-Assess tool software.
💭 Motivation and context
The Egress-Assess AMI will be used for egress testing of networks during engagements.
🧪 Testing
An AMI was built, deployed, and successfully tested in a staging environment.
Tests were conducted to ensure proper connection and exchange of data between an Egress-Assess client and the server running in the staging environment.
✅ Pre-approval checklist

[x] This PR has an informative and human-readable title.
[x] Changes are limited to a single goal - eschew scope creep!
[x] All relevant type-of-change labels have been added.
[x] I have read the CONTRIBUTING document.
[x] These code changes follow cisagov code standards.
[x] All relevant repo and/or project documentation has been updated to reflect the changes in this PR.
[x] All new and existing tests pass.

✅ Pre-merge checklist

[ ] Build final pre-release AMI and test
[ ] Finalize version.

✅ Post-merge checklist

[ ] Create a release.

@m1j09830 - don't forget to finalize the version before merging.

@mcdonnnj - Please review this PR even though it already has 2 dev reviews, since I helped @m1j09830 significantly and therefore my review is fairly biased.


Corrected build error in rx65n_sbram.c

Corrected build error in rx65n_sbram_open()
Pull Request Template
Title guidelines
Following the guidelines for writting good commit messages (https://chris.beams.io/posts/git-commit/) and creating a meaningful title is key in effective team communication.
do's

stm32h7:  Add SDMMC Support
nsh:  Separate source and sh for POSIX compliance
nxstyle:  Fixed Camel case detection
drivers/serial:  Fixed style violation

dont's

fixed bug
nxstyle
PR #12

Description
Describe problem solved by the PR
A clear and concise description of the problem, if any, this PR will solve. Please also include relevant motivation and context: E.g. The current nsh sh command violates the POSIX ...
List any dependencies that are required for this change.
Describe your solution
A clear and concise description of what you have implemented.
Describe possible alternatives
A clear and concise description of alternative solutions  you've considered.
Additional context
Add any other context or screenshots for the pr.
Fixes # (issue)
Type of change
Delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[ ] Test A
[ ] Test B

Test Configuration:

Nuttx board/config:
Hardware:
Toolchain:

Checklist:

[ ] My code follows the style guidelines of this project (NEED link to how to run checkpatch)
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules
[ ] I have checked my code and corrected any misspellings (NEED link to how to run checkpatch spelling)

@anjana-tel could you spend sometime to enable renesas precheck? So we can ensure new PR don't break your boards in the furture. You just need modify few files:
https://github.com/apache/incubator-nuttx-testing/blob/master/cibuild.sh
https://github.com/apache/incubator-nuttx-testing/blob/master/testlist/mips-riscv-x86-xtensa.dat
https://github.com/apache/incubator-nuttx/blob/master/.github/workflows/build.yml
https://github.com/apache/incubator-nuttx-testing/blob/master/docker/linux/Dockerfile
Most work is setup the toolchain.


postTweetMedia

Hi,
Was trying to post an image to a tweet, but without any luck yet. Have read the twitter API docs I can see this:
This end point has been deprecated. Please refer to Uploading Media: Multiple Photos for uploading one or more media entities.
https://dev.twitter.com/rest/reference/post/statuses/update_with_media
any chance to update the code to reflect this?
thanks

This will be in the next version wich is coming soon.

Today, I updated twitter package and met 'postTweetMedia' method error. I am looking forward the next version to fix the error.
Anyway, thanks for your good work.

Hi.
This method is deprecated : https://dev.twitter.com/rest/reference/post/statuses/update_with_media
I added an example for uploading a media : https://github.com/thujohn/twitter#examples

The problem is solved. Thanks.


⚠ Bofrost.be has degraded performance

In a93fec1, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1812 ms

Resolved: Bofrost.be performance has improved in 4c5f074.


refactor: Master => develop

Merge latest release back into develop

Builds ready [8f616b8]builds: chrome, firefox, opera
bundle viz: background, ui, portal-inpage, portal-contentscript, ui-libs, bg-libs, phishing-detect
dep viz: background
all artifacts


Updating overlay stops after several updates

OpenSeadragon Version: 2.4.0
I'm trying to draw an overlay underneath the mouse cursor, but the overlay stops updating after several cycles. If I pan or zoom then the overlay updates again for a few cycles and then stops.
I'm getting the mouse position from MouseTracker. drawMouseOverlay is called whenever viewerMousePosition is updated.
drawMouseOverlay = () => {
    const { selectedSlideViewer } = this.props;

    const createCursorElement = (viewerId: number): HTMLSpanElement => {
      const element: HTMLSpanElement = document.createElement('span');
      element.className = 'synch-cursor fa fa-crosshairs fa-2x';
      element.id = `sharedview-mouse-cursor-${viewerId}`;
      return element;
    };

    const element: HTMLSpanElement = createCursorElement(0);
    const { x, y } = this.props.viewerMousePosition;

    const existingOverlay = selectedSlideViewer.getOverlayById(element.id);

    if (existingOverlay) {
      const { width, height } = existingOverlay.element.getBoundingClientRect();

      existingOverlay.update(
        selectedSlideViewer.viewport.pointFromPixel(
          new OpenSeadragon.Point(x - (width/2), y - (height/2))
        ),
        OpenSeadragon.Placement.TOP_LEFT
      );
    } else {
      selectedSlideViewer.addOverlay({
        element: element,
        location: selectedSlideViewer.viewport.pointFromPixel(
          new OpenSeadragon.Point(x, y)
        ),
        placement: OpenSeadragon.Placement.TOP_LEFT
      });
    }
  };

I've also tried destroying the overlay and readding it but this leaves artifacts of the destroyed overlays when I pan or zoom.
Any idea what is going on here? I've verified that the existingOverlay.location is updated when I call existingOverlay.update() but the overlay stops moving.

Hmm... you might try calling selectedSlideViewer.forceRedraw() after doing your update, to see if that helps.
Beyond that, I'd probably have to see a running example to get to the bottom of it.

selectedSlideViewer.forceRedraw() fixed my issue completely. Thank you very much for your reply.


[Code Addition Request]:

Have you completed your first issue?

[X] I have completed my first issue

Guidelines

[X] I have read the guidelines
[ ] I have the link to my latest merged PR

Latest Merged PR Link
N/A
Project Description
Project Description
The Crater Detection Model is an advanced object detection system developed using YOLOv8, aimed at identifying all types of lunar craters on the moon's surface. This model is particularly effective in low-light conditions, making it suitable for analyzing regions such as Permanently Shadowed Regions (PSR) where visibility is limited.
Key Features:

Object Detection: Accurately detects and localizes craters in lunar images.
Training Dataset: Utilizes the Impact Moon Craters (LU3M6TGT) dataset for robust training and validation.
Flexible Input: Designed to process images of various resolutions, enhancing usability across different applications.

This model serves as a valuable tool for lunar exploration, enabling researchers and
enthusiasts to gain insights into the moon's geological features and inform future missions.
Full Name
Kaibalya Mohapatra
Participant Role
GSSOC, HACKTOBERFEST

Update: Cloning error due to invalid file path on Windows
I attempted to clone the repository, but I encountered an error related to an invalid file path on my Windows system. The error seems to be caused by a folder name ending with a trailing period, which is not allowed in Windows.
Here is the error I received:
error: invalid path 'Machine_Learning/Linear regression using gradient decent./Linear regression using gradient decent.ipynb'
fatal: unable to checkout working tree

This issue prevents me from successfully cloning the repo and setting up my environment.

I will resolve the issue by today, you can try cloning in evening.


ETags

As far as I can tell, jsDAV does not provide an ETag header in GET responses (tested with the current HEAD). Grepping the source reveals various occurrences of "etag" though, so I'm not sure whether this is an intentional omission.
(For context, Apache's WebDAV implementation does provide ETags and also obeys the If-Match header for PUT requests.)

I'm afraid this doesn't seem to work (anymore?) after all: Using the code below, I can either get ETags (with the tree property) or directory indexes (with the node property), but never both.
It's probably a simple oversight, but I couldn't find any documentation and reading the source code didn't get me anywhere either - so any help would be greatly appreciated.
#!/usr/bin/env node
"use strict";

var jsDAV = require("jsDAV/lib/jsdav");
var Tree = require("jsDAV/lib/DAV/backends/fsext/tree"); // required for ETags
var path = require("path");

var root = path.join(__dirname, "data");

jsDAV.debugMode = true;
jsDAV.createServer({
    //node: root,
    tree: Tree.new(root)
}, 8000);

I can confirm: setting node breaks ETags.

It looks like node just is shorthand to instantiate an fs tree at node, and overwrites any passed tree.
Directory listings/PROPFIND don't work in fsext though.

Just for fun I copied the getETag impl from fsext to fs and it seems to work fine (both the ETag header and the getetag PROPFIND property). Probably needs optimization though.


Contributor Console Gerrit Notification

Summary
Contributor Console Gerrit Notification.
Background
Add contributor console notification for Gerrit that reminds the user to logout/login again to realize the updated permissions. See meeting video at the specified timeslot for the discussion on this topic.
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
See video referenced below.
Environment
Please complete the following information:

Environment:

[x] ALL
[ ] DEV
[ ] STAGING
[ ] PROD

Browser:

[x] ALL
[ ] Chrome/Brave
[ ] Firefox
[ ] Edge
[ ] Lynx

Version: v1.0.XX

Acceptance Criteria
The "done" criteria when this feature or problem is resolved. Such as:

Unit Tests added and running in CI
Functional Tests updated to cover feature, if applicable
Demonstrate the set of capabilities to the product team while the code is running in the STAGING environment.

References
Meeting video/audio: copy the link below to share this recording with viewers:
https://zoom.us/rec/share/U7eGoOmvKlsN0MbSuumuFDPRA089ESL-InAH_OX18sI7hM4puDeCGkNXfBEt2BP3.LaKwvh4bw6vuD0mW
My Gerrit Video: https://cla-project-logo-prod.s3.amazonaws.com/20210316+Gerrit+Workflow+Video.mov
for the Gerrit discussion, start at around 00:31:38 time
00:37:53 - add contributor console notification for Gerrit that reminds the user to logout/login again to realize the updated permissions

Show the following message for a Gerrit user
You have completed the CLA steps necessary to contribute. Please note that you will need to logout and login on Gerrit to get the updated permissions. You can now return to writing awesome stuff.
instead of the regular message
You have completed the CLA steps necessary to contribute. You can now return to writing awesome stuff.
Note there is no change needed for GitHub case.

As discussed in the standup, this need to be handled in both CCLA & ICLA flows.

For the case of only ICLA in Gerrit flow I have added the below message. @pranab-bajpai can you please confirm the same?
title = 'Important Note';
message = 'Please note that after you have signed the CLA, you will need to logout and login on Gerrit to get the updated permissions. After completing this, the EasyCLA check will be complete and enabled for all future code contributions for this project.'

Please note that after you have signed the CLA, you will need to logout and log back in to your Gerrit account to get the updated permissions. After completing this, the EasyCLA check will be complete and enabled for all future code contributions for this project.

Verified on Staging environment and found the new proposed message is getting displayed to the user when user proceed to sign ICLA and is as follows:

This has been deployed to PROD on 19th March.


LDAP plugin no filter (Extend LDAP functionality)

Extend LDAP functionality.
Summary
The LDAP plugin should have a filter option, so users are not duplicated (cause in my case the same user will be use for JIRA, GitLab, Remine, etc ....) If I do not have a filter option then it would be impossible not to duplicate or triplicate the users. So far the above support LDAP filter and I think LDAP plugin should do it as well when requesting info to the LDAP server.

Hi @joaquin386
Please try to avoid cross posting to Gitter, GitHub, and the mailing list. Since this is not an actual issue running Kong (as the GitHub boilerplate that you read notes), we can have this discussion on the mailing list (https://groups.google.com/forum/#!topic/konglayer/TVeHilwYCpc). Thanks!


Add ability to extract file from namespace

We should add the capability for scope to get a file from a namespace, and save it to a destination on the host's disk.
This capability is a key to:

Our crash analysis model. For example, allow the daemon to grab a core dump from another container.
Getting events / metrics / logs from another container's filesystem.

The approach taken is as follows:

cli wants to read a file
it execs scope in constructor mode as a child process, with a constructor arg
change namespace to container ns
read file into memory
change back to original ns
write file to disk/wherever
return from our child cli process and then have that file available for use by the cli if it was retrieved successfully.

The command added is --getfile SRC_FILE,DEST_FILE and supports --namespace NSID


[FLINK-11183][metrics] Properly measure current memory usage

What is the purpose of the change
With this PR we properly re-calculate the memory usage metrics. Currently we access a single MemoryUsage object once and, but this is never updated.
Verifying this change
This change is a trivial rework / code cleanup without any test coverage.

Shouldn't there be a test to check that this works and to guarantee that it will continue working in the future?
If I understand the JIRA correctly, the problem was that the reported metric was never updated. If measuring memory is not easy to test (because it depends on what runs on the machine at any point in time), at least we can measure once, allocate a big object, and then measure again and see that the two measurements differ. Of course, sth more accurate would be even better.

Or, we could even extract as separate methods the:
MetricGroup heap = metrics.addGroup("Heap");
heap.<Long, Gauge<Long>>gauge("Used", () -> memoryBean.getHeapMemoryUsage().getUsed());
heap.<Long, Gauge<Long>>gauge("Committed", () -> memoryBean.getHeapMemoryUsage().getCommitted());
heap.<Long, Gauge<Long>>gauge("Max", () -> memoryBean.getHeapMemoryUsage().getMax());

and the same lines for the off-heap memory, with descriptive names, and test those methods separately.
This will also help at enforcing the "each method should have a single responsibility" principle.
What do you think @zentol ?

I don't think tests are worth it in this case. You have to do some funky things to test these metrics, and even then you can't even cover all of them.
Max can't be tested since usually is either undefined (-1) or fixed from startup. In any case we can't influence this value.
For heap metrics we can indeed allocate an array to test Used, however to test Committed we have to exceed the current committed space. If the JVM is configured to commit little memory on the start everything is fine, but it's not unthinkable that people set this to some really large value (say multiple GB) In the worst-case committed == max so you have to guard against this edge-case.
This gets even worse for non-heap metrics since you can't influence it as easily. You'd have to load a ton of classes to affect Committed.
At the end of the day the test would be significantly more complex than the code that it tests, so I'm against adding it.


Webpack + 3rd party library: Specifying alternate file instead of package entry point

OS

Mac OSX El Capitan

Versions

angular-cli: 1.0.0-beta.20-4
node: 6.7.0
os: darwin x64

Repro steps.

Install 'mapbox-gl' library: npm install mapbox-gl --save
Install relevant typings: npm install @types/mapbox-gl --save-dev
Attempt to import: import * as mapboxgl from 'mapbox-gl'

Forgive my naiveté – I'm new to Angular 2 and webpack – but I am trying to use a 3rd party library (mapbox-gl-js) that was designed to work with Browserify.
As a result, in order to adapt it with webpack, their installation instructions require you to use the distribution build instead of the package entry point:
import mapboxgl from 'mapbox-gl/dist/mapbox-gl.js';
Normally, to work around this, I would add a resolve.alias property in my webpack.config.js file:
resolve: {
  alias: {
    'mapbox-gl': path.resolve('./node_modules/mapbox-gl/dist/mapbox-gl.js')
  }
}

I understand I cannot override the webpack configuration settings in angular-cli, based on the discussion in issue #1656.
So, my question is: what are the current best workarounds to this problem?

Import and re-export?

I'm sorry, I don't understand.

Have something like shared/index.ts (or whatever naming you choose), with something like:
export mapboxgl from 'mapbox-gl/dist/mapbox-gl.js';

I am trying this, without success. I am getting an error that the module cannot be found.
I'm not sure this works in this situation... I think I still need to change the library's file path in Webpack.
./src/shared/index.ts? Or a different folder?

You'd be importing from ./shared or../shared .

I can't even get the export statement to work correctly, no matter which directory I place it in. I'm sorry.

Oops. I know that export * from ... works and kinda assumed the previous one.
I suppose you still can write something like:
import _mapboxgl from 'mapbox-gl/dist/mapbox-gl.js';

export var mapboxgl = _mapboxgl;

Thanks Meligy, this is helpful, but I'm still unable to resolve the module in either an import or export statement. I get the error TS2307: Cannot find module 'mapbox-gl/dist/mapbox-gl.js'.
I've tried a bunch of different kinds of relative paths (e.g. '/node_modules/mapbox-gl/dist/mapbox-gl.js') without success.
What's puzzling is that TypeScript should be able to locate it via the Node.js module resolution mechanism when it's in my node_modules folder...

TypeScript will not see it without a declaration file, either in the package itself, or installed via typings tool or @types NPM packages. This is not specific to Angular CLI.

As I noted in my original post, I have already installed the @types NPM
package for this library.
On Thu, Dec 1, 2016, 6:00 PM Meligy, GuruStop.NET<EMAIL_ADDRESS>wrote:

TypeScript will not see it without a declaration file, either in the
package itself, or installed via typings tool or @types NPM packages.
This is not specific to Angular CLI.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/angular/angular-cli/issues/3335#issuecomment-264353093,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF2pRpYtbuzx5WUU7kNBEFm-oThAY0pBks5rD3vDgaJpZM4LA-iA
.

Typescript can find it, the problem is that webpack does not find it.
Unfortunately the only way to use it for now will be to const mapboxGl = require('...');. They should publish their code to NPM without Flow or using a different main field in their package.json.  Looking at their code, there's no reason they should be incompatible with webpack while also supporting browserify.

Although this issue is already closed...
Just tried using latest cli etc.
Works:
import 'mapbox-gl/dist/mapbox-gl';
Does not work:
import * as mapboxgl from 'mapbox-gl/dist/mapbox-gl';
Returns:
ERROR in /developer/src/mapbox-demo/src/app/mapbox/services/mapbox.service.ts (2,27): Cannot find module 'mapbox-gl/dist/mapbox-gl'.)
webpack: Failed to compile.

Working solution has one problem... no access to mapbox ;)

Thanks for sharing this, Bernhard!

Working solution has one problem... no access to mapbox ;)

Do you mean, no mapbox API access?

No with importing like
import 'mapbox-gl/dist/mapbox-gl';
You do not have an instance to talk to ;) It is just magically imported by webpack, but not usable inside of my component.

Oh, I see. 🙁  So this isn't really a working solution then.

Only if you find the mapbox object somwhere inside of the window object...
So basically... no solution =)
@MattSidor look at https://github.com/mapbox/mapbox-gl-js/pull/4391/
I really hope to get it merged into mapbox-gl... so those issues will be gone.


Cleanup fails if nodes are undefined

It is possible to get into a state where one or more of the nodes are shut-off.
These nodes need to be undefined and not destroyed.
In this file:
kubeinit/kubeinit/roles/kubeinit_libvirt/tasks/10_cleanup.yml

It tries to destroy the node and then undefine it. But if a node is shut-off the destroy step will fail and undefine won't get a chance to run. One option is to just ignore errors in the destroy step.

Fixed by https://github.com/Kubeinit/kubeinit/pull/334

Still failing and reproduced here: https://storage.googleapis.com/kubeinit-ci/jobs/pr/313019047-k8s.ovn-libvirt-3-1-default-2<IP_ADDRESS>.30.14-1/index.html

@ccamacho This is now fixed, right?

Indeed closing, thanks @gmarcy for the final fix!


Multiple VWAN Hubs in same region is available

In the Virtual WAN Hub section, we currently this statement, "There can only be one hub per Azure region." Per the VWAN FAQ (https://docs.microsoft.com/en-us/azure/virtual-wan/virtual-wan-faq#is-it-possible-to-create-multiple-virtual-wan-hubs-in-the-same-region), we can now have multiple virtual hubs per region for the same VWAN. I have corrected the statement and added a link directly to the corresponding FAQ.

#assign: @EdPrice-MSFT
@EdPrice-MSFT
Can you review this PR?
IMPORTANT: When this content is ready to merge, you must add #sign-off in a comment or the approval may get overlooked.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#Sign-Off


Update Standalone

Updated standalone and dyncomp model to develop models for node and container level. Updated predict for standalone model to accept a dict containing feature names and corresponding values. Updated extractor_test.py to contain additional examples. Updated train_types to include enums for Label groups instead of only feature groups.

By pushing this, I should also be able to test on kind cluster. kepler-model-server-base already installed the sklearn dependency. By making a second push, kepler-model-server should then also install the sklearn dependency from kepler-model-server-base (please let me know if this is incorrect assumption). Also, for the next PR which will be pushed today, I plan to update extractor.py to accept the new kepler metrics. The following PR will then be focused on setting up the dyn-comp pipeline to handle multiple models (for package power and dram power). How this is implemented will depend on @sunya-ch implementation of the new pipeline api.


Implement VR Support

Implement a canvas with world space that would attach to a camera.
I made it working by adding a couple of lines to LogDrawer. Thank you for this library, it's awesome :) Saved me a lot of time.

Thanks, any way you can integrate your work in a pull request?


Set context after button push for facebook webview

Hi,
Really struggling here to find the correct (or any) solution, so any help would be appreciated.
I am using messenger as my interface. I am creating a prototype where I have a scenario where

the user is presented with a carousel of movies (for example) each with a button "Choose viewing times"
This button will open a url (using webview) with the movie id in the querystring.
The webview opens with a list of days and times, and the user selects one
An api call is made and a response is created (using facebook send api) to display the message "you selected Tuesday 7pm" and a set of options about whether the user is happy with their selection "yes,no,choose again"

I have this working visually, but not with actual data passing around. There are a bunch of questions I have around this:-

I would rather use my "agent" to do the message handling rather than shoehorning messages via the facebook graph api, but once I leave the ecosystem of the agent and open the webview I don't know how to get a handle on the agent to jump back into the conversation.
How can I set a context when the button to open the webview is clicked so when they come back, the yes/no options are in context for this "select movie" intent - as well as having the values in context in case they don't use the buttons

Again, any assistance would be helpful. I think this is a bit too new for stackoverflow given the lack of Q/A there.
Cheers,
Jarrod

Hi Jarrod, sorry you're having trouble. I don't think this is the right venue for the question as it doesn't relate to the fulfillment library and isn't very visible to others that might be having similar issues.  Here is a StackOverflow question and answer you might find useful: https://stackoverflow.com/questions/50864959/passing-facebook-messenger-bot-webview-data-to-dialogflow-and-continuing-the-con/50864994#50864994


metadata-retriever sidecar version bump

Description
metadata-retriever sidecar version bump
GitHub Issues
List the GitHub issues impacted by this PR:

GitHub Issue #

https://github.com/dell/csm/issues/885

Checklist:

[x] I have performed a self-review of my own code to ensure there are no formatting, vetting, linting, or security issues
[ ] I have verified that new and existing unit tests pass locally with my changes
[ ] I have not allowed coverage numbers to degenerate
[ ] I have maintained at least 90% code coverage
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[x] I have maintained backward compatibility

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Please also list any relevant details for your test configuration

[ ] Test A
[ ] Test B

powerscale team will do basic sanity testing, this PR can be raised after that.


@conda-forge-admin,please add bot automerge

Hi! This is the friendly automated conda-forge-webservice.
I just wanted to let you know that I added bot automerge in conda-forge/ticcutils-feedstock#19.


fix: replace NULL by nullptr

Codecov Report
Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (7d973f9) 63.92% compared to head (98fe61f) 64.16%.

Files
Patch %
Lines

src/glog/logging.h.in
50.00%
0 Missing and 2 partials :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #993      +/-   ##
==========================================
+ Coverage   63.92%   64.16%   +0.24%
==========================================
  Files          17       17
  Lines        3326     3326
  Branches     1125     1125
==========================================
+ Hits         2126     2134       +8
+ Misses        774      765       -9
- Partials      426      427       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


f173d91430ad30604cb9cb98d0cbfd617002bda0be08fb14b5226de56ef09c24

Expected behavior
Actual behavior
Information about the Issue
Steps to reproduce the behavior

...
...

failed to register layer: re-exec error: exit status 1: output: ProcessBaseLayer \?\D:\ProgramData\Docker\windowsfilter\f173d91430ad30604cb9cb98d0cbfd617002bda0be08fb14b5226de56ef09c24: The system cannot find the path specified.


SSE4.1 support cannot be enabled on machines with AVX

It is due to this check:
https://github.com/BinomialLLC/basis_universal/blob/8903f6d69849fd782b72a551a4dd04a264434e20/encoder/basisu_kernels_sse.cpp#L26-L27
Looks like some unfortunate refactoring artefact from old code that supported more than one SIMD type.

If AVX || AVX2 || AVX512F are defined here then the generated SSE code won't execute on very old (ancient!) CPU's that don't support AVX. That's why these macros are here. I'll double check that this still works, though.

I've verified that SSE supports works as intended under Windows and Linux. Can you describe what you're doing that's causing these code gen sanity checks to trigger? Your options are probably not the same as the ones used to compile the tool.
I think it's fine to remove them if you're ok with the generated code not working on old CPU's. Perhaps I can put these checks in a guard.


Not work for EfficientNet

I am looking for the same estimation for efficientNet. Did you make it work?

I am looking for the same estimation for efficientNet. Did you make it work?

Unfortunately, no

I have also had this experience, with EfficentNet-B4.  My error looks like this:
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/segmentation_models_pytorch/encoders/efficientnet.py", line 66, in forward
    x = stages[i](x)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py", line 139, in forward
    input = module(input)
  File "/usr/local/lib/python3.6/dist-packages/torchstat/model_hook.py", line 44, in wrap_call
    assert module.__class__ in self._origin_call
AssertionError


Update n-series-driver-setup.md

The previous procedure for CentOS/RHEL 7 was not working anymore. I updated this doc based on the information from the below nvidia link and worked fine:
https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=CentOS&target_version=7&target_type=rpm_network

@vikancha-MSFT
Can you review the proposed changes?
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team


Podman stats with no containers listed throws an error

Fixes: #1028
Signed-off-by: haircommander<EMAIL_ADDRESS>

@mheon @rhatdan PTAL

One nit I'm pretty sure is going to trip gofmt, otherwise LGTM

@mheon apparently stats should run with no containers, given the test that is failing. Is it worth changing the tests, or is this not really what we want functionality wise?

... apparently stats should run with no containers, given the test that is failing.

Is there a stats-related failure?  It may just be flakes.  For example, the smoketested error:
setsebool:  SELinux is disabled.
neverallow check failed at /var/lib/selinux/targeted/tmp/modules/100/base/cil:9191
  (neverallow base_typeattr_7 unlabeled_t (file (entrypoint)))
    <root>
    allow at /var/lib/selinux/targeted/tmp/modules/200/container/cil:1554
      (allow spc_t unlabeled_t (file (entrypoint)))
    <root>
    allow at /var/lib/selinux/targeted/tmp/modules/100/sandboxX/cil:866
      (allow sandbox_x_domain exec_type (file (entrypoint)))
    <root>
    allow at /var/lib/selinux/targeted/tmp/modules/100/virt/cil:1677
      (allow virtd_lxc_t exec_type (file (entrypoint)))
    <root>
    allow at /var/lib/selinux/targeted/tmp/modules/100/virt/cil:2068
      (allow svirt_sandbox_domain exec_type (file (entrypoint)))

Failed to generate binary
/usr/sbin/semodule:  Failed!

Travis is going to fail, we broke it just prior to release today (one of the tests times out on travis nodes - oops. Got to increase the timeout there). The others look like there might be a legitimate problem. Let me test this.

Alright, Docker does allow stats with no container IDs, and seems to interpret it as --all... So we may be doing the right thing.
Still, we do have a bug here - we're not printing the header if there are no containers. Docker does do that.

@wking that is a bad libsemange package that we have worked around until it gets updated in fedora.  SHould be soon.  For now you should ignore.

@haircommander Ok lets have podman stats  imply --all

LGTM

@rh-atomic-bot r+


Make it possible to add or exclude certain fatal signals on Linux/OSX

If using czmq you don't want the signal handler to catch SIGTERM as that is used within that library. Similar, some users might want to have a ctrl-c to be caught (although I don't recommend it)
I.e. the following code should instead install signals from a set of signals. That set should be possible to manipulate prior to the the Install call of the logger.
   //
   // Installs FATAL signal handler that is enough to handle most fatal events
   //  on *NIX systems
   void installSignalHandler() {
#if !(defined(DISABLE_FATAL_SIGNALHANDLING))
      struct sigaction action;
      memset(&action, 0, sizeof (action));
      sigemptyset(&action.sa_mask);
      action.sa_sigaction = &signalHandler; // callback to crashHandler for fatal signals
      // sigaction to use sa_sigaction file. ref: http://www.linuxprogrammingblog.com/code-examples/sigaction
      action.sa_flags = SA_SIGINFO;

      // do it verbose style - install all signal actions
      if (sigaction(SIGABRT, &action, NULL) < 0)
         perror("sigaction - SIGABRT");
      if (sigaction(SIGFPE, &action, NULL) < 0)
         perror("sigaction - SIGFPE");
      if (sigaction(SIGILL, &action, NULL) < 0)
         perror("sigaction - SIGILL");
      if (sigaction(SIGSEGV, &action, NULL) < 0)
         perror("sigaction - SIGSEGV");
      if (sigaction(SIGTERM, &action, NULL) < 0)
         perror("sigaction - SIGTERM");
#endif
   }

Look into this as well. In case of exhausted stack: +

// http://man7.org/linux/man-pages/man2/sigaltstack.2.html

https://github.com/KjellKod/g3log/pull/48


fix: Resolve no clean images detected issue; Remove extra blank image name

Fixes- https://github.com/konveyor/move2kube/issues/1054
$ move2kube collect -a dockerswarm --log-level debug

INFO[0000] Begin collection
INFO[0000] [*collector.ImagesCollector] Begin collection
DEBU[0000] images : [quay.io/konveyor/move2kube-ui:latest nginx:stable-alpine ]
DEBU[0000] UserID not available in image metadata for [quay.io/konveyor/move2kube-ui:latest]
DEBU[0000] UserID not available in image metadata for [nginx:stable-alpine]
WARN[0000] Image [] not available in local image repo. Run "docker pull "
INFO[0000] [*collector.ImagesCollector] Done
INFO[0000] Collection done
INFO[0000] Collect Output in [/Users/akash/github/move2kube-demos/m2k_collect]. Copy this directory into the source directory to be used for planning.

There was a image with blank name getting appended to the images list because of \n character at the end of outputStr - [quay.io/konveyor/move2kube-ui:latest nginx:stable-alpine ].
Signed-off-by: Akash Nayak<EMAIL_ADDRESS>

Can we use json as the format? https://stackoverflow.com/questions/50285252/docker-images-output-as-json-using-format It outputs in JSONL format
$ docker images --format '{{json .}}'
{"Containers":"N/A","CreatedAt":"2021-12-01 19:05:10 +0530 IST","CreatedSince":"19 months ago","Digest":"\u003cnone\u003e","ID":"cc3b650a3f92","Repository":"registry.access.redhat.com/ubi8/go-toolset","SharedSize":"N/A","Size":"1.09GB","Tag":"1.16.7","UniqueSize":"N/A","VirtualSize":"1.086GB"}
{"Containers":"N/A","CreatedAt":"2021-11-16 17:33:48 +0530 IST","CreatedSince":"19 months ago","Digest":"\u003cnone\u003e","ID":"c04867c5b867","Repository":"quay.io/keycloak/keycloak","SharedSize":"N/A","Size":"717MB","Tag":"15.0.2","UniqueSize":"N/A","VirtualSize":"716.6MB"}
{"Containers":"N/A","CreatedAt":"2021-09-24 04:09:54 +0530 IST","CreatedSince":"21 months ago","Digest":"\u003cnone\u003e","ID":"df5477cea558","Repository":"hello-world","SharedSize":"N/A","Size":"9.06kB","Tag":"latest","UniqueSize":"N/A","VirtualSize":"9.056kB"}
{"Containers":"N/A","CreatedAt":"2021-01-25 22:46:49 +0530 IST","CreatedSince":"2 years ago","Digest":"\u003cnone\u003e","ID":"dd91347f1318","Repository":"gcr.io/cassandra-operator/cassandra-operator","SharedSize":"N/A","Size":"41.9MB","Tag":"latest","UniqueSize":"N/A","VirtualSize":"41.9MB"}
{"Containers":"N/A","CreatedAt":"2016-10-22 02:11:46 +0530 IST","CreatedSince":"6 years ago","Digest":"\u003cnone\u003e","ID":"1999482cb0a5","Repository":"mongo","SharedSize":"N/A","Size":"391MB","Tag":"2.6","UniqueSize":"N/A","VirtualSize":"391MB"}
{"Containers":"N/A","CreatedAt":"1980-01-01 05:30:01 +0530 IST","CreatedSince":"43 years ago","Digest":"\u003cnone\u003e","ID":"98ac5ea57540","Repository":"paketobuildpacks/builder","SharedSize":"N/A","Size":"1.97GB","Tag":"full","UniqueSize":"N/A","VirtualSize":"1.966GB"}

Using the '{{json .}}' format in place of '{{.Repository}}:{{.Tag}}', it captures lot of extra information that we do not need require, so should we still go for the json format.
$ docker images --format '{{.Repository}}:{{.Tag}}'
quay.io/konveyor/move2kube-ui:latest
nginx:stable-alpine

$ docker images --format '{{json .}}'
{"Containers":"N/A","CreatedAt":"2023-06-22 14:13:14 +0530 IST","CreatedSince":"6 days ago","Digest":"\u003cnone\u003e","ID":"a192738e1b92","Repository":"quay.io/konveyor/move2kube-ui","SharedSize":"N/A","Size":"212MB","Tag":"latest","UniqueSize":"N/A","VirtualSize":"212.1MB"}
{"Containers":"N/A","CreatedAt":"2023-06-15 08:29:26 +0530 IST","CreatedSince":"13 days ago","Digest":"\u003cnone\u003e","ID":"ba1fca8fb480","Repository":"nginx","SharedSize":"N/A","Size":"40.7MB","Tag":"stable-alpine","UniqueSize":"N/A","VirtualSize":"40.7MB"}


forbidden when access image candidates url from threads

maybe it's not an issue. but currently, I can't load any image candidates url, do you have any suggestions to solve this?

BTW, thanks for the API! 🔥

Hi, @gadingnst! Nice to see you experimenting. 👍
You need to proxy requests using a separate server. If you're using Next.js >= 12.3, you can use <Image /> and the config images. remotePatterns like the following:
/** @type {import('next').NextConfig} */
const nextConfig = {
  images: {
    remotePatterns: [
      {
        protocol: 'https',
        hostname: '**.cdninstagram.com',
      },
      {
        protocol: 'https',
        hostname: '**.fbcdn.net',
      },
    ],
  },
};

module.exports = nextConfig;

If not, you should use/implement something like Blazity/next-image-proxy.

solved. thanks, mate.


Add support for filtering and output of Decimal-typed fields.

Resolves #90.

Coveralls seems to have a bug -- the run completed and says it successfully submitted its coverage info, but Coveralls says it never got it. The issue is between the Coveralls client and server, so ignoring it for now.