Add transcoding example

After I learned and messed I think this is the best script:

[x] 1 - a transmuxing example - mp4 -> mp4 dbab188a9509884ee15e778e285721aba06a2d57
[x] 2 - a transmuxing example - mp4 -> fmp4) 9f316100b68fa7c6c9f280c733dd89a82c9f2e2f
[ ] 3 - a transcoding example - h264 -> h264 (fixed gop)
[ ] 4 - a transcoding example - h264 -> h265
[ ] 5 - a transrating example - h264 -> h264 (smaller bit rate)

Issues
Wanna help me?
If you want to test or run it locally make sure you have docker installed and that you had ran make fetch_small_bunny_video to download the sample video.
git clone --single-branch --branch transcoding-chapter https://github.com/leandromoreira/ffmpeg-libav-tutorial.git

make run_transcoding

# this is going to create the file bunny_1s_gop.mp4

Simple transmuxing and fmp4 is working!

https://github.com/leandromoreira/ffmpeg-libav-tutorial/pull/52

While coding the 3 38c6c782aa9530c61537e74b1e1781d59f3eddc8 it works but:

the fps has changed 59.99 fps

Video: h264, yuv420p, 1920x1080, 7632 kb/s, 59.99 fps, 60 tbr, 16384 tbn, 0.03 tbc (default)

a warning message has appeared

[libx264 @ 0x1e78200] forced frame type (5) at 597 was changed to frame type (3)

While dealing with 3 6ad6a82fb37f7b8d187f88e3e3ee3aeeb240e832 now it seems that the gop is fixed:

fps still seems to be variable

Frame rate mode                          : Variable
Frame rate                               : 60.000 FPS
Minimum frame rate                       : 56.302 FPS
Maximum frame rate                       : 60.015 FPS

a lot of warning messages have appeared:

[libx264 @ 0x1a5a200] forced frame type (5) at 80 was changed to frame type (3)
[libx264 @ 0x1a5a200] forced frame type (3) at 81 was changed to frame type (1)
[libx264 @ 0x1a5a200] forced frame type (5) at 140 was changed to frame type (3)
[libx264 @ 0x1a5a200] forced frame type (3) at 141 was changed to frame type (1)
[libx264 @ 0x1a5a200] forced frame type (5) at 200 was changed to frame type (3)
[libx264 @ 0x1a5a200] forced frame type (3) at 201 was changed to frame type (1)
[libx264 @ 0x1a5a200] forced frame type (5) at 310 was changed to frame type (1)
[libx264 @ 0x1a5a200] forced frame type (5) at 369 was changed to frame type (3)
[libx264 @ 0x1a5a200] forced frame type (5) at 370 was changed to frame type (1)
[libx264 @ 0x1a5a200] forced frame type (5) at 429 was changed to frame type (3)
[libx264 @ 0x1a5a200] forced frame type (5) at 430 was changed to frame type (1)
[libx264 @ 0x1a5a200] forced frame type (5) at 489 was changed to frame type (3)
...

✅ Playable in VLC

When trying to change the bit rate if you change only AVCodecContext->bit_rate is going to only change the nominal bit rate:
mediainfo bunny_1s_gop.mp4
Bit rate                                 : 39.9 Mb/s
Nominal bit rate                         : 2 000 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels

I tried to control rate by:
  int M = 100; // when I use 1000 it gets worst.
  sc->video_avcc->bit_rate = 2100 * M;
  sc->video_avcc->rc_max_rate = 2600 * M;
  sc->video_avcc->rc_min_rate = 2600 * M;
  sc->video_avcc->rc_buffer_size = 2 * 2600 * M;

The final bit rate is still no how it was suppose to be:
Bit rate                                 : 4 932 kb/s
Nominal bit rate                         : 210 kb/s

While I'm trying to figure out all the issues I went deeper into ffmpeg (the command line) "transcode" path source code:

main

transcode

transcode_init

init_output_stream

init_output_stream_encode

transcode_step

And I noticed that there are lots of lines of code to take care of guessing the frame rate, set up the context (timing, streaming params) to encode a video stream and many others checks and adjustments.
I think maybe it'll be too time-consuming for me and too hard for the readers to grasp 😞 therefore I think I'll reduce the chapter to: transmuxing (mp4->mp4, mp4->fmp4) and transcoding (h264->h264 fixed gop, h264->h265)

I just put libvpx-vp9 as the video codec and it worked BUT again with a huge bit rate 😕
    Stream #0:0(und): Video: vp9 (Profile 0) (vp09 / 0x39307076), yuv420p(tv), 1920x1080, 93847 kb/s, 60 fps, 60 tbr, 15360 tbn, 15360 tbc (default)


Idea: Saturn-like gas giants? Other planet types?

I'd love to see gas giants that looked like Saturn, pale yellow with rings, and you could choose to generate terrestrial planets with rings as well.
Also, I'd like to see other planet types, such as habitable desert planets (The whole planet is one vanilla desert biome), jungle planets, Venus-like planets, And possibly also plains, savannah and taiga planets as well. Basically a planet type for each main biome,
so these biomes would be made into planet types:
Desert
Plains
Roofed Forest
Taiga
Ice Plains
Extreme Hills
Ocean
Mesa
Jungle
Mushroom
And also Venus- and Saturn-like planets. I'm hoping to recreate the entire Solar System in this mod instead of Galacticraft.

I'll add rings to the todo list for the first content update.
Also single biome planets are already a thing.  You can either configure biomes allowed to be the only biome on a planet via the "SingleBiomes" option in the Advanced Rocketry config or you can specify manually used the advanced planet XML config as detailed here: http://<IP_ADDRESS>:8000/AdvancedRocketry/concepts/AdvancedPlanetConfiguration.php

What I meant was different planet icons and LEO images so that planets with those biomes would look different from Earthlike planets.
Also ,will this update be for 1.7.10? Because most of my mods haven't updated to 1.10.2 yet and I don't plan on upgrading until they do.

I'd have to devise a method of doing that that would not destroy frame
rates, probably would involve layering several noise images with different
opacities and colors.  Also I plan to support 1.7.10 unless I explicitly
state otherwise.

Thanks for letting me know. I can tell 1.7.10 is still the most popular version of the game.
Do you think you might be able to add horizontal thrusters as well as the vertical ones, and perhaps antimatter or fusion engines which require the respective fuel in order to work, but have much higher thrust to weight ratios? Like Antimatter would have 20:1 and Fusion would have  10:1?  So you'd only need 1 antimatter engine to put a 20 block rocket into orbit?

Also, what about binary star systems?

Binary stars would be interesting to have.  Not quite sure what application horizonal rockets would have though.  Could you elaborate on that?

Is there a way to change the size of a planet or star in the PlanetDefs.xml?
I didn t found a way to that, am I missing sth?
And is it possible to enable rendering for moons when looking at a solarsystem?

Planet sizes are directly proportional to the planet's Gravitational Multiplier, it's not actually stated in the docs currently and I will make sure to add it.
the size of a star cannot be changed as of right now.
I can look into rendering moons around planets in the solar view.  I've been apprehensive to do that because it already feels somewhat cluttered.

ty for the quick response

I'm back.
Well, the reason I suggested horizontal rockets is because having a horizontal warp ship or a shuttle with vertical rockets when the ship is parallel to the ground looks quite silly.
I use a small shuttle to go from my warp ship to the planet the ship is orbiting, and the ship is horizontal but the engines are on the bottom which looks strange. I'd prefer if you could have horizontal rockets that would do one of two things:
If on a warp ship: They'd decrease the warp time and allow for moving the ship into different areas of the orbit (To dock with a station directly in front of and above the ship, for example)
And if on a rocket: They'd act like vertical thrusters but just look better on certain designs.

Horizontally launched rockets have their own tracker, binay/ n-ary star systems implemented.


C.btrfs_get_subvol into subvolMap call

https://github.com/stevvooe/go-btrfs/issues/2

LGTM
May have to confirm that this works correctly with subpaths.

Yes, I'm not certain that it was an idempotent transform ...

@gliptak The btrfs-tools c code is a little obfuscated but they are similar. It looks like they both follow through and correctly resolve the mnt path, so this looks right.
To be honest, it is very hard to tell from btrfs what is correct.


Issues on evaluating latency using TVM.

Hi I’m currently working on compiling I-ViT using TVM. On this project, The error appears.

Check failed: value < 1LL << (dtype.bits() - 1) (192 vs. 128) : ValueError: Literal value 192 exceeds maximum of int8

by changing value 192 lower than 128 on build_model.py seems to sove the issue.
   if name == 'deit_tiny_patch16_224':         #embed_dim = 192         embed_dim = 92          num_heads = 3
But, strictly speaking, 'But this method' involves arbitrarily modifying the model's structure, so it is not an appropriate solution.
should changing TVM's version solve this issue?
Thanks always.

@rkdgmlqja
I have a same issue.
Did you solve it?

The Error was caused by quantized layer norm from layers.py
def quantized_layernorm(data,
                    bias_int):
    data = relay.cast(data, 'int32')
    mean = relay.mean(data, axis=2, keepdims=True)
    data = data - mean
    data_sq = data * data

This should fix it

hi,  @rkdgmlqja . did you successfully run tvm deployment(evaluate_accuracy.py, evaluate_latency.py)?
I got stuck with it.
I've tried it in the same environment as the author guided(tvm 0.9dev0 from source build, timm 0.4.2).
but the inference results(top 5) was totally different from torch.
also, I always failed in the auto tuning progress with a unknown sudden error.
what version of tvm, timm or some other settings did you use? is there some more things I should change to run it correctly?
if possible please help me~!


Added a registar test

Added test to ensure that registar appends n function lists when applied to the same stream

Also, I was thinking of changing registrat implementation to use sets instead of lists, but then I wondered if that could break declaring the same tests with different integrations attached - we can probably talk about that tomorrow


Added a method to get ip of node

build_scripts/deploy.sh uses the statement
"python -m jiocloud.utils get_ip_of_node" in multiple
places, So added a function in common.sh to pack it.

my instinct is to reject and close this. Having a method in bash to wrap a single call to something that has already been encapsulated as a python command line invocation seems like too much extra code for no value. Feel free to disagree if you feel passionate about this patch. /cc @sorenh @hkumarmk in case you guys want to disagree.


Trying to create a Mass Action Mapping

Doug,
Love the concept of this app, it solves a glaring business need for us. I have been trying to install and use correctly, but I have run into a number of issues. I had previously been unable to choose an action, but today I noticed you released an update that would bypass the need for named credentials. I uninstalled the package and reinstalled the new version.
I am now having another issue when trying to create a new mass action mapping. Check out this screenshot;

I am prompted to either link to a previous config, or create a new mass action configuration. Since I don't have any created, I went to create a new record but I am just linked back to the same screen and unable to create anything. What should I do? Have I installed this incorrectly?
Thanks,
Brian Purcell

Strange, I just went and tried to set up the process again and it appears to be working!

Hi @bpurcell89, thanks for your interest in Mass Action Scheduler.
Per the earlier screen shot, it looks like you were accidentally on the “Mass Action Mapping” object instead of “Mass Action Configuration”.  Per you update, sounds like you eventually got to the right page. Sorry for any confusion / lack of intuitive UI there with such similar object names.
You should only ever have to go to the “Mass Action Configuration” object and use the “Configure” wizard to create your configuration and mappings. I don’t recommend editing the record directly (eg data loader, Classic UI, etc) or it’s mappings related list as it could cause a config record to not work and you’d need to recreate it.


Creating a new file from the terminal helper throws a ENOENT exception

(sorry if this is a dup, hard to narrow down the search of open issues -- a bunch of ENOENT cases)
If I want to create a new file with the atom helper, an exception is thrown:
~/Work/iojs-build ±master $ atom mynewfile

..throws:
events.js:141
Hide Stack Trace
Error: ENOENT: no such file or directory, open '/Users/jbergstroem/Work/iojs-build/mynewfile'
  at Error (native)

I can't seem to reproduce this with Atom 0.195.0-910cc50 on OS X 10.10.3 - what version of Atom is this? What OS?

@mnquintana 0.194.0 on OS X 10.10.3. If I wasn't clear enough, "mynewfile" doesn't exist. I can reproduce this on every attempt.

Can you try running atom --safe filethatdoesntexist and see if you get the same error?

@mnquintana can't reproduce any longer. Means I need to triage addons, I guess?

@jbergstroem Pretty much - can you post a list of your installed packages (apm list --installed)?

@mnquintana ran through them all. After disabling statusbar (from core) I could no longer reproduce. I was hoping for some kind of combination with another package, but status bar was one of the last 5 I disabled.

This is a duplicate of #6372 but I'll keep this open until you find the offending package combination :).

@50Wliu Ok.
I find triaging very cumbersome. I'm in a position where I'm able to reproduce it, disable a package, try opening a non-existent file and essentially rinse and repeat. Once I hit a package I then enable it again, followed by the next package I disabled - but now everything works. Is there some caching or similar I'm not aware of although I open/close atom between testing?

/cc @benogle @kevinsawicki

but now everything works. Is there some caching or similar I'm not aware of although I open/close atom between testing

Some packages may not properly implement activate/deactivate so enabling/disabling them without reloading may not highlight the problem.
You might want to open ~/.atom/packages in Atom and then do a search for fs.open , fs.open(, fs.readFile , and fs.readFile( and see what packages use that API, that could help narrowing things down.


New short section on AI/ML

People will use AI & ML whether or not we say anything, so let's say something. There's a lot we can say, here is what I think are the basics.
Unfortunately every evasion countermeasure I've seen analyzed has been shown ineffective. That's hard to word, but I think a warning is needed. Here's my chop:

As of this writing we don't know of any countermeasure for evasion
(adversarial inputs) with a strong industry-wide
confidence that it fully counters
determined adversaries (we would love to know of one).

I looked years ago & came up empty. But that may be old news. I'd love to get an update.
Signed-off-by: David A. Wheeler<EMAIL_ADDRESS>

Note: Click on the "Files changed" tab to see the specific proposed text.

I'm told there's a typo in this PR. I'm awaiting a tweaked PR or comment specifics to tell me where, so I can fix it and then merge this change. My intent is to merge this soon (once the typo is fixed).


fix: add full support of nuxt public runtime config

Types of changes

[ ] Chore (a non-breaking change which is related to package maintenance)
[X] Bug fix (a non-breaking change which fixes an issue)
[ ] New feature (a non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Description
Today, the public nuxt runtime config is not fully supported. For the module to work, we need to set options in the root level of the nuxt configuration file.
So, I added the full support of the Nuxt runtime config to avoid double config and only use the PublicRuntimeConfig
export default defineNuxtConfig({
  modules: ['@nuxtjs/prismic'],
  runtimeConfig: {
    public: {
      prismic: {
        endpoint: '200629-sms-hoy',
        clientConfig: {
          routes: [
            {
              type: 'kitchen_sink_2',
              path: '/'
            },
          ]
        }
      }
    }
  }
  // primisic: {} <-- is no longer necessary but always works 🔥⚠
})

Checklist:

[X] My change requires an update to the official documentation.
[X] All TSDoc comments are up-to-date and new ones have been added where necessary.
[X] All new and existing tests are passing.

Very much interested in that.

Hey @LouisMazel, @sylr, sorry for the lack of feedback so far, I'll review the above.

Might be tomorrow! I'm just giving the opportunity to @LouisMazel to share their feedback on the above first

Available in RC.5, thank you! 🎉

According to a 3rd party developer which is using this plugin for a site they do for us this runtime configuration is only applied when setting up the plugin at build time and not at the runtime.
Can you confirm ?

https://github.com/nuxt-modules/prismic/issues/192


[SPARK-49000][SQL][WIP] Fix "select count(distinct 1) from t" where t is empty table

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

This approach is a dead end. Continuing the effort in https://github.com/apache/spark/pull/47525.


Rename SpecialistDocument js to MarkdownPreview

This JavaScript is used to add a markdown preview function to the edit
pages for Manuals and Sections. It feels like MarkdownPreview is a
better name for it than SpecialistDocument.
We came across this because we're naming SpecialistDocument to
Section in another branch. When it came to renaming this we felt
that giving it a more generic name made more sense.

I'm not totally convinced about this namespace rename, because the addPreviewFeature function (a) duplicates part of the new namespace and (b) implies that the intention is/was to have multiple "features" within a single namespace.
I might be more comfortable with it if the function was also renamed from addPreviewFeature to init or initialise.
Anyway, I'm going to approve this and let you decide whether to do anything about my comments.

We've had a look at the other JavaScript in the app and they don't appear to share a consistent approach/style. We think the rename in this PR is an improvement so we'll merge it as it is and come back to the style if/when it becomes a problem.


Tutorial on "follow the workload"

Closes #1843

This change is

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/94a8385839ce9cb40e628290adf610560fd9259f/

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/f68a5ecf6089a55e7c37509f0a8ce928fca3622f/

@a-robinson, @awoods187, PTAL. The overview section is still missing some images to clarify the behavior (ideally animated gifs), but the rest should be pretty solid. I haven't seen the blog post on this topic yet, so I hope this serves as a good companion.
@spencerkimball and @Amruta-Ranade, would appreciate your eyes on this, too.

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/8489e5ac0e76209f37a5b6fffc4ca779a02f9b40/

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/3dd69648bbf0c15087be6e859e572384810d4a33/

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/22b6d5003f7f86d4db202eb537c325347090ef83/

Added images, so this is now a complete draft.

This is great, Jesse. I gave it a quick glance-over and wanted some quick detail about how a user would be able to implement this feature on their own cluster, and didn't find a "scannable" solution. Maybe adding a section with the head "Implement Follow the Workload on Your Cluster" would make it more obvious how to achieve it myself?

@sploiselle, all the user needs to do is start each node with the --locality flag. It's important for me to know that that doesn't come across on "scan". I'll work on that.

Review status: 0 of 8 files reviewed at latest revision, 4 unresolved discussions, some commit checks failed.

v1.1/demo-follow-the-workload.md, line 7 at r1 (raw file):

---

"Follow the workload" refers to CockroachDB's ability to dynamically optimize read latency for the location from which most of the workload is originating. This page describes how this works and walks you through a simple demonstration using a local cluster.

Nit but I think it should be "follow-the-workload"

v1.1/demo-follow-the-workload.md, line 25 at r1 (raw file):

### How It Works

"Follow the workload" is based on the way **range leases** handle read requests. Read requests bypass the Raft consensus protocol, accessesing the range replica with which the range lease is co-located (the leaseholder) and sending the results to the client without needing to coordinate with any of the other range replicas. Bypassing Raft, and the network roundtrips involved, is possible because the leaseholder is guaranteed to be up-to-date already due to the way write request are handled by Raft.

nit --"with which the range" is a bit hard to read

Comments from Reviewable

really like this tutorial explanation--i'm going to run through it later this week and will circle back with any additional feedback

Review status: 0 of 8 files reviewed at latest revision, 4 unresolved discussions, some commit checks failed.

Comments from Reviewable

I also really like this, nice work!

Reviewed 2 of 2 files at r1, 4 of 4 files at r2, 3 of 3 files at r3.
Review status: all files reviewed at latest revision, 9 unresolved discussions, some commit checks failed.

v1.1/demo-follow-the-workload.md, line 25 at r1 (raw file):

the range replica with which the range lease is co-located

Yeah, I think "the range replica which holds the range lease" would be much clearer.
Also, s/accessesing/accessing

v1.1/demo-follow-the-workload.md, line 25 at r1 (raw file):

due to the way write request are handled by Raft.

It's more due to the fact that all write requests also go to the leaseholder, so it's guaranteed to be up-to-date.

v1.1/demo-follow-the-workload.md, line 65 at r1 (raw file):

~~~

This command causes a 100 millisecond delay for all requests on your local workstation.

If we want to be more specific, it introduces delay for all requests on the loopback interface of your local workstation, meaning it will only effect connections from the machine to itself, not to/from the Internet. We don't necessarily need to be more specific, though.

v1.1/demo-follow-the-workload.md, line 133 at r1 (raw file):

    {% include copy-clipboard.html %}
    ~~~ shell
    $ kv -duration 1m -max-rate 100 'postgresql://root@localhost:26259?sslmode=disable'

Note that this means there will also be considerable latency between the kv process and the cockroachdb processes. That doesn't effect the demonstration of the feature at all, but it does make the latency/throughput numbers below look pretty terrible.
There are a couple options:

Add more concurrent clients using kv's -concurrency flag (e.g. -concurrency 1000). This won't help the latency numbers, but will make the throughput look less terrible.
Start the cockroach processes with --advertise-host=localhost instead of --host=localhost, then connect to them via kv with a connection string more like `"postgresql://root@$(hostname):26259?sslmode=disable". This way all the node-to-node traffic would be affected by comcast's rules, but the kv-to-cockroach traffic wouldn't. I'm not 100% sure this'll work since the kv traffic might end up on the loopback interface anyway. I'll check.

The two could also be done in tandem. I still need to test option 2. I tried it locally, but realized that not even your original instructions were working to initialize the cluster. It's possible that my laptop's network interface is messed up -- I'll try to figure out what's going on and whether there's anything wrong with the instructions.

v1.1/demo-follow-the-workload.md, line 152 at r1 (raw file):

    ~~~
    _elapsed___errors____________ops___ops/sec(cum)__p95(ms)__p99(ms)_pMax(ms)_____seq(begin/end)
      120.0s        0           2330           19.4    419.4    637.5   1610.6           0 / 2338

Nit, but this clearly ran for 2 minutes, which is out of sync with the recommendation to run for 1 minute.

v1.1/demo-follow-the-workload.md, line 154 at r1 (raw file):

      120.0s        0           2330           19.4    419.4    637.5   1610.6           0 / 2338

    BenchmarkBlocks     2330      51502608.4 ns/op

Is this garbage output from something else? I don't think it's from kv.

Comments from Reviewable

Review status: all files reviewed at latest revision, 10 unresolved discussions, some commit checks failed.

v1.1/demo-follow-the-workload.md, line 62 at r3 (raw file):

{% include copy-clipboard.html %}
~~~ shell
$ comcast --device lo0 --latency 100

This device name is going to be different on different machines. On a GCE VM, it was just lo, which is the standard name for Linux machines.

Comments from Reviewable

Review status: all files reviewed at latest revision, 11 unresolved discussions, some commit checks failed.

v1.1/demo-follow-the-workload.md, line 133 at r3 (raw file):

    {% include copy-clipboard.html %}
    ~~~ shell
    $ kv -duration 1m -max-rate 100 'postgresql://root@localhost:26259?sslmode=disable'

Also, this is going to 50% reads, 50% writes, but this demo is primarily about read latency, so I'd recommend setting -read-percent 100

Comments from Reviewable

Review status: all files reviewed at latest revision, 11 unresolved discussions, some commit checks failed.

v1.1/demo-follow-the-workload.md, line 133 at r1 (raw file):
Previously, a-robinson (Alex Robinson) wrote…
Note that this means there will also be considerable latency between the kv process and the cockroachdb processes. That doesn't effect the demonstration of the feature at all, but it does make the latency/throughput numbers below look pretty terrible.
There are a couple options:

Add more concurrent clients using kv's -concurrency flag (e.g. -concurrency 1000). This won't help the latency numbers, but will make the throughput look less terrible.
Start the cockroach processes with --advertise-host=localhost instead of --host=localhost, then connect to them via kv with a connection string more like `"postgresql://root@$(hostname):26259?sslmode=disable". This way all the node-to-node traffic would be affected by comcast's rules, but the kv-to-cockroach traffic wouldn't. I'm not 100% sure this'll work since the kv traffic might end up on the loopback interface anyway. I'll check.

The two could also be done in tandem. I still need to test option 2. I tried it locally, but realized that not even your original instructions were working to initialize the cluster. It's possible that my laptop's network interface is messed up -- I'll try to figure out what's going on and whether there's anything wrong with the instructions.

Alright, the second option doesn't fix it even if I manually put in in the host's non-loopback IP address, since the kernel is too smart. We should just bump the concurrency a bit to -concurrency 32, set -read-percent 100, and add a note here explaining why the latency visible in the kv output is 200ms instead of just a couple milliseconds (it's because the kv process's packets get the same 100ms-each-way delay as the packets between the cockroach processes -- this shows that the cockroach process receiving the request is the leaseholder since it doesn't have to consult the others).

Comments from Reviewable

Review status: all files reviewed at latest revision, 11 unresolved discussions, some commit checks failed.

v1.1/demo-follow-the-workload.md, line 133 at r1 (raw file):
Previously, a-robinson (Alex Robinson) wrote…
Alright, the second option doesn't fix it even if I manually put in in the host's non-loopback IP address, since the kernel is too smart. We should just bump the concurrency a bit to -concurrency 32, set -read-percent 100, and add a note here explaining why the latency visible in the kv output is 200ms instead of just a couple milliseconds (it's because the kv process's packets get the same 100ms-each-way delay as the packets between the cockroach processes -- this shows that the cockroach process receiving the request is the leaseholder since it doesn't have to consult the others).

(I should add that there are more advanced ways we could get around this, but they'd just distract from the main point of the demo)

Comments from Reviewable

Review status: all files reviewed at latest revision, 11 unresolved discussions, some commit checks failed.

v1.1/demo-follow-the-workload.md, line 133 at r1 (raw file):
Previously, a-robinson (Alex Robinson) wrote…
(I should add that there are more advanced ways we could get around this, but they'd just distract from the main point of the demo)

And I figured out what was going on on my laptop, I was accidentally using an old binary that didn't support the init command.

Comments from Reviewable

Review status: all files reviewed at latest revision, 11 unresolved discussions, some commit checks failed.

v1.1/demo-follow-the-workload.md, line 7 at r1 (raw file):
Previously, awoods187 (Andy Woods) wrote…
Nit but I think it should be "follow-the-workload"

Done.

v1.1/demo-follow-the-workload.md, line 25 at r1 (raw file):
Previously, a-robinson (Alex Robinson) wrote…

the range replica with which the range lease is co-located

Yeah, I think "the range replica which holds the range lease" would be much clearer.
Also, s/accessesing/accessing

Done.

v1.1/demo-follow-the-workload.md, line 25 at r1 (raw file):
Previously, a-robinson (Alex Robinson) wrote…

due to the way write request are handled by Raft.

It's more due to the fact that all write requests also go to the leaseholder, so it's guaranteed to be up-to-date.

Done.

v1.1/demo-follow-the-workload.md, line 65 at r1 (raw file):
Previously, a-robinson (Alex Robinson) wrote…
If we want to be more specific, it introduces delay for all requests on the loopback interface of your local workstation, meaning it will only effect connections from the machine to itself, not to/from the Internet. We don't necessarily need to be more specific, though.

Done.

v1.1/demo-follow-the-workload.md, line 133 at r1 (raw file):
Previously, a-robinson (Alex Robinson) wrote…
And I figured out what was going on on my laptop, I was accidentally using an old binary that didn't support the init command.

Done.

v1.1/demo-follow-the-workload.md, line 152 at r1 (raw file):
Previously, a-robinson (Alex Robinson) wrote…
Nit, but this clearly ran for 2 minutes, which is out of sync with the recommendation to run for 1 minute.

Done.

v1.1/demo-follow-the-workload.md, line 154 at r1 (raw file):
Previously, a-robinson (Alex Robinson) wrote…
Is this garbage output from something else? I don't think it's from kv.

I ran it again and still see that final line. But I don't think this summary is necessary anyway, so I've removed.

v1.1/demo-follow-the-workload.md, line 29 at r3 (raw file):
Previously, Amruta-Ranade (Amruta Ranade) wrote…
Nit: "--locality flag" can be hyperlinked to the locality flag documentation? Also, the locality flag documentation can be updated to reference this document.

Good call. I updated the --locality flag description and cross-referenced.

v1.1/demo-follow-the-workload.md, line 45 at r3 (raw file):
Previously, Amruta-Ranade (Amruta Ranade) wrote…
Nit: Can we phrase this as "In this tutorial, you'll use CockroachDB, comcast (a Go network tool....)..."
The way it reads now seems odd - as if CockroachDB is the Go network tool :)

Done.

v1.1/demo-follow-the-workload.md, line 62 at r3 (raw file):
Previously, a-robinson (Alex Robinson) wrote…
This device name is going to be different on different machines. On a GCE VM, it was just lo, which is the standard name for Linux machines.

Ah, right. Is there any easy command to find this device name on any machine?

v1.1/demo-follow-the-workload.md, line 133 at r3 (raw file):
Previously, a-robinson (Alex Robinson) wrote…
Also, this is going to 50% reads, 50% writes, but this demo is primarily about read latency, so I'd recommend setting -read-percent 100

Done.

Comments from Reviewable

Reviewed 1 of 5 files at r4, 4 of 4 files at r5.
Review status: 8 of 9 files reviewed at latest revision, 9 unresolved discussions.

v1.1/demo-follow-the-workload.md, line 154 at r1 (raw file):
Previously, jseldess wrote…
I ran it again and still see that final line. But I don't think this summary is necessary anyway, so I've removed.

Ah, I see. You're just running an older version of kv. The current version prints BenchmarkLoadgenKV. Removing it LGTM.

v1.1/demo-follow-the-workload.md, line 62 at r3 (raw file):
Previously, jseldess wrote…
Ah, right. Is there any easy command to find this device name on any machine?

I think we can just say to use lo0 if you're on Mac, lo if you're on Linux, and to check the output of ifconfig for the interface responsible for <IP_ADDRESS> if neither works. There might be an easy command out there somewhere, but I don't know it and a couple different google searches didn't turn up anything that does more than just listing the available interfaces.

v1.1/demo-follow-the-workload.md, line 65 at r6 (raw file):

~~~

This command causes a 100 millisecond delay for all requests on the loopback interface of your local workstation. It will only effect connections from the machine to itself, not to/from the Internet.

s/effect/affect

v1.1/demo-follow-the-workload.md, line 136 at r6 (raw file):

    ~~~

    This command initiates 32 concurrent read-only workloads for 1 minute but limits each worker to 100 operations per second (since you're running everything on a single machine). While `kv` is running, it will print some stats to the terminal:

It doesn't limit each "worker" to 100 operations per second, it limits the entire kv process to 100 operations per second.

v1.1/demo-follow-the-workload.md, line 148 at r6 (raw file):

    ~~~

    {{site.data.alerts.callout_info}}The latency numbers printed reflect the 100 millisecond delay caused by the comcast tool. Normally, these numbers would be much lower.{{site.data.alerts.end}}

Maybe it's just me, but this doesn't reflect the point I'd like to get across about how the delay is coming in between the kv process and cockroach, not within cockroach, and thus in a real deployment the latency would be drastically lower. This is a bit wordy, but for example:
"The latency numbers printed reflect are over 200 milliseconds because the 100 millisecond delay in each direction (200ms round-trip) caused by the comcast tool also applies to the traffic going from the kv process to the cockroach process. If you were to set up more advanced rules that excluded the kv process's traffic or to run this on a real network with real network delay, you'd see these numbers would be down in the single-digit milliseconds."

Comments from Reviewable

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/23cb680b6dd2fdb5ce8490cfe5514aca3ad4d7a9/

Review status: 8 of 9 files reviewed at latest revision, 9 unresolved discussions, all commit checks successful.

v1.1/demo-follow-the-workload.md, line 62 at r3 (raw file):
Previously, a-robinson (Alex Robinson) wrote…
I think we can just say to use lo0 if you're on Mac, lo if you're on Linux, and to check the output of ifconfig for the interface responsible for <IP_ADDRESS> if neither works. There might be an easy command out there somewhere, but I don't know it and a couple different google searches didn't turn up anything that does more than just listing the available interfaces.

Done.

v1.1/demo-follow-the-workload.md, line 65 at r6 (raw file):
Previously, a-robinson (Alex Robinson) wrote…
s/effect/affect

Done.

v1.1/demo-follow-the-workload.md, line 136 at r6 (raw file):
Previously, a-robinson (Alex Robinson) wrote…
It doesn't limit each "worker" to 100 operations per second, it limits the entire kv process to 100 operations per second.

Done.

v1.1/demo-follow-the-workload.md, line 148 at r6 (raw file):
Previously, a-robinson (Alex Robinson) wrote…
Maybe it's just me, but this doesn't reflect the point I'd like to get across about how the delay is coming in between the kv process and cockroach, not within cockroach, and thus in a real deployment the latency would be drastically lower. This is a bit wordy, but for example:
"The latency numbers printed reflect are over 200 milliseconds because the 100 millisecond delay in each direction (200ms round-trip) caused by the comcast tool also applies to the traffic going from the kv process to the cockroach process. If you were to set up more advanced rules that excluded the kv process's traffic or to run this on a real network with real network delay, you'd see these numbers would be down in the single-digit milliseconds."

Done. Thanks, Alex.

Comments from Reviewable

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/f8eadc32e6f2944655345a855241745a98718f6b/

:thumbsup:

Reviewed 1 of 6 files at r7.
Review status: 3 of 9 files reviewed at latest revision, 5 unresolved discussions, some commit checks failed.

v1.1/demo-follow-the-workload.md, line 62 at r3 (raw file):
Previously, jseldess wrote…
Done.

Super minor, but in case someone doesn't know what ifconfig is, it might be more understandable to say "check the output of running the ifconfig command to find the interface ..."

Comments from Reviewable

Review status: 3 of 9 files reviewed at latest revision, 5 unresolved discussions, some commit checks failed.

v1.1/demo-follow-the-workload.md, line 62 at r3 (raw file):
Previously, a-robinson (Alex Robinson) wrote…
Super minor, but in case someone doesn't know what ifconfig is, it might be more understandable to say "check the output of running the ifconfig command to find the interface ..."

Done.

Comments from Reviewable

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/dc8e9677427a120653f97747792de98b681f95be/

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/702c0f7fb48bb2976abb2de41d4a8ad6fb89f401/

Added a callout that the only use action required to enable this feature is using --locality on cockroach start.

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/edbe244a9725ee6d899f53ea0145040935d33598/


opt check and remove unavailable brpc stubs (#43212)

What problem does this PR solve?
pick #43212 , #44047 and #44481
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)

Test

[ ] Regression test
[ ] Unit Test
[ ] Manual test (add detailed scripts or steps below)
[ ] No need to test or manual test. Explain why:

[ ] This is a refactor/code format and no logic has been changed.
[ ] Previous test can cover this change.
[ ] No code files have been changed.
[ ] Other reason

Behavior changed:

[ ] No.
[ ] Yes.

Does this need documentation?

[ ] No.
[ ] Yes.

Check List (For Reviewer who merge this PR)

[ ] Confirm the release note
[ ] Confirm test cases
[ ] Confirm document
[ ] Add branch pick label

run buildall

run external

run p0

run buildall

run buildall


java.lang.UnsatisfiedLinkError: no jniopencv_videoio in java.library.path due to java.lang.UnsatisfiedLinkError: ... (libgtk-x11-2.0.so.0)

Hi,
I'm running scala (compiled using sbt) app that uses several OpenCV's functionalities. Eventually I managed to run some of them, but I still encounter the following exception:

Name: java.lang.NoClassDefFoundError
Message: Could not initialize class org.bytedeco.javacpp.opencv_objdetect$CascadeClassifier
StackTrace:

Notes:

My project/plugins.sbt file looks like this:

logLevel := Level.Warn
resolvers += Resolver.sonatypeRepo("releases")
addSbtPlugin("org.bytedeco" % "sbt-javacv" % "1.12")

My build.sbt looks like this:

name := "..."
version := "1.0"
scalaVersion := "2.11.8"
javaCppPresetLibs ++= Seq("opencv" -> "3.1.0", "ffmpeg" -> "3.1.2")
javaCppPlatform := Seq("linux-x86_64")

I'm coding+debugging on Windows machine where everything runs smoothly. Then I tried adding the linux-x86_64 platform to the build.sbt as in (2) and use sbt-assembly to package my linux-ready jar. I uploaded that jar into my linux cluster, then cloned the 3.1.0 version of opencv from _https://github.com/Itseez/opencv_ and ran cmake, make and make install.

Finally, in scala, this piece of code works great (uses org.bytedeco.javacpp.opencv_imgcodecs & org.bytedeco.javacpp.opencv_imgproc):

import org.bytedeco.javacpp.BytePointer
import org.bytedeco.javacpp.opencv_core.{Mat, _}
import org.bytedeco.javacpp.opencv_imgcodecs._
import org.bytedeco.javacpp.opencv_imgproc._
import org.bytedeco.javacpp.opencv_objdetect.CascadeClassifier

val bytes = ByteBuffer.wrap( /* some Byte array */ )
val imgCvMat = cvDecodeImageM(cvMat(1, bytes .array().length, CV_8UC1, new BytePointer(bytes )))
val imgMat = cvarrToMat(imgCvMat)

while this code:
val faceCascade = new org.bytedeco.javacpp.opencv_objdetect.CascadeClassifier( /* path to xml of classifier configuration */ )

fails with the following exception:

java.lang.UnsatisfiedLinkError: no jniopencv_videoio in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:784)
at org.bytedeco.javacpp.Loader.load(Loader.java:586)
at org.bytedeco.javacpp.Loader.load(Loader.java:503)
at org.bytedeco.javacpp.opencv_videoio.(opencv_videoio.java:14)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.bytedeco.javacpp.Loader.load(Loader.java:558)
at org.bytedeco.javacpp.Loader.load(Loader.java:503)
at org.bytedeco.javacpp.opencv_objdetect$CascadeClassifier.(opencv_objdetect.java:566)
... 52 elided
Caused by: java.lang.UnsatisfiedLinkError: /home/hadoop/.javacpp/cache/mlageinf.jar/org/bytedeco/javacpp/linux-x86_64/libjniopencv_videoio.so: libgtk-x11-2.0.so.0: cannot open shared object file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:767)
... 60 more

Note that theeres a missing libgtk-x11-2.0.so.0 - and I can't figure out what to do here

That's strange, I have not seen that error before. In the interest of being through, can you make sure you're using the latest JavaCPP by setting the following?
javaCppVersion := "1.3.1"

Also, ffmpeg's latest version is "3.2.1" I believe.
It might be possible that on that platform the shared library object wasnt packaged with the jar. @saudet any idea?

@lloydmeta thanks for the quick reply. I tried adding the version argument, same result here.
I'm using Amazon's AWS machines, and they have their own linux distribution so I also don't seem to find any way of installing this specific gtk version manually.

btw, looking at /myjar.jar/org/bytedeco/javacpp/linux-x86_64/ i have a a lot of libjni*.so files, none of them contain "gtk", so apparently it isn't packaged.

GTK comes preinstalled with pretty much any Linux distribution... except the one from Amazon that is? So it looks like we might have to do something about that. As usual, contributions are welcome!

I'm closing this issue for now because it doesn't seem to be related to the SBT plugin.
The discussion should probably continue on the javacpp-presets project, or this PR

For anyone that stumbles upon this later looking for answers:
If you are on Amazon Linux 2, sudo yum install gtk2-devel

Duplicate of https://github.com/bytedeco/javacpp-presets/issues/593


Move cobertura to the top-level pom and configure it to generate aggregate results.

This makes
$ mvn cobertura:cobertura -P all-modules

generate an aggregate coverage report at target/site/cobertura/index.html.

+1


Minor correction to backup documentation

Hi Alexander,
I've been using your excellent scripts to get SSL-certificate on my UDMP - thanks!
I found, when following your documentation, a few minor errors in the area related to backup of the original SSL-certificate and have made this PR for it.
Best regards,
Jacob

Thank you Jacob @jvandborg for fixing this!


Strip shell's backslashes from the computed include

Conditional C/C++ Preprocessor: Strip shell's backslashes from the computed include (-DFOO_H="foo.h")
Related discussion: https://community.platformio.org/t/ldf-not-following-environment-variables-in-6-1-0rc1/28616/5?u=ivankravets
Contributor Checklist:

[x] I have created a new test or updated the unit tests to cover the new/changed functionality.
[X] I have updated CHANGES.txt (and read the README.rst)
[x] I have updated the appropriate documentation

The example is here https://github.com/SCons/scons/blob/master/SCons/Scanner/CTests.py#L449
We use SCons.Scanner.C.CConditionalScanner() as a helper to find project dependencies (libraries). See https://github.com/platformio/platformio-core/blob/develop/platformio/builder/tools/piolib.py#L368
I don't know how to switch SCons to the CConditionalScanner for the generic C/C++ development.

I guess this isn't really the place to store this for posterity, since it's off-topic for the completed PR, but I find with some experiments that this works inside an SConscript:
import SCons.Scanner.C
CConditionalScanner = SCons.Scanner.C.CConditionalScanner()
SourceFileScanner.add_scanner('.c', CConditionalScanner)
# do more of those as needed for other suffixes (see $CPPSUFFIXES)

add_scanner doesn't do any checking for "already exists", so it works fine to update/replace; it then calls add_skey with the suffix, so you'll have duplicate suffixes in the list of scanner keys, but that doesn't cause any problems, it seems.
@bdbaddog is there someplace you want me to put that? Cookbook maybe?


Create a repo: aries-acapy-plugins

Please include:

GitHub repo to create: aries-acapy-plugins
Project to be added to: Aries
Teams with which access: (examples on GitHub team list)

Team name: Please added the ACA-Py teams to be maintainers/admins/etc.
Role:

Read
Triage
Write
Manage
Admin

Thanks!

@dbluhm @usingtechnology

here you go!


Fixed linting errors

Fixed a large, large number of linting errors by:

Running eslint --fix on several files
Creating new linting rules for test files (We use this approach in
cfgov-refresh)
Chopping up extra lines in some of our tests

Testing

Without merging #382, you can run eslint by running npm i eslint -g on each affected file

eslint gulpfile.js
eslint test/accessibility/wcag.js
eslint test/cf-expandables.js
eslint test/cf-tables.js

Review

@anselmbradford

Screenshots
Before
lint:build

lint:scripts

lint:tests

After
Nothing for lint:build
lint:scripts

lint:tests

👍 Add something like **capital-framework:** [PATCH] Added JavaScript linting task to the changelog.

👉 f27de32


brew cask download fails when cache directory is missing

General troubleshooting steps

[x] I have retried my command with --force and the issue is still present.
[x] I have checked the instructions for reporting bugs (or making requests) before opening the issue.

[x] None of the templates was appropriate for my issue, or I’m not sure.

[x] I ran brew update-reset && brew update and retried my command.
[x] I ran brew doctor, fixed as many issues as possible and retried my command.
[x] I understand that if I ignore these instructions, my issue may be closed without review.

Description of issue
brew cask upgrade fails downloading if ~Library/Caches/Homebrew/Cask is missing. After manually creating the Cask folder it works fine again.
brew doctor or --force did not create the folder. It should be created before starting to download packages.
Command that failed
$ brew cask upgrade
==> Upgrading 11 outdated packages, with result:
…
==> Satisfying dependencies
==> Downloading …
Warning: Failed to create the file
Warning: ~/Library/Caches/Homebrew/Cask/….incomp
Warning: lete: No such file or directory

curl: (23) Failed writing body (0 != 16084)
==> Purging files for version … of Cask …
Error: Download failed on Cask '…' with message: Download failed: …

Output of command with --force --verbose --debug
==> Upgrading 11 outdated packages, with result:
…
==> Started upgrade process for Cask …
==> Hbc::Installer#fetch
==> Satisfying dependencies
==> Downloading
==> Downloading …
/usr/bin/curl -q --show-error --user-agent Homebrew/1.6.2-33-g27ec9da (Macintosh; Intel Mac OS X 10.13.4) curl/7.54.0 --fail --location --remote-time --continue-at - --output ~/Library/Caches/Homebrew/Cask/….incomplete https://…
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Failed to create the file
Warning: ~/Library/Caches/Homebrew/Cask/….incomp
Warning: lete: No such file or directory
  0 23.7M    0 16084    0     0  62688      0  0:06:37 --:--:--  0:06:37 62828
curl: (23) Failed writing body (0 != 16084)
==> Purging files for version … of Cask …
Error: Download failed on Cask '…' with message: Download failed: …
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/download.rb:42:in `rescue in fetch'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/download.rb:40:in `fetch'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/download.rb:15:in `perform'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/installer.rb:126:in `download'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/installer.rb:51:in `fetch'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/upgrade.rb:56:in `block in run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/upgrade.rb:29:in `each'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/upgrade.rb:29:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/abstract_command.rb:35:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:98:in `run_command'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:168:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:132:in `run'
/usr/local/Homebrew/Library/Homebrew/cmd/cask.rb:8:in `cask'
/usr/local/Homebrew/Library/Homebrew/brew.rb:101:in `<main>'
Error: Kernel.exit
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:173:in `exit'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:173:in `rescue in run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:156:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:132:in `run'
/usr/local/Homebrew/Library/Homebrew/cmd/cask.rb:8:in `cask'
/usr/local/Homebrew/Library/Homebrew/brew.rb:101:in `<main>'

Output of brew cask doctor
Warning: Unbrewed dylibs were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected dylibs:
…

Warning: You have unlinked kegs in your Cellar
Leaving kegs unlinked can lead to build-trouble and cause brews that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
…

Output of brew tap
caskroom/cask
caskroom/drivers
caskroom/eid
caskroom/fonts
caskroom/versions
…
homebrew/bundle
homebrew/core
homebrew/dupes
homebrew/gui
homebrew/services
homebrew/versions
…

Please post brew cask doctor

Please post brew cask doctor

Sorry, missed that. Fixed above.

Reproducible. Only occurs with upgrade, not fetch, install or reinstall.

I didn't manually delete the directory. I tried reproducing this brew [cask] cleanup [-s] but the directory did not disappear. So I'm not sure. I think this has happened before, but as [re]install seems to restore it the issue might have been not immediately visible for me.

I'm seeing this issue as well. For some reason Travis CI seems to not have a cache folder for Homebrew Cask, which causes all brew upgrades to fail.


Live sync Aug25

PWA changes
Patch changes from Azure

Docs Build status updates of commit 3e77d89:
:warning: Validation status: warnings

File
Status
Preview URL
Details

microsoft-community-training/frequently-asked-questions/faqs-general.md
:warning:Warning
View
Details

microsoft-community-training/content-management/create-content/create-course-category/content-marketplace.md
:bulb:Suggestion
View
Details

microsoft-community-training/frequently-asked-questions/faqs-user-management.md
:bulb:Suggestion
View
Details

microsoft-community-training/infrastructure-management/configure-your-platform-infrastructure/Add-multiple-AAD-to-B2C-as-a-social-account.md
:bulb:Suggestion
View
Details

microsoft-community-training/infrastructure-management/install-your-platform-instance/configure-multiple-authentications-in-a-single-instance.md
:bulb:Suggestion
View
Details

microsoft-community-training/infrastructure-management/monitor-your-platform-infrastructure/troubleshoot-guide-for-azure-support.md
:bulb:Suggestion
View
Details

microsoft-community-training/learning-in-offline-mode/azure-IOT-edge-device-setup.md
:bulb:Suggestion
View
Details

microsoft-community-training/learning-in-offline-mode/azure-IOT-hub-initial-setup.md
:bulb:Suggestion
View
Details

microsoft-community-training/user-management/manage-users/De-assigning-content-from-user.md
:bulb:Suggestion
View
Details

microsoft-community-training/whats-new-in-microsoft-community-training/2020/whats-new-in-february-2020.md
:bulb:Suggestion
View
Details

microsoft-community-training/analytics/analytics-overview.md
:white_check_mark:Succeeded
View

microsoft-community-training/analytics/custom-reports/database-schema.md
:white_check_mark:Succeeded
View

microsoft-community-training/analytics/in-built-reports/category-view-report.md
:white_check_mark:Succeeded
View

microsoft-community-training/analytics/in-built-reports/course-view-report.md
:white_check_mark:Succeeded
View

microsoft-community-training/analytics/in-built-reports/group-view-report.md
:white_check_mark:Succeeded
View

microsoft-community-training/analytics/in-built-reports/learner-report-card-view.md
:white_check_mark:Succeeded
View

microsoft-community-training/analytics/in-built-reports/overall-summary.md
:white_check_mark:Succeeded
View

microsoft-community-training/content-management/content-management-overview.md
:white_check_mark:Succeeded
View

microsoft-community-training/content-management/create-content/create-course-category/add-assessments-to-a-course.md
:white_check_mark:Succeeded
View

microsoft-community-training/content-management/create-content/create-course-category/add-feedback-form-for-a-course.md
:white_check_mark:Succeeded
View

microsoft-community-training/content-management/create-content/create-course-category/add-subtitles-or-captions-to-video-content.md
:white_check_mark:Succeeded
View

microsoft-community-training/content-management/create-content/create-course-category/create-a-category.md
:white_check_mark:Succeeded
View

microsoft-community-training/content-management/create-content/create-course-category/create-a-new-course.md
:white_check_mark:Succeeded
View

microsoft-community-training/content-management/create-content/create-course-category/publishing-course.md
:white_check_mark:Succeeded
View

microsoft-community-training/content-management/create-content/create-course-category/upload-content-to-a-course.md
:white_check_mark:Succeeded
View

This comment lists only the first 25 files in the pull request.

microsoft-community-training/frequently-asked-questions/faqs-general.md

Line 196, Column 1: [Warning: hard-coded-locale - See documentation] Link 'https://docs.microsoft.com/en-us/azure/active-directory-b2c/language-customization?pivots=b2c-user-flow#support-requested-languages-for-ui_locales' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.
Line 196, Column 1: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://docs.microsoft.com/en-us/azure/active-directory-b2c/language-customization?pivots=b2c-user-flow#support-requested-languages-for-ui_locales' will be broken in isolated environments. Replace with a relative link.

microsoft-community-training/content-management/create-content/create-course-category/content-marketplace.md

Line 20, Column 1: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://docs.microsoft.com/learn/support/faq?pivots=general' will be broken in isolated environments. Replace with a relative link.

microsoft-community-training/frequently-asked-questions/faqs-user-management.md

Line 24, Column 239: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://docs.microsoft.com/azure/active-directory-b2c/overview' will be broken in isolated environments. Replace with a relative link.

microsoft-community-training/infrastructure-management/configure-your-platform-infrastructure/Add-multiple-AAD-to-B2C-as-a-social-account.md

Line 5, Column 12: [Suggestion: ms-author-invalid - See documentation] Invalid value for 'ms.author', 'v-samarth' is not a valid Microsoft alias.

microsoft-community-training/infrastructure-management/install-your-platform-instance/configure-multiple-authentications-in-a-single-instance.md

Line 14, Column 92: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://docs.microsoft.com/azure/active-directory-b2c/overview' will be broken in isolated environments. Replace with a relative link.
Line 24, Column 21: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://docs.microsoft.com/azure/active-directory-b2c/tenant-management#get-your-tenant-name' will be broken in isolated environments. Replace with a relative link.
Line 79, Column 19: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://docs.microsoft.com/azure/active-directory-b2c/tenant-management#get-your-tenant-name' will be broken in isolated environments. Replace with a relative link.
Line 80, Column 277: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://docs.microsoft.com/azure/active-directory/fundamentals/active-directory-how-to-find-tenant' will be broken in isolated environments. Replace with a relative link.

microsoft-community-training/infrastructure-management/monitor-your-platform-infrastructure/troubleshoot-guide-for-azure-support.md

Line 25, Column 63: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://docs.microsoft.com/azure/azure-resource-manager/managed-applications/' will be broken in isolated environments. Replace with a relative link.

microsoft-community-training/learning-in-offline-mode/azure-IOT-edge-device-setup.md

Line 18, Column 8: [Suggestion: preserve-view-not-set - See documentation] You've pinned this link to a specific version of content with the view parameter. It's recommended not to pin a version unless that version is A) not the default view and B) the context is about that version specifically. To proceed with pinning a version add the &preserve-view=true to the URL.  Otherwise, remove the view parameter. URL: /azure/iot-edge/how-to-register-device?view=iotedge-2018-06&tabs=azure-portal
Line 22, Column 8: [Suggestion: preserve-view-not-set - See documentation] You've pinned this link to a specific version of content with the view parameter. It's recommended not to pin a version unless that version is A) not the default view and B) the context is about that version specifically. To proceed with pinning a version add the &preserve-view=true to the URL.  Otherwise, remove the view parameter. URL: /azure/iot-edge/how-to-install-iot-edge?view=iotedge-2018-06
Line 43, Column 8: [Suggestion: preserve-view-not-set - See documentation] You've pinned this link to a specific version of content with the view parameter. It's recommended not to pin a version unless that version is A) not the default view and B) the context is about that version specifically. To proceed with pinning a version add the &preserve-view=true to the URL.  Otherwise, remove the view parameter. URL: /azure/iot-edge/how-to-deploy-modules-cli?view=iotedge-2018-06

microsoft-community-training/learning-in-offline-mode/azure-IOT-hub-initial-setup.md

Line 63, Column 155: [Suggestion: preserve-view-not-set - See documentation] You've pinned this link to a specific version of content with the view parameter. It's recommended not to pin a version unless that version is A) not the default view and B) the context is about that version specifically. To proceed with pinning a version add the &preserve-view=true to the URL.  Otherwise, remove the view parameter. URL: /sql/relational-databases/security/password-policy?view=sql-server-ver15

microsoft-community-training/user-management/manage-users/De-assigning-content-from-user.md

Line 5, Column 12: [Suggestion: ms-author-invalid - See documentation] Invalid value for 'ms.author', 'v-samarth' is not a valid Microsoft alias.

microsoft-community-training/whats-new-in-microsoft-community-training/2020/whats-new-in-february-2020.md

Line 43, Column 31: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://docs.microsoft.com/azure/industry/training-services/microsoft-community-training/' will be broken in isolated environments. Replace with a relative link.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Issue when running training code

Firstly, I'd like to that you for sharing this opensource version of OpenDVC!
I was able to run through the inference on the BasketBall pass dataset, however, I ran into some issues while trying to train the model using the train instructions given
python OpenDVC_train_PSNR.py --l 1024

I get the following error -
2021-02-14 07:13:48.321268: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA
2021-02-14 07:13:48.337753: I tensorflow/core/common_runtime/process_util.cc:69] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
2021-02-14 07:14:06.276314: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at mkl_slice_op.cc:303 : Aborted: Operation received an exception:Status: 5, message: could not create a view primitive descriptor, in file tensorflow/core/kernels/mkl_slice_op.cc:300

I am currently using TF 1.12 and other libraries as specified in the run instructions and have installed a compatible version of MKL and MKLDNN.
Would you have any insights on what might be causing this issue? Any help would be really appreciated!

I am not so sure why the error occurs, but the inference and training codes should depend on the same packages. If the inference is unable, it seems that the environment is correct. Are you training on CPU or GPU (what kind of GPU?)? Maybe could try to train on a better device. Also, please paste the whole log so that we could look into it better.

Thank you for the fast response!
I'm running the codes on GPU in a DGX system. This has 4 32 GB V100s, so hopefully, the device is not an issue. One potential
Here's the full error log -
'''
(RLVC) csr_guest@csr-dgx1-03:~/aniranja/RLVC/OpenDVC$ python OpenDVC_train_PSNR.py --l 1024
/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:532: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
2021-02-15 23:28:49.746333: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA
2021-02-15 23:28:49.763038: I tensorflow/core/common_runtime/process_util.cc:69] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
2021-02-15 23:29:07.956119: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at mkl_slice_op.cc:303 : Aborted: Operation received an exception:Status: 5, message: could not create a view primitive descriptor, in file tensorflow/core/kernels/mkl_slice_op.cc:300
Traceback (most recent call last):
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
return fn(*args)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.AbortedError: Operation received an exception:Status: 5, message: could not create a view primitive descriptor, in file tensorflow/core/kernels/mkl_slice_op.cc:300
[[{{node gradients/flow_motion/flow_cnn_4/concat_grad/Slice_2}} = _MklSlice[Index=DT_INT32, T=DT_FLOAT, _kernel="MklOp", _device="/job:localhost/replica:0/task:0/device:CPU:0"](gradients/flow_motion/flow_cnn_4/conv2d/Conv2D_grad/Conv2DBackpropInput, ConstantFolding/gradients/flow_motion/flow_cnn_1/concat_grad/ConcatOffset-folded-2, gradients/flow_motion/flow_cnn_4/concat_grad/Shape_2, gradients/flow_motion/flow_cnn_4/conv2d/Conv2D_grad/Conv2DBackpropInput:1, DMT/_476, DMT/_477)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "OpenDVC_train_PSNR.py", line 166, in
learning_rate: lr})
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
run_metadata_ptr)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
run_metadata)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.AbortedError: Operation received an exception:Status: 5, message: could not create a view primitive descriptor, in file tensorflow/core/kernels/mkl_slice_op.cc:300
[[node gradients/flow_motion/flow_cnn_4/concat_grad/Slice_2 (defined at OpenDVC_train_PSNR.py:102)  = _MklSlice[Index=DT_INT32, T=DT_FLOAT, _kernel="MklOp", _device="/job:localhost/replica:0/task:0/device:CPU:0"](gradients/flow_motion/flow_cnn_4/conv2d/Conv2D_grad/Conv2DBackpropInput, ConstantFolding/gradients/flow_motion/flow_cnn_1/concat_grad/ConcatOffset-folded-2, gradients/flow_motion/flow_cnn_4/concat_grad/Shape_2, gradients/flow_motion/flow_cnn_4/conv2d/Conv2D_grad/Conv2DBackpropInput:1, DMT/_476, DMT/_477)]]
Caused by op 'gradients/flow_motion/flow_cnn_4/concat_grad/Slice_2', defined at:
File "OpenDVC_train_PSNR.py", line 102, in
train_MV = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(train_loss_MV, global_step=step)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py", line 400, in minimize
grad_loss=grad_loss)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py", line 519, in compute_gradients
colocate_gradients_with_ops=colocate_gradients_with_ops)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 630, in gradients
gate_gradients, aggregation_method, stop_gradients)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 814, in _GradientsHelper
lambda: grad_fn(op, *out_grads))
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 408, in _MaybeCompile
return grad_fn()  # Exit early
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 814, in
lambda: grad_fn(op, *out_grads))
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/array_grad.py", line 222, in _ConcatGradV2
op, grad, start_value_index=0, end_value_index=-1, dim_index=-1)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/array_grad.py", line 150, in _ConcatGradHelper
out_grads.append(array_ops.slice(grad, begin, size))
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 578, in slice
return gen_array_ops.slice(input, begin, size, name=name)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 7466, in _slice
"Slice", input=input, begin=begin, size=size, name=name)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
op_def=op_def)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in init
self._traceback = tf_stack.extract_stack()
...which was originally created as op 'flow_motion/flow_cnn_4/concat', defined at:
File "OpenDVC_train_PSNR.py", line 45, in
flow_tensor, _, _, _, _, _ = motion.optical_flow(Y0_com, Y1_raw, batch_size, Height, Width)
File "/home/csr_guest/aniranja/RLVC/OpenDVC/motion.py", line 54, in optical_flow
loss_4, flow_4 = loss(flow_3, im1_4, im2_4, 4)
File "/home/csr_guest/aniranja/RLVC/OpenDVC/motion.py", line 27, in loss
res = convnet(im1_warped, im2, flow, layer)
File "/home/csr_guest/aniranja/RLVC/OpenDVC/motion.py", line 7, in convnet
input = tf.concat([im1_warp, im2, flow], axis=-1)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1124, in concat
return gen_array_ops.concat_v2(values=values, axis=axis, name=name)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1033, in concat_v2
"ConcatV2", values=values, axis=axis, name=name)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
op_def=op_def)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in init
self._traceback = tf_stack.extract_stack()
AbortedError (see above for traceback): Operation received an exception:Status: 5, message: could not create a view primitive descriptor, in file tensorflow/core/kernels/mkl_slice_op.cc:300
[[node gradients/flow_motion/flow_cnn_4/concat_grad/Slice_2 (defined at OpenDVC_train_PSNR.py:102)  = _MklSlice[Index=DT_INT32, T=DT_FLOAT, _kernel="MklOp", _device="/job:localhost/replica:0/task:0/device:CPU:0"](gradients/flow_motion/flow_cnn_4/conv2d/Conv2D_grad/Conv2DBackpropInput, ConstantFolding/gradients/flow_motion/flow_cnn_1/concat_grad/ConcatOffset-folded-2, gradients/flow_motion/flow_cnn_4/concat_grad/Shape_2, gradients/flow_motion/flow_cnn_4/conv2d/Conv2D_grad/Conv2DBackpropInput:1, DMT/_476, DMT/_477)]]

Same after some cleanup and removal of deprecation errors,

2021-02-15 23:29:07.956119: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at mkl_slice_op.cc:303 : Aborted: Operation received an exception:Status: 5, message: could not create a view primitive descriptor, in file tensorflow/core/kernels/mkl_slice_op.cc:300
Traceback (most recent call last):
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
return fn(*args)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.AbortedError: Operation received an exception:Status: 5, message: could not create a view primitive descriptor, in file tensorflow/core/kernels/mkl_slice_op.cc:300
[[{{node gradients/flow_motion/flow_cnn_4/concat_grad/Slice_2}} = _MklSlice[Index=DT_INT32, T=DT_FLOAT, _kernel="MklOp", _device="/job:localhost/replica:0/task:0/device:CPU:0"](gradients/flow_motion/flow_cnn_4/conv2d/Conv2D_grad/Conv2DBackpropInput, ConstantFolding/gradients/flow_motion/flow_cnn_1/concat_grad/ConcatOffset-folded-2, gradients/flow_motion/flow_cnn_4/concat_grad/Shape_2, gradients/flow_motion/flow_cnn_4/conv2d/Conv2D_grad/Conv2DBackpropInput:1, DMT/_476, DMT/_477)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "OpenDVC_train_PSNR.py", line 166, in
learning_rate: lr})
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
run_metadata_ptr)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
run_metadata)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.AbortedError: Operation received an exception:Status: 5, message: could not create a view primitive descriptor, in file tensorflow/core/kernels/mkl_slice_op.cc:300
[[node gradients/flow_motion/flow_cnn_4/concat_grad/Slice_2 (defined at OpenDVC_train_PSNR.py:102)  = _MklSlice[Index=DT_INT32, T=DT_FLOAT, _kernel="MklOp", _device="/job:localhost/replica:0/task:0/device:CPU:0"](gradients/flow_motion/flow_cnn_4/conv2d/Conv2D_grad/Conv2DBackpropInput, ConstantFolding/gradients/flow_motion/flow_cnn_1/concat_grad/ConcatOffset-folded-2, gradients/flow_motion/flow_cnn_4/concat_grad/Shape_2, gradients/flow_motion/flow_cnn_4/conv2d/Conv2D_grad/Conv2DBackpropInput:1, DMT/_476, DMT/_477)]]
Caused by op 'gradients/flow_motion/flow_cnn_4/concat_grad/Slice_2', defined at:
File "OpenDVC_train_PSNR.py", line 102, in
train_MV = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(train_loss_MV, global_step=step)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py", line 400, in minimize
grad_loss=grad_loss)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py", line 519, in compute_gradients
colocate_gradients_with_ops=colocate_gradients_with_ops)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 630, in gradients
gate_gradients, aggregation_method, stop_gradients)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 814, in _GradientsHelper
lambda: grad_fn(op, *out_grads))
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 408, in _MaybeCompile
return grad_fn()  # Exit early
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 814, in
lambda: grad_fn(op, *out_grads))
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/array_grad.py", line 222, in _ConcatGradV2
op, grad, start_value_index=0, end_value_index=-1, dim_index=-1)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/array_grad.py", line 150, in _ConcatGradHelper
out_grads.append(array_ops.slice(grad, begin, size))
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 578, in slice
return gen_array_ops.slice(input, begin, size, name=name)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 7466, in _slice
"Slice", input=input, begin=begin, size=size, name=name)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
op_def=op_def)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in init
self._traceback = tf_stack.extract_stack()
...which was originally created as op 'flow_motion/flow_cnn_4/concat', defined at:
File "OpenDVC_train_PSNR.py", line 45, in
flow_tensor, _, _, _, _, _ = motion.optical_flow(Y0_com, Y1_raw, batch_size, Height, Width)
File "/home/csr_guest/aniranja/RLVC/OpenDVC/motion.py", line 54, in optical_flow
loss_4, flow_4 = loss(flow_3, im1_4, im2_4, 4)
File "/home/csr_guest/aniranja/RLVC/OpenDVC/motion.py", line 27, in loss
res = convnet(im1_warped, im2, flow, layer)
File "/home/csr_guest/aniranja/RLVC/OpenDVC/motion.py", line 7, in convnet
input = tf.concat([im1_warp, im2, flow], axis=-1)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1124, in concat
return gen_array_ops.concat_v2(values=values, axis=axis, name=name)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1033, in concat_v2
"ConcatV2", values=values, axis=axis, name=name)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
op_def=op_def)
File "/home/csr_guest/anaconda3/envs/RLVC/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in init
self._traceback = tf_stack.extract_stack()
AbortedError (see above for traceback): Operation received an exception:Status: 5, message: could not create a view primitive descriptor, in file tensorflow/core/kernels/mkl_slice_op.cc:300
[[node gradients/flow_motion/flow_cnn_4/concat_grad/Slice_2 (defined at OpenDVC_train_PSNR.py:102)  = _MklSlice[Index=DT_INT32, T=DT_FLOAT, _kernel="MklOp", _device="/job:localhost/replica:0/task:0/device:CPU:0"](gradients/flow_motion/flow_cnn_4/conv2d/Conv2D_grad/Conv2DBackpropInput, ConstantFolding/gradients/flow_motion/flow_cnn_1/concat_grad/ConcatOffset-folded-2, gradients/flow_motion/flow_cnn_4/concat_grad/Shape_2, gradients/flow_motion/flow_cnn_4/conv2d/Conv2D_grad/Conv2DBackpropInput:1, DMT/_476, DMT/_477)]]

I am sorry that I currently have no idea what the problem is, as our hardware is different and I have never met that issue. Could you try to train a simpler network to see if the problem still exists, e.g., just keep the motion_flow function and optimize the MSE between warped and raw images?
just keep
flow_tensor, _, _, _, _, _ = motion.optical_flow(Y0_com, Y1_raw, batch_size, Height, Width)
Y1_warp_0 = tf.contrib.image.dense_image_warp(Y0_com, flow_tensor)
only try to minimize
tf.reduce_mean(tf.squared_difference(Y1_warp_0, Y1_raw))
and see if workable?

Resolved this issue with Tensorflow 1.14 and tensorflow-compression 1.2
I installed tfc using pip rather than using a folder in the same path. This seems to have fixed the error.
Thank you for helping :)
Are there any plans to release RLVC soon/ would you have any guidelines for replicating the training loop, now that the work is accepted?

Happy to hear that the problem is solved. The RLVC has been released at https://github.com/RenYang-home/RLVC, the training could follow the description in our paper, we may not release the training codes. Thanks for understanding.

Okay - thanks for the update. One last question - I noticed you had mentioned we could get the training code in HLVC using substitution in OpenDVC in the discussion thread here.. Would a similar substitution/modification be possible for RLVC? - just wanted your thoughts

Since the RLVC contains a recurrent framework, its training is more complicated than OpenDVC/HLVC. It needs to be first warmed by training on one time step, and then fine-tuned to a longer time length, so its training procedure cannot be directly borrowed from OpenDVC, but in the end, the general ideas for training them are almost the same, i.e., a progressive training, flow -> flow + motion compression -> then + MC -> then + residual compression -> then fine-tune to a longer time length.

Right - that makes sense. Thanks once again for all the responses!


Fix gatekeeper assessment

💻 Environment
Platform: macOS
📒 Description
Gatekeeper assessment fails. We need to look into ways to solve this.
References
Is connected to notarizing or event fixed when we add notarizing  #2919
Possible solutions
https://forums.developer.apple.com/thread/77430
https://stackoverflow.com/questions/42066545/successfully-codesigned-app-failed-to-pass-gatekeeper#comment95373004_42066545

This is preventing new users from installing the app without going in to their Gatekeeper settings and allowing the app to run.

This issue is now fixed and we created and we build with the correct signed app (the version is same 7.4.484). Get it from https://toggl.github.io/toggldesktop/download/macos-stable/


feat: add state field on DataPlaneInstance response

What this PR changes/adds
add state field on DataPlaneInstance response
Why it does that
Briefly state why the change was necessary.
Further notes
List other areas of code that have changed but are not necessarily linked to the main feature. This could be method
signature changes, package declarations, bugs that were encountered and were fixed inline, etc.
Linked Issue(s)
Closes #4335
Please be sure to take a look at the contributing guidelines and our etiquette for pull requests.

@ndr-brt added

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 75.31%. Comparing base (7f20ba5) to head (cc60e9b).
Report is 349 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4350      +/-   ##
==========================================
+ Coverage   71.74%   75.31%   +3.56%
==========================================
  Files         919     1053     +134
  Lines       18457    21173    +2716
  Branches     1037     1182     +145
==========================================
+ Hits        13242    15946    +2704
+ Misses       4756     4709      -47
- Partials      459      518      +59

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Night Config

Minecraft version
1.19.4
MoreVanillaLib version
1.16.4-1.4.1
Main extension mod
Vanilla Hammers [Forge]
Forge version
36.2.34
The latest.log file
https://pastebin.com/iEQBhD58
Issue description
When starting minecraft MoreVanillaLib gives this error: com.electronwill.nightconfig.parsingexception .

Steps to reproduce

Start Minecraft
U get this error

Other information
I tried some things, like i used more ram, uninstalled java, used a mod for Night config, but nothing seemed to work.

You’re using a version for 1.16.4 as the version name of the mod says, try with a version for 1.19.4


Add waterfall support

I map a huge park now. We have many beautiful waterfalls.
There is no waterfalls in editor right now.
Of course they are in OSM specification.
https://wiki.openstreetmap.org/wiki/Waterfalls
To comply to specification, I manually convert chunks of streams by overwriting tag waterway=waterfall on them.

I was wondered how huge waterfalls as Niagara was done. And there they done the same as I do. There is a dot waterway=waterfall 8)
https://www.openstreetmap.org/edit?relation=176131#map=17/43.07915/-79.07353
Thank you.

Good suggestion, thanks!


PyPi package doesn't install header files in the header file search path.

Use case: I maintain a python package which includes an extension module, and which uses the py3c headers, which, up to now, I have bundled with the project. Now that version 1.0 of py3c is available on PyPi, I had hoped that it would be possible to stop bundling the py3c headers and add py3c to the install_requires of my packages setup.py.
Unfortunately, however, pip install py3c puts the headers in a place which is not on the header file search path:
$ ls venv/include/site/python3.6/py3c/
capsulethunk.h  compat.h    py3c.h      tpflags.h
comparison.h    fileshim.h  py3shims.h

That's from doing a pip install py3c with the virtualenv activated. You can see that this directory is not on the standard compiler search path for header files:
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m/py3c -I/home/jgu/Projects/python-lz4/venv/include -I/usr/include/python3.6m -c lz4/_version.c -o build/temp.linux-x86_64-3.6/lz4/_version.o
lz4/_version.c:38:10: fatal error: py3c/py3c.h: No such file or directory
 #include <py3c/py3c.h>

Do you know how this should be solved?
py3c uses the setuptools' headers argument.

Do you know how this should be solved?

I don't. I spent a couple of hours yesterday trying to get to the bottom of it. Here's what I found out:

The headers argument is inherited directly from distutils, and so setuptools just passes it on.
Setuptools mangles data_files such that it's not possible to specify a directory for them to be installed in e.g. the files might end up in a wheel, and therefore unavailable to compiler.

The only solution I could see was:

Don't use setuptools
Abuse distutils data_files to somehow put the headers in the right place. But that's tricky, when you think that the package might be installed into a virtual env. So, you need to somehow query for the correct path at package install time.

It seems like this use case just isn't accommodated with the tools we have today, alas.

@xoviat, if you have a bit of time, could you sketch how you're using the PyPI package? It seems we're missing something here.

from sysconfig import get_paths
from wheel.paths import get_install_paths

sysconfig = distutils.sysconfig.get_config_vars()

def get_wheel_header_paths():
    return [os.path.dirname(get_install_paths('dummy')['headers'])]

py3c_include_dirs = get_wheel_header_paths() + \
    [get_paths()['include']] + \
    [get_paths()['platinclude']]

I think the main problem here is that make install (the preferred way how to install py3c by py3c authors) does something different than python setup.py install (this is something that pip install very likely does).
The make install puts py3c.h in the well known include path and all other headers into the py3c/ subdir.  Something like this:
$ find /usr/include/py3c* -type f | sort
/usr/include/py3c.h
/usr/include/py3c/capsulethunk.h
/usr/include/py3c/comparison.h
/usr/include/py3c/compat.h
/usr/include/py3c/fileshim.h
/usr/include/py3c/py3shims.h
/usr/include/py3c/tpflags.h
$

It also installs the py3c.pc file which suggests that to find py3c you should use -I/usr/include (which is usual default, so py3c.pc is usually just noop).
OTOH, python setup.py install puts all headers into the py3c/ subdir:
$ find /usr/include/py3c* -type f | sort
/usr/include/py3c/capsulethunk.h
/usr/include/py3c/comparison.h
/usr/include/py3c/compat.h
/usr/include/py3c/fileshim.h
/usr/include/py3c/py3c.h
/usr/include/py3c/py3shims.h
/usr/include/py3c/tpflags.h
$

The /usr/include/py3c/py3c.h file is not in the default include path and since python setup.py install does not produce the pkgconfig file (py3c.pc) there is no way how any external software could find py3c automatically.  In this case the py3c.pc should suggest both -I/usr/include and -I/usr/include/py3c.
So I think the easiest solution, at least for now, is to teach python setup.py install how to install the pkgconfig file too.  I'm not sure how exactly one could do that, but I know that pycairo provides install_pkgconfig setup.py subcommand which is (IIRC) called automatically by python setup.py install to install the .pc file.

python setup.py install (which should nowadays be python -m pip install .) installs (preferably) into a virtual environment, and that doesn't really have a concept of header file locations or pkgconfig.
make install installs system-wide, assuming a UNIX-style system (probably with some Linux-isms, unintentionally). Python packaging is not really meant for this kind of system integration.
IMO, the best solution is to ignore PyPI and treat py3c as a C library.

I'm sorry, but pip install does not by default install into a virtual environment:
$ cd /tmp
$ wget https://github.com/encukou/py3c/archive/refs/tags/v1.4.tar.gz
$ tar xf v1.4.tar.gz
$ cd py3c-1.4
$ python -m pip install .
Defaulting to user installation because normal site-packages is not writeable
Processing /tmp/py3c-1.4
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: py3c
  Building wheel for py3c (setup.py) ... done
  Created wheel for py3c: filename=py3c-1.4-py2.py3-none-any.whl size=9399 sha256=86726f711842852d2a92388248550c20f183e0c8066299ec42ec621bad1f9bc1
  Stored in directory: /home/marcel/.cache/pip/wheels/87/88/2a/5ecb2637b4c8361d8d726048b40f1355dccf2690db6c1ac543
Successfully built py3c
Installing collected packages: py3c
Successfully installed py3c-1.4
$ find ~/.local/
/home/marcel/.local/
/home/marcel/.local/include
/home/marcel/.local/include/python3.9
/home/marcel/.local/include/python3.9/py3c
/home/marcel/.local/include/python3.9/py3c/tpflags.h
/home/marcel/.local/include/python3.9/py3c/comparison.h
/home/marcel/.local/include/python3.9/py3c/py3c.h
/home/marcel/.local/include/python3.9/py3c/capsulethunk.h
/home/marcel/.local/include/python3.9/py3c/compat.h
/home/marcel/.local/include/python3.9/py3c/py3shims.h
/home/marcel/.local/include/python3.9/py3c/fileshim.h
/home/marcel/.local/lib
/home/marcel/.local/lib/python3.9
/home/marcel/.local/lib/python3.9/site-packages
/home/marcel/.local/lib/python3.9/site-packages/py3c-1.4.dist-info
/home/marcel/.local/lib/python3.9/site-packages/py3c-1.4.dist-info/top_level.txt
/home/marcel/.local/lib/python3.9/site-packages/py3c-1.4.dist-info/REQUESTED
/home/marcel/.local/lib/python3.9/site-packages/py3c-1.4.dist-info/direct_url.json
/home/marcel/.local/lib/python3.9/site-packages/py3c-1.4.dist-info/RECORD
/home/marcel/.local/lib/python3.9/site-packages/py3c-1.4.dist-info/LICENSE.MIT
/home/marcel/.local/lib/python3.9/site-packages/py3c-1.4.dist-info/WHEEL
/home/marcel/.local/lib/python3.9/site-packages/py3c-1.4.dist-info/INSTALLER
/home/marcel/.local/lib/python3.9/site-packages/py3c-1.4.dist-info/METADATA
$ pip list --user
Package Version
------- -------
py3c    1.4
$ python -V
Python 3.9.16
$ pip --version
pip 23.0 from /usr/lib/python3.9/vendor-packages/pip (python 3.9)
$

pip install does not by default install into a virtual environment:

Right, although some see it as a historical mistake.
I meant that pip is designed to install to virtualenvs (among other things), so it can't assume a header path or pkgconfig location is available. The packaging formats made for pip/PyPI don't handle *nix system integration well.


hotfix/copyright -> dev

This change is

Reviewed 1 of 1 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

a discussion (no related file):
Wait on approval from Joann (and copyright office)

Comments from Reviewable


Restrict bevy dependency to something smaller, ie bevy_core

At a glance aery only uses the bevy_ecs and bevy_derive crates, would you be open to restricting the bevy dependency to these? It could reduce compile times and make aery more feasable for non-graphics applications.

Can do for 0.5

I'll do it for the next release. I'm having hardware issues rn so just getting 0.5 out with 0.12 support is more urgent.

Added in #20


Start Writing backend

Is your feature request related to a problem? Please describe.
Currently when you write a blog it is not storing in database
Describe the solution you'd like
I want to create the backend of the writing page where user can store there blog and which can be later used to show the blog
Additional context

@ANSHIKA-26 please assign me this issue

@ANSHIKA-26 is anyone working on the backend of blog writing page?


Incorect time zone for Sofia

moment.tz(moment.tz.guess()).utcOffset() is returning 180 for Europe/Sofia -> we are always +2. There are some countries in our area like Belarus that are +3 during the summer and + 2 winter

Hello fellow Sofia dweller!
https://www.timeanddate.com/worldclock/bulgaria/sofia
Sofia can be +2 or +3, currently its +3.


Unable to run my project on circle ci

I added gem 'rake' to the Gemfile and rake in the Dependencies in the Gemfile.lock as was suggested by the error in the previous run. But now I am getting error. It now says that I have a older version of Ruby bundler , i.e. 1.15.4 but in the Gemfile.lock, it's written in the end, that, Bundled with 1.16.2
Following warning is coming and build is failing.
Warning: the running version of Bundler (1.15.4) is older than the version that created the lockfile (1.16.2). We suggest you upgrade to the latest version of Bundler by running gem install bundler.
Tried searching on the below link as well, but no solution.
https://github.com/bundler/bundler/search?q=Unable+to+find+a+spec+satisfying+rake+(>%3D+0)+in+the+set.+Perhaps+the+lockfile+is+corrupted%3F&type=Issues

You need to run your commands in bundler environment. It is not clear for me how do you execute commands, but you need to prepend them with bundle exec or generate binstubs and run those.
https://bundler.io/v2.0/bundle_exec.html

The message you have quoted is a warning and break your build. Can you please attach the error that is causing your build to fail.


fix: typo in comment that could lead to confusion

The TodoResolver is not a root-level resolver

Thank you!