Parallel process image data

This PR adds ImageAlgo methods for parallel processing of tiles/channels, and re-implements both ImagePlug::image() and ImageWriter to use this.
ImageAlgo has 4 new methods:

parallelProcessTiles

Two versions

One takes an imagePlug, a functor and an optional window to process (default will be imagePlug->dataWindowPlug()
One takes the same parameters as above, plus a vector of channel names, and will call the functor once per tile per channel

This will call the defined functor once per tile (per channel)

parallelGatherTiles

Same two versions as above
Also takes:

Gather functor, which will be called for the same tiles/channels, but in series.
tileOrder, which defines the order that the gather functor is called in. One of:

Unordered : No guarantee of ordering. Will tend towards bottom to top.
BottomToTop : Guaranteed iteration through channels, tiles in X, tiles in Y, from the bottom
TopToBottom : Same as BottomToTop, but from the top in Y

ImagePlug::image(), which previously dealt with its own parallelising, now uses parallelProcessTiles() to set off parallel processing of channel data to create an ImagePrimitive.
The parallelising that it used to do had a bug where it would often end up processing the same tiles multiple times, due to a misunderstanding of how tbb's blocked_range dealt with splitting up ranges. It would sometimes pass through ranges that were smaller than grainsize. As such, it now does all of the splitting using IDs, and then these are turned back into coordinates internally.
ImageWriter used to call ImagePlug::image() to get a full image before writing it. It now has its own classes for storing the data and writing it to an OIIO::ImageOutput object.
There is also a new test - ImageNodeTest.testCacheThreadSafety(), which uses GafferImageTest.traverseTiles() to call imagePlug->channelData() on each tile and channel.

Hmmm - Travis is properly crashing out on ImageWriterTest.testWriteEmptyImage
That's not a crash I've seen myself locally (OSX, debug build). Going to try a debug build on Linux and a release build on OSX and see if I can get it to crash.
Move along people. Nothing to see here.

Comments so far addressed in new commit.

Also, I've been able to replicate the crash on a Linux machine. I'm going to get my Ubuntu VM back up and running on my OSX machine, and I'll work on it there.

OK, I've done a partial review of the ImageWriter mostly just on the basic organisation - I'll take a closer look at the internal logic when you have the crash fixed.
Overally I'm feeling good about this though - it's nice to see the details of the threading getting abstracted away like this, and the performance improvements you've made are very welcome too :)

Great - thanks.
I'm almost done making your suggested changes, then I'll have the ImagePlug::imageHash() re-implementation to do, and the crash to fix. I'll let you know when that's all in there.
Cheers

I fixed the crash. After finding out what it was, I'm somewhat shocked that it wasn't crashing on OSX!
I'm just working on ImagePlug::imageHash() at the moment. Would I be right in saying that it does make a difference in which order hashes are appended?

Good news! And yes, order is important. Don't worry about matching the current imageHash() results exactly if that would be fiddly, but do ensure that the order of appending is consistent in the new version, and have a unit test assert that repeated calls give the same result...

Cool.
I've just pushed a couple of commits that adds the imageHash() parallel processing, so all should be good, as long as Mr Travis agrees.

Just done a small tweak to the test to get it to run imageHash() 20 times in a row and check if they're all the same, just in case parallelisation was causing it to run the same order some times. It should be good now.

That looks like quite a dramatic crash there! Will have to dig into that one a bit... It might be a few days until I can get into this.

I think I've fixed the crash.
Note to self - don't trust OSX to protect from crashes. Always run tests on Linux before submitting a PR...

Am I right in thinking that the ImageWriter is trying to deal with situations whereby the output tile size is not exactly the same as Gaffer's native tile size?

Yes. It was written in a way that was general enough that it could deal with these situations. I don't believe it adds any significant overhead vs assuming the same tile size as Gaffer's internal tile size.
However, output tiles, even if the same size, will usually not line up to Gaffer tiles, as output tiles align to the data window origin, and are based on the top left rather than the bottom left.

Is this expected to be fully functional currently? I'm getting a seg fault with a fairly awkward use case (a custom node derived from ImagePrimitiveSource) which should bottleneck the threading, but is instead crashing immediately. I haven't looked into it further since I didn't know where we're at on thinking there are outstanding crashes here or not.

I was thinking it was fully-functional, so would be good to know how best to replicate this crash...

I'll try to get you a minimal repro. Its not actually a use case we want in the long run (ImagePrimitiveSource should die), but until we have a proper Warp base node in Gaffer, I'm stuck diving out into IECore::ModifyOps unfortunately.

Cool - let me know when you have something and I'll figure out what's happening.
Cheers

@HughMacdonald, my question about arbitrary tile sizes was motivated mainly by the complexity of trying to review and maintain the code, rather than any runtime overhead. I have to admit to glazing over when trying to follow all the logic in ImageWriter, and I had hoped that things could be simplified somewhat. But as you say the tiles won't line up anyway even if of the same size, so I guess some of the complexity is needed.
@andrewkaufman, it might be good if you could git bisect to find the commit that's causing the crash. I had been considering merging the first half of this PR on it's own and then continuing to improve the ImageWriter part - if the problem is in the ImageWriter then I'll be even more motivated to do so.
Just to confuse things further, I did some tidying up as I reviewed this branch, which can be found here :
https://github.com/johnhaddon/gaffer/commits/HughMacdonald-parallelProcessImageData
@HughMacdonald, I think it's worth paying particular attention to https://github.com/johnhaddon/gaffer/commit/51ee62c46fd4a7930a457fc4014481f5af227774. Assuming I've understood right, there really wasn't any need for that intermediate storage and indexing logic I removed - this is the kind of simplification we're striving to enable when introducing abstractions like parallelGatherTiles(). I feel like there might be similar abstractions we could use to improve the readability of things like ImageWriter. Maybe some abstractions in ImageAlgo like "copy the data from this tile into that tile where they intersect"? No need to jump straight into that, but I think it might be worth bearing in mind for the future...

@HughMacdonald, my question about arbitrary tile sizes was motivated mainly by the complexity of trying to review and maintain the code, rather than any runtime overhead. I have to admit to glazing over when trying to follow all the logic in ImageWriter, and I had hoped that things could be simplified somewhat. But as you say the tiles won't line up anyway even if of the same size, so I guess some of the complexity is needed.

The only change to assuming that tiles are the same size would be that you would know that you'd never need more than 4 input tiles for an output tile, but I don't think that the code complexity would decrease at all for this.
I can talk you through the code tomorrow - it was a little tricky to figure out, and I get that it's a bit confusing, and I'd love to have a chat about how it could be simplified.

@HughMacdonald, I think it's worth paying particular attention to johnhaddon@51ee62c.

I definitely prefer what you've done there. I'm not sure why I didn't go with parallelGatherTiles in the first place, as that's exactly what that was designed for. Mindfart from me on that one!

I've been looking at what a generic "copyDataWindow" method in ImageAlgo would be.
It's tricky because it would also need to deal with interleaved channel data (which ImageWriter requires), and would we want it to be able to deal with input interleaved channel data too? What should it do if both the input and output data is interleaved?
This prototype would be able to do what ImageWriter requires, while also supporting the same thing on both input and output data. If input and output data have interleaved channels, it would still only copy 1 channel's worth of data.
void copyDataWindow(
	ConstFloatVectorDataPtr inData,
	FloatVectorDataPtr outData,
	const Imath::Box2i &inWindow,
	const Imath::Box2i &outWindow,
	const bool inDataYDown = false,
	const bool outDataYDown = false,
	const size_t inDataPixelOffset = 0,
	const size_t outDataPixelOffset = 0,
	const size_t inDataPixelStride = 1,
	const size_t outDataPixelStride = 1 );

@HughMacdonald and I went over the ImageWriter part of this in person yesterday. Hugh spotted a couple of bugs and explained the general workings, and we discussed ways of improving the clarity of the code and reducing the likelihood of other bugs. The plan is to make those revisions and then have @andrewkaufman do a review with fresh eyes. Andrew, I don't suppose you've had any luck reproducing your crash or using git bisect?

I haven't yet, no. I need to get my ImageReader PR ready for re-review, then I'll take a look at getting a repro here.

25b6d2b90ead99f1e40c3772900d5602d75fa5aa is the first bad commit
commit 25b6d2b90ead99f1e40c3772900d5602d75fa5aa
Author: Hugh Macdonald<EMAIL_ADDRESS>Date:   Fri Oct 23 16:44:49 2015 +0200

    ImagePlug : Modify imageHash() to use parallelProcessTiles

Now that I actually read the commit names, that was kind of obvious since my node calls imageHash() an un-godly number of times...

Ok, I have a repro. cherry-pick this one https://github.com/andrewkaufman/gaffer/commit/c81325dfe50e4c026f789ae39a86ca18465e26e2 and run gaffer test GafferImageTest.CortexPremultTest, which should seg fault.
If you then rebase to drop https://github.com/HughMacdonald/gaffer/commit/1c218c9dbd36523a5232740f48343272b9f650b1, the test passes.

Might be worth dropping in my simplified parallel imageHash() to see if that improves things. @HughMacdonald, I'm going to leave this to you for now so I can get on with other priorities, but I should be able to spend some time on it later next week if you don't get anywhere.

I'll merge John's changes into my branch and test from there. Thanks guys

I'm having trouble recreating this crash... Even compiling directly from https://github.com/andrewkaufman/gaffer/commit/c81325dfe50e4c026f789ae39a86ca18465e26e2 with all preceding commits, the test passes.
I've got two branches on mine - https://github.com/HughMacdonald/gaffer/tree/parallelProcessImageData (as before), which now contains @johnhaddon's commits (including the WIP one!), and https://github.com/HughMacdonald/gaffer/tree/parallelProcessImageData_crash, which should fail (but doesn't), which has @andrewkaufman's commit on the end.

It's not crashing on OSX or Linux for me...
John significantly simplified the parallel imageHash, so please do try your end based on the latest commit in my parallelProcessImageData and see if it still happens...

Its weird that you can't reproduce it, when it definitely crashed for me every time... but the new code seems to work fine for me now, so lets just ignore it and move on...

Weird. Glad to hear it though. I'll work from the current head then... @johnhaddon, this does include your WIP commit...

Hi all,
I've done a major update to this PR, so it should be good to review again. I've addressed @johnhaddon's notes, and added better comments, so it would be good, @andrewkaufman, if you could see if the methodology for tiles and scanline writing makes sense (the big comment blocks at the top of FlatTileWriter and FlatScanlineWriter)
Cheers

Oh, and I also realised there was a place where we may have ended up writing image data that was in tiles, but outside the data window. I wrote a test that failed, and then fixed the issue.

I went through it, and I think I understand the whole Scanline vs Tile Writer logic. I made a couple fairly innocuous comments... overall I understand the reasoning for the approach, though it did take multiple read throughs of the big comment explanation.
I do find the code a bit difficult to navigate. Unfortunately I don't have any suggestions for improvement there as the topic seems complex enough that I'm not sure a simpler implementation is possible... Some of the maths I'm taking your word on, so it's entirely possible an off-by-one sort of error has gone unnoticed somewhere, but hopefully the test cases show that it hasn't.
What do you reckon @johnhaddon? What might be nice is some profile data, just to show what sort of speed vs memory tradeoff can be expected from this?

I've addressed @andrewkaufman's note about writeFilledTiles

I've just pushed some more changes, which splits parts of ImageAlgo out into BufferAlgo. Any methods that work on Box2is are now in BufferAlgo. ImageAlgo contains the colour channel methods, and the parallel processing methods.
I'm thinking that copyBufferArea (currently here: https://github.com/HughMacdonald/gaffer/commit/6fc468a8fe87ad9e499165cd6b52c9b161873e78#diff-7d9752bff586c1eda65c502bb6202d2cR76) would make sense to move into BufferAlgo too.

Hi Hugh,
The build seems to be failing - I've got a commit that appears to fix it at my end :
https://github.com/johnhaddon/gaffer/commit/c6e24edb80a97d16e5163cd3ba0975e7efd59221
With that compilation is OK, but I get 4 failing ImageWriter tests. I don't know if that is because my fix is wrong (I don't think it is though), or if there is some other problem. Could you cherry pick that commit onto your branch and see what you get?
Cheers...
John

Thanks for that - I've been a little snowed under here so hadn't had a chance to get onto this. Hopefully will be back on it later this week and will let you know.

No worries - I've been slow to get to it in any case. I'm starting to look at an improved ImageView using the parallelGatherTiles stuff, so I may just cherry pick the earlier commits to get that moving.

You know what - give me an hour or so this evening and I'll see if it's a quick fix. I've just put a render on, so should have a bit of time.

Fixed it.
I was trying to be clever and not bother blanking out the scanlines before adding data. I added a fix because it had the possibility of writing data outside the data window, and this ended up meaning that old scanline data from a previous row of tiles could sneak through, which was what was happening here.
I've added back in code to blank out m_scanlinesData before each row of tiles.

I found another compilation error. Fixed that one too. Let's see what Travis says!

Hmm - so something very wrong there! I'll take a look first thing in the morning.

It looks like I've fixed the crash...

I've fixed the issue - it now passes your test, plus another few I added in.

I've put in a new PR which should make all that BufferAlgo::empty() hoop jumping unnecessary, so we can go back to the original implementation. I'd like to keep the additional tests you added, but once #1586 is merged, do you think you could rebase without c75b8f2 and 18ec477 Hugh?

Our messages on the two PRs passed like ships in the night... Will do absolutely that. When do you think #1586 will be merged? It going to wait for @andrewkaufman? The reason I ask is because I'm off for a few days from tomorrow lunchtime...

I'd like @andrewkaufman to give #1586 a once over before merging, but I'm hoping he'll have time for that today (he recently got back from holidays, so I expect he's a bit slammed). I also just realised that ab9ea78 can be dropped as well.

Cool stuff. I've dropped those commits, and will rebase on the new master when it exists.

Thought I'd do a bit of performance testing with the new ImageWriter while we're waiting on the other bits and bobs. With a contrived test upsizing from HD to 10K and writing the result I got a reduction in runtime of 65% and a reduction in peak memory usage of 60%. Nice one Hugh!

Woop! Nice to know it was worth it!

I've rebased on top of the new build changes, but I'm still seeing a failure on one of them so far (the other 3 are still running)
gcc 4.8 with C++98 is failing with the same error as before:
include/GafferImage/BufferAlgo.inl: In member function 'void* tbb::interface6::internal::concrete_filter<T, void, Body>::operator()(void*) [with T = boost::tuples::tuple<long unsigned int, Imath_2_2::Vec2<int>, boost::intrusive_ptr<const IECore::TypedData<std::vector<float> > >, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type>; Body = GafferImage::Detail::GatherFunctorFilter<{anonymous}::FlatScanlineWriter, {anonymous}::TileProcessor>]':
include/GafferImage/BufferAlgo.inl:51:3: error: assuming signed overflow does not occur when assuming that (X + c) < X is always false [-Werror=strict-overflow]
   if( window.max[i] <= window.min[i] )
   ^
include/GafferImage/BufferAlgo.inl:51:3: error: assuming signed overflow does not occur when assuming that (X + c) < X is always false [-Werror=strict-overflow]
cc1plus: all warnings being treated as errors
scons: *** [src/GafferImage/ImageWriter.os] Error 1
scons: building terminated because of errors.

Same failure (but double the time!) on gcc 4.8 with C++11

Plus, the default gcc, with C++98 flags the warning, but doesn't fail on it:
In file included from ./build/gaffer-<IP_ADDRESS>-linux/include/tbb/tbb.h:67:0,
                 from include/GafferImage/ImageAlgo.inl:40,
                 from include/GafferImage/ImageAlgo.h:130,
                 from src/GafferImage/ImageWriter.cpp:56:
include/GafferImage/BufferAlgo.inl: In member function 'void* tbb::interface6::internal::concrete_filter<T, void, Body>::operator()(void*) [with T = boost::tuples::tuple<long unsigned int, Imath_2_2::Vec2<int>, boost::intrusive_ptr<const IECore::TypedData<std::vector<float> > >, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type>, Body = GafferImage::Detail::GatherFunctorFilter<{anonymous}::FlatScanlineWriter, {anonymous}::TileProcessor>]':
include/GafferImage/BufferAlgo.inl:51:3: warning: assuming signed overflow does not occur when assuming that (X + c) < X is always false [-Wstrict-overflow]
include/GafferImage/BufferAlgo.inl:51:3: warning: assuming signed overflow does not occur when assuming that (X + c) < X is always false [-Wstrict-overflow]

Yeah, I'd downgraded the warning only for GCC < 4.8, as in my testing it hadn't failed in 4.8. I'll just downgrade the warning for all versions and kick your tests off again.

Cool. If you've got a commit you want me to cherry-pick into this PR (to keep the PR history), I can do that, or rebase on top of a new master. Let me know.

I'll fix things on master, which I think means you shouldn't need to do anything, because Travis should run the tests on a merge between master and your branch. Or so I think...

Ahh - cool - I did not know that! I guess we shall see...

Taking a quick look at the Travis docs, it says:

Rather than test the commits from the branches the pull request is sent from, we test the merge between the origin and the upstream branch.

So this one should hopefully all work

Indeed it has...thanks Hugh.

Yay! Thanks! Glad it's finally made it in!
Now to rebase deepImageCore (going to have a LOT of conflicts to resolve) and get back to that one!


[BUG] wash reg still documented in the help output (and docs site for v 1.0) but not supported anymore

Affected project(s)

[X] documentation
[ ] examples
[ ] wasmCloud host
[X] wasmCloud CLI (wash)
[ ] wasmCloud dashboard UI (washboard)
[ ] capability providers
[ ] provider bindgen
[ ] control interface client
[ ] other / not sure

Describe the bug
output for wash help contains
Publish:
  pull         Pull an artifact from an OCI compliant registry
  push         Push an artifact to an OCI compliant registry
  reg          Perform operations on an OCI registry

but reg command is not available and seems to be deprecated
Steps to reproduce

Run command wash help
See docs https://wasmcloud.com/docs/cli/reg

Expected behavior
Info about reg command must be removed
Environment

OS: ALL
Shell: ALL
wash version:  0.29.2

Screenshots / Logs / Additional context
No response

I can provide a quick fix for wash cli, but not sure about docs.

Hey @deem0n our docs are over at https://github.com/wasmCloud/wasmcloud.com -- feel free to fork that and contribute that if you don't mind!
If not we can get on it as well, shouldn't be too hard of an update to make :)

Should we recover wash reg ping in some way? I guess it is not so useful feature and we can forget it.

@deem0n I liked the idea of wash reg ping but, in practice, it didn't really get much use and it would be pretty small functionality to put as a top level command. What we have considered is a wash login to authenticate to an OCI registry, which I think would bring that functionality essentially back and be able to manage pulling/pushing to that registry from then on. That should be a separate feature, I think.

wash reg should be gone from everywhere now -- both PRs to wash-cli and docs have been merged -- let me know if I missed anything @deem0n and we'll re-open!

@vados-cosmonic: Thanks! Everything seems to be fine.


Implement the 'Add module' modal

Implement the 'Add module' modal per design.

This should provide a pretty solid base for all of the more complex stuff we'll be doing. Reassigning to @Coenego for review.

I was thinking of minimising the number of functions we call in initIndex in /apps/timetable/ui/js/index.js.
Now we have:
     var initIndex = function() {
         addBinding();
         renderHeader();
         renderPickers();
         renderLoginModal();
         setUpCalendar();
         getTripos();
     };

It might be better to have something like (where setUpHeader takes care of all the header/picker/tripos initialisation):
     var initIndex = function() {
         addBinding();
         renderLoginModal();
         setUpCalendar();
         setUpHeader();
     };

Followed up on review.

Coverage remained the same when pulling<PHONE_NUMBER>1d4c1493aef9474b28445af19ad45b on bp323:addModule into f3c8010ac04ac7838817b22ac6abe8aa7a592260 on CUL-DigitalServices:master.


Can't find policy method

I'm running a feature test using capybara, but I'm getting the following error:
undefined method `edit?

The method is defined in my policy file but for some reason it's not being found.
def job_openings
    authorize current_user, :edit?
    @company = current_user.company
    @jobs = @company.jobs.order 'created_at DESC'
end

Please re-read the README. You probably don't want to pass current_user to authorize from the looks of it.

OK...@jnicklas, I'm still having this problem but I'll try doing it without using current_user.

That suggestion fixed it...thx.


fix(CLI): accept glob pattern on file arg

Closes #2, #3

I installed two dependencies,
chokidar requested in issue #2
glob - to list the files found by glob pattern
I have added a test to validate the pattern shown in READ.ME


Create Drafts.js

Create drafts.js

Pre-merge test failed:

After changing a draft, the pokemon that was undrafted cannot be drafted by another user, additionally the pokemon that was drafted vis this method can be drafted again.

@xExecute everything else looks good


Initial Microsoft.Sql Swagger Spec

This is the initial Swagger spec for the Microsoft.Sql REST APIs.  Testing and additional validation is still in progress.

This change is

Hi @akromm, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

It looks like you're working at Microsoft (adamkr). If you're full-time, we DON'T require a contribution license agreement.

If you are a vendor, DO please sign the electronic contribution license agreement. It will take 2 minutes and there's no faxing! https://cla.azure.com.

TTYL, AZPRBOT;

            "operationId": "Databases_PauseDataWarehouse",

is Data Warehouse is same as Database? If yes then just use Database_Pause

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:24 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "operationId": "Databases_ResumeDataWarehouse",

Same comment as Pause

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:63 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "operationId": "RecommendedElasticPools_GetDatabases",

RecommendedElasticPools_GetDatabases -> RecommendedElasticPools_GetDatabase

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:997 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "description": "Returns information about an Azure SQL Database inside of an Azure Sql Recommended Elastic Pool.",

Some places it is SQL and at some it is Sql, better to be consistent

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1077 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                    "modelAsString": true

Why modelAsString:true?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1476 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                    "description": "The comma separated list of child objects to expand in the response."

Can you mention which list of properties to expand on?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:261 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                    "description": "The filter to apply on the operation."

It would be nice if you can specify which properties can the user filter on? Are there any mutually exclusive properties that the user cannot specify at the same time? An example or a link to a doc that explains this would be helpful. If you do not have a link to MSDN doc then you can create a doc in the rest-api-specs repo and provide a rawgithub or aka.ms link to it.

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:299 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                "ServerUpgrades"

@akromm - Most of the times I feel that the value that you have in the tags array makes a perfect candidate for the Noun in operationId. For example: "ServerUpgrades_Cancel" would just be the perfect value for OperationId.

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:495 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                "202": {

Is this a long running operation?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:514 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                "202": {

long running?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:583 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "operationId": "ElasticPools_List",

ElasticPools_ListByServer

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:757 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "description": "Returns information about activity on Azure SQL databases inside of an Azure SQL elastic pool.",

activities

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:833 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "operationId": "ElasticPools_GetDatabases",

ElasticPools_GetDatabas~~es~~

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:873 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "operationId": "Databases_SetTransparentDataEncryption",

Databases_CreateOrUpdateSetTransparentDataEncryption

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1316 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                "201": {

long running?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1355 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                "description": "Gets or sets the DTU."

what is dtu ? Some logical explanation

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1453 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "Name": {

All the other models have properties in camelCase and this model has properties in PascalCase. This looks weird. Please make sure this is correct otherwise it may cause issues in [de]serialization

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1587 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "TargetEdition": {

enum candidate?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1591 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "state": {

enum?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1610 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "dtu": {

the description of dtu databaseDtuMax databaseDtuMin storageMB is starting with the verb Gets. It does not have sets. So are these readOnly properties? If yes then please mark them correctly.

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1628 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "value": {

value -> required

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1668 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "value": {

value is required

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1901 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "version": {

enum candidate?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1918 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "Version": {

enum?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1978 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "Edition": {

enum?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2025 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "ScheduleUpgradeAfterUtcDateTime": {

camelCase?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1982 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "serverUpgradeProperties": {

"x-ms-client-flatten": true

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2009 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "Name": {

Is the name changeable/ updatable? If not then it should be marked as readonly.

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2021 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

        "properties": {

does this object contain any  required or readOnly properties ?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2020 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "action": {

enum? Documentation definitely says so.

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2065 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "state": {

enum?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2070 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "indexType": {

enum?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2087 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                    "modelAsString": true

why true?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2169 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

        "properties": {

PascalCase properties?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2020 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "collation": {

enum?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2192 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                    "modelAsString": true

why true?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2234 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "status": {

enum ?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2257 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "value": {

for all the listResult type of models "value" should always be a required property.

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2328 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                    "modelAsString": true

why true?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2526 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "unit": {

enum candidate?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2565 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                    "modelAsString": true

why true?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2655 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

    "ServerNameParameterGeneric": {

duplicate with ServerNameParameter. One of them should go away.

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:3004 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

        "properties": {

should there be sku as a top level property of the "Resource" ?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2922 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

@yoavrubin is added to the review. #Closed

            "operationId": "Databases_PauseDataWarehouse",

They are not the same.

In reply to: 249277837 [](ancestors = 249277837)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:24 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "operationId": "Databases_ResumeDataWarehouse",

They are not the same.

In reply to: 249278226 [](ancestors = 249278226)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:63 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                    "modelAsString": true

Enum is subject to change

In reply to: 249294810 [](ancestors = 249294810)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1476 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

    "operationId": "ElasticPools_GetDatabases",

GetDatabase rather than GetDatabases

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:873 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

                    "description": "The filter to apply on the operation."

Removing this since we don't use it in this version of Databases_List

In reply to: 249308162 [](ancestors = 249308162)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:299 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                "ServerUpgrades"

Updated - please have a look under this URI.
@akromm - please have a look

In reply to: 249309514 [](ancestors = 249309514)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:495 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

"ServerProperties": {

SqlServerProperties

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1911 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

"Server": {

SqlServer

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1933 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

"ServerListResult": {

SqlServerListResult

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1948 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

            "description": "Returns information about activity on Azure SQL databases inside of an Azure SQL elastic pool.",

For the description, I think "activity" is grammatically correct as well, and doesn't have the connotation of being a list of pastimes like "activities" does

In reply to: 249319450 [](ancestors = 249319450)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:833 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                "description": "Gets or sets the DTU."

Added a link, please let me know if this works.

In reply to: 249322893 [](ancestors = 249322893)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1453 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "Name": {

for some reason, these were uppercase in the Hyak specs.  Changing to lower case.

In reply to: 249323182 [](ancestors = 249323182)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1587 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "Name": {

Checked the backend implementation - they expect PascalCase for this one

In reply to: 249323182 [](ancestors = 249323182)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1587 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

    "operationId": "Databases_List",

Databases_ListBySqlServer
Applicable to all child resources ListBy

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:279 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

    "operationId": "Servers_List",

ListByResourceGroup

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:466 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

    "operationId": "Servers_List",

I am not able to find List at subscription level, I tried the API it worked, looks like you are missing api
/subscriptions/{subscriptionId}/providers/Microsoft.Sql/servers

In reply to: 253362372 [](ancestors = 253362372)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:466 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

In general the SQL RP only contains hierarchical list operations (i.e. list servers in resource group, list databases in server), listing all in a subscription is not implemented.
From: Anudeep Sharma<EMAIL_ADDRESS>Sent: Wednesday, October 12, 2016 3:57 PM
To: Azure/azure-rest-api-specs<EMAIL_ADDRESS>Cc: Jared Moore<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [Azure/azure-rest-api-specs] Initial Microsoft.Sql Swagger Spec (#555)
"operationId": "Servers_List",

I am not able to find List at subscription level, I tried the API it worked, looks like you are missing api
/subscriptions/{subscriptionId}/providers/Microsoft.Sql/servers

In reply to: 253362372https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-rest-api-specs%2Fpull%2F555%23issuecomment-253362372&data=01|01|Jared.Moore%40microsoft.com|440a809a56a447274a7c08d3f2f30558|72f988bf86f141af91ab2d7cd011db47|1&sdata=x9PMGJfGR1tHTGdTO4BPPmE4aBGe5UAorjWSdnTR8TE%3D&reserved=0

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:466 in 5f0c465https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-rest-api-specs%2Fcommit%2F5f0c465770d8c5842d5a103838c2950897ed4b66&data=01|01|Jared.Moore%40microsoft.com|440a809a56a447274a7c08d3f2f30558|72f988bf86f141af91ab2d7cd011db47|1&sdata=tnKfWGSijskRxkc1JFIeReYdZIVoXFt8kSNhCS06%2BmE%3D&reserved=0.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-rest-api-specs%2Fpull%2F555%23issuecomment-253363349&data=01|01|Jared.Moore%40microsoft.com|440a809a56a447274a7c08d3f2f30558|72f988bf86f141af91ab2d7cd011db47|1&sdata=jaxJbkeO7%2BQ1LMyYm3uofwnYDes7zibLE7DN%2FuOMRqo%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FADGS99xseUmFNxjCX_JztRMBYE86xh-Yks5qzWWjgaJpZM4KBATA&data=01|01|Jared.Moore%40microsoft.com|440a809a56a447274a7c08d3f2f30558|72f988bf86f141af91ab2d7cd011db47|1&sdata=tbaKkpaegx4LMQz0vKAlDlyDsetwWb%2Beu37AB8a3uxA%3D&reserved=0.

                "202": {

yes

In reply to: 249319207 [](ancestors = 249319207)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:583 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                "202": {

Yes

In reply to: 249319207 [](ancestors = 249319207)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:583 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                "202": {

Yes

In reply to: 249318878 [](ancestors = 249318878)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:514 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

    }

should not this be read only, I am assuming you won't be returning the password in get call

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1929 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

"ServerProperties": {

When I get the SQL Server through rest API, there are 2 more properties returned externalAdministratorLogin and externalAdministratorSid, in case these are settable then in case of piping scenario these properties will be set to null with get and put call

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1911 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

    },

Is version enum, can I pass any value in, is it required?

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1921 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

    "operationId": "Databases_TransparentDataEncryption",

Revise

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1309 in e53d017. [](commit_id = e53d017dd26ce908be94f9b678dd68d28278f999, deletion_comment = False)

            "value": {

Applying this to all List Result definitions

In reply to: 249323883 [](ancestors = 249323883)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1901 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                "201": {

This just does a PUT against the TDE settings, overriding it with On/Off

In reply to: 249320271 [](ancestors = 249320271)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1355 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "ScheduleUpgradeAfterUtcDateTime": {

This was implemented with PascalCase.

In reply to: 249325149 [](ancestors = 249325149)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1982 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "dtu": {

They are not readonly

In reply to: 249323444 [](ancestors = 249323444)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1628 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "Name": {

This is expected to be passed in.

In reply to: 249325513 [](ancestors = 249325513)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2021 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

        "properties": {

This is how it's implemented

In reply to: 249326629 [](ancestors = 249326629)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2020 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "collation": {

I think there are so many options it's probably going to be hard to keep an enum up to date.

In reply to: 249326793 [](ancestors = 249326793)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2192 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

            "collation": {

I'm putting a link to collations so if someone is actually trying to set this in their request, they have somewhere to begin

In reply to: 253680997 [](ancestors = 253680997,249326793)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2192 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

"ServerProperties": {

This doesn't match our other naming patterns...

In reply to: 253080150 [](ancestors = 253080150)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1911 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

"ServerProperties": {

Good catch.  Those two appear unused right now so I'm guessing they will be used in the future.

In reply to: 253612408 [](ancestors = 253612408)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1911 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

SQL supports this API, thats why I am asking it to be in swagger.
https://management.azure.com/subscriptions/9657ab5d-4a4a-4fd2-ae7a-4cd9fbd030ef/resourcegroups/ans/providers/Microsoft.Sql/servers?api-version=2014-04-01 resulted me all the servers in the subscription. This is only supported for servers and I get it.

In reply to: 253363816 [](ancestors = 253363816)

e53d017dd26ce908be94f9b678dd68d28278f999

merge error

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1934 in 559334f. [](commit_id = 559334fa9559107b323324efb4183353b814e707, deletion_comment = False)

      "description": "The recovery period starte date of the Azure SQL Database. This records the start date and time when recovery is available for this Azure SQL Database."

typo - starte = start

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2292 in 559334f. [](commit_id = 559334fa9559107b323324efb4183353b814e707, deletion_comment = False)

            "status": {

This one is subject to updates and probably shouldn't be an enum.

In reply to: 249326875 [](ancestors = 249326875)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2257 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                    "modelAsString": true

Since it's on/off, removing for now

In reply to: 249326204 [](ancestors = 249326204)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2169 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                    "modelAsString": true

Subject to change, so I'm leaving this one

In reply to: 249326847 [](ancestors = 249326847)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2234 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

                    "modelAsString": true

Looks like it won't change, removing for now

In reply to: 249327143 [](ancestors = 249327143)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2526 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

"Server": {

Discussed offline, leaving it as Server for now and it will be overwritten in Fluent

In reply to: 253080183 [](ancestors = 253080183)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1933 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

"ServerListResult": {

Discussed offline, leaving it as Server for now and it will be overwritten in Fluent

In reply to: 253080244 [](ancestors = 253080244)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1948 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

    "operationId": "Databases_List",

Discussed offline, leaving it as Server for now and it will be overwritten in Fluent

In reply to: 253362293 [](ancestors = 253362293)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:279 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

This worked for me too.

In reply to: 253686627 [](ancestors = 253686627,253363816)

      "format": "uuid",

this is not a guid, I get the values like "S3" back in response on get on SQLDatabase

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2313 in 943ad32. [](commit_id = 943ad327b56a6193c4eaa44f76705fb91b426a99, deletion_comment = False)

    },

Looks like read only property, as setting it to any value other a specific one leads to validation error.

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2300 in 943ad32. [](commit_id = 943ad327b56a6193c4eaa44f76705fb91b426a99, deletion_comment = False)

    },

Tried it on a premium db - this can be changed.

In reply to: 254306591 [](ancestors = 254306591)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2300 in 943ad32. [](commit_id = 943ad327b56a6193c4eaa44f76705fb91b426a99, deletion_comment = False)

    },

Databases by default are at their max size, so if you were trying to increase, that may also be why it didn't work

In reply to: 254319034 [](ancestors = 254319034,254306591)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2300 in 943ad32. [](commit_id = 943ad327b56a6193c4eaa44f76705fb91b426a99, deletion_comment = False)

"DatabaseProperties": {

One missing property containmentState

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2278 in 97d4077. [](commit_id = 97d40778ddbbb3ea69e9b92cacb29d733493429c, deletion_comment = False)

    },

I tried downsizing as well.
2 problems here

Your service takes/returns string rather than number, string is representing a number though.
Although you are expecting number of bytes, but you support multiples of GBs/MBs (I have not done full testing to find out what it is).

Please check what is the valid values, and at least put comment for the same, better put validation here.
@amarzavery - any guideline here for this problem.

In reply to: 254319377 [](ancestors = 254319377,254319034,254306591)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2300 in 943ad32. [](commit_id = 943ad327b56a6193c4eaa44f76705fb91b426a99, deletion_comment = False)

    }

Make sure Schema has all the properties of Resources, from this file it seems Schema is contained in a Database resource as an array

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2726 in 97d4077. [](commit_id = 97d40778ddbbb3ea69e9b92cacb29d733493429c, deletion_comment = False)

    }

I am not able to find it's other usage, where it should be a gettable resource on it's own, is some path is missing.

In reply to: 254339114 [](ancestors = 254339114)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2726 in 97d4077. [](commit_id = 97d40778ddbbb3ea69e9b92cacb29d733493429c, deletion_comment = False)

    },

Comment added.

In reply to: 254338528 [](ancestors = 254338528,254319377,254319034,254306591)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2300 in 943ad32. [](commit_id = 943ad327b56a6193c4eaa44f76705fb91b426a99, deletion_comment = False)

        "description": "The comma separated list of child objects to expand in the response. Possible properties: serviceTierAdvisors, upgradeHint, transparentDataEncryption, advisors, recommendedIndexes."

expanding advisors giving me 500.
For more information on backend
x-ms-correlation-request-id: 6c48303f-b995-416d-a18f-93831a7bfd39
x-ms-path-query: /subscriptions/9657ab5d-4a4a-4fd2-ae7a-4cd9fbd030ef/resourceGroups/ans/providers/Microsoft.Sql/servers/ans-secondary/databases/importeddb?api-version=2014-04-01-preview&$expand=advisors

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:261 in 97d4077. [](commit_id = 97d40778ddbbb3ea69e9b92cacb29d733493429c, deletion_comment = False)

    },

Updated integer to string.

In reply to: 254362734 [](ancestors = 254362734,254338528,254319377,254319034,254306591)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2300 in 943ad32. [](commit_id = 943ad327b56a6193c4eaa44f76705fb91b426a99, deletion_comment = False)

        "description": "The comma separated list of child objects to expand in the response. Possible properties: serviceTierAdvisors, upgradeHint, transparentDataEncryption, advisors, recommendedIndexes."

recommendedIndexes giving me 400 (bad request).
x-ms-path-query: /subscriptions/9657ab5d-4a4a-4fd2-ae7a-4cd9fbd030ef/resourceGroups/ans/providers/Microsoft.Sql/servers/ans-secondary/databases/importeddb?api-version=2014-04-01-preview&$expand=recommendedIndexes
Response - {"code":"BadRequest","message":"Could not find a property named \u0027recommendedIndexes\u0027 on type \u0027Microsoft.SqlServer.Management.Service.Domain.ArmResourceProvider.Database\u0027.","target":null,"details":[],"innererror":[]}

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:261 in 97d4077. [](commit_id = 97d40778ddbbb3ea69e9b92cacb29d733493429c, deletion_comment = False)

    }

This is not gettable on its own.

In reply to: 254339530 [](ancestors = 254339530,254339114)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2726 in 97d4077. [](commit_id = 97d40778ddbbb3ea69e9b92cacb29d733493429c, deletion_comment = False)

    }

Not gettable on its own yet.

In reply to: 254339530 [](ancestors = 254339530,254339114)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2726 in 97d4077. [](commit_id = 97d40778ddbbb3ea69e9b92cacb29d733493429c, deletion_comment = False)

    "operationId": "Servers_ListBySubscription",

Servers_List

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:355 in 97d4077. [](commit_id = 97d40778ddbbb3ea69e9b92cacb29d733493429c, deletion_comment = False)

"SubResource": {

Rename it something else, as this is clashing with the name in SDK framework. You can name it SqlSubResource

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:3047 in 97d4077. [](commit_id = 97d40778ddbbb3ea69e9b92cacb29d733493429c, deletion_comment = False)

        "description": "The comma separated list of child objects to expand in the response. Possible properties: serviceTierAdvisors, upgradeHint, transparentDataEncryption, advisors, recommendedIndexes."

Removed.

In reply to: 254364661 [](ancestors = 254364661)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:261 in 97d4077. [](commit_id = 97d40778ddbbb3ea69e9b92cacb29d733493429c, deletion_comment = False)

        "description": "The comma separated list of child objects to expand in the response. Possible properties: serviceTierAdvisors, upgradeHint, transparentDataEncryption, advisors, recommendedIndexes."

Following up with the team that owns this.

In reply to: 254364352 [](ancestors = 254364352)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:261 in 97d4077. [](commit_id = 97d40778ddbbb3ea69e9b92cacb29d733493429c, deletion_comment = False)

"ServerProperties": {

Discussed offline.  Leaving as Servers for now.

In reply to: 253684763 [](ancestors = 253684763,253080150)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1911 in 5f0c465. [](commit_id = 5f0c465770d8c5842d5a103838c2950897ed4b66, deletion_comment = False)

        "description": "The comma separated list of child objects to expand in the response. Possible properties: serviceTierAdvisors, upgradeHint, transparentDataEncryption, advisors, recommendedIndexes."

Apparently the implementation for this is unsupported now.

In reply to: 254383312 [](ancestors = 254383312,254364352)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:261 in 97d4077. [](commit_id = 97d40778ddbbb3ea69e9b92cacb29d733493429c, deletion_comment = False)

            "unit": {

Talked to the owning team and they said it's not really used

In reply to: 249327222 [](ancestors = 249327222)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2565 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

      "202": {

Revert

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:176 in 36b30dd. [](commit_id = 36b30dde6114d3cd385a49caa089d6466bde2fd2, deletion_comment = True)

    "operationId": "Databases_ListRestorePointss",

Points

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:102 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

    "requestedServiceObjectiveId": {

this looks like optional parameter while creating the database, can it be any new GUID or some kind of enum

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2342 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

    "serviceLevelObjective": {

this looks more like the enum to me judging from result S3 (looks like it is UI's Standard S3 pricing tier).

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2351 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

    "requestedServiceObjectiveName": {

looks like enum, how this is different from serviceLevelObjective, should not both be same

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2347 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

    "edition": {

master database has edition as System

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2319 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

    "edition": {

master is getting returned in the Database_ListByServer, although get on master results in 404.

In reply to: 254888657 [](ancestors = 254888657)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2319 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

    "operationId": "Databases_CreateOrUpdate",

Log an bug on service that for Databases location should not be a required parameter.

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:143 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

    "requestedServiceObjectiveId": {

Looked further into this, and found that these are hard coded values against requestedServerObjectiveName, make it read only, no user is going to pass in hardcoded guids for which he has no knowledge

In reply to: 254885356 [](ancestors = 254885356)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2342 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

This was originally a deliberate decision to nest firewall rules methods inside the SqlServer class in generated code to reduce the number of top-level classes and provide better structure. The thinking was that a firewall rule is a very simple nested resource so it doesn’t need to be a full-fledged top level class in generated code. Are you recommending we change this plan? How do other RPs manage having a large number of resource types?
From: Anudeep Sharma<EMAIL_ADDRESS>Sent: Wednesday, October 19, 2016 11:20 AM
To: Azure/azure-rest-api-specs<EMAIL_ADDRESS>Cc: Jared Moore<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [Azure/azure-rest-api-specs] Initial Microsoft.Sql Swagger Spec (#555)
@anudeepsharma commented on this pull request.

In arm-sql/2014-04-01/swagger/serverFirewallRules.jsonhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-rest-api-specs%2Fpull%2F555%23pullrequestreview-4931104&data=02|01|Jared.Moore%40microsoft.com|50386d4226a44f37a98308d3f84c930c|72f988bf86f141af91ab2d7cd011db47|1|0|636124980177019411&sdata=EYMkGc105M%2FB1HGFrpLOwhMVGUr%2BwVHYBnEWz00%2BCh4%3D&reserved=0:

"schemes": [

"https"

],

"consumes": [

"application/json"

],

"produces": [

"application/json"

],

"paths": {

"/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Sql/servers/{serverName}/firewallRules/{firewallRule}": {

 "put": {

   "tags": [

     "FirewallRules"

   ],

   "operationId": "Servers_CreateOrUpdateFirewallRule",

Servers_CreateOrUpdateFirewallRule
SqlServerFirewallRules_CreateOrUpdate, similarly other apis
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-rest-api-specs%2Fpull%2F555%23pullrequestreview-4931104&data=02|01|Jared.Moore%40microsoft.com|50386d4226a44f37a98308d3f84c930c|72f988bf86f141af91ab2d7cd011db47|1|0|636124980177019411&sdata=EYMkGc105M%2FB1HGFrpLOwhMVGUr%2BwVHYBnEWz00%2BCh4%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FADGS932iB9FWTcN22huOiLPs0TApEy4Jks5q1l9egaJpZM4KBATA&data=02|01|Jared.Moore%40microsoft.com|50386d4226a44f37a98308d3f84c930c|72f988bf86f141af91ab2d7cd011db47|1|0|636124980177019411&sdata=wAeKLNhO9mcoEcw6PzueVKVTKD1%2FT1VMHuQOKk3WcxI%3D&reserved=0.

    "requestedServiceObjectiveId": {

Looked in to capabilities and looks like from there user will be able to get these values, so not sure how you want to design the APIs, do you want user to look at the latest capabilities and then set these values. IMO ServerLevelObjectives should be enums, modeled as string, as you will not be taking values away but putting it in.

In reply to: 254891285 [](ancestors = 254891285,254885356)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2342 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

    },

should this be enum, this is minor comment as this is read only property

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1651 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

arm-sql/2014-04-01/swagger/serverFirewallRules.json, line 24 at r1 (raw file):

SqlServerFirewallRules_CreateOrUpdate, similarly other apis

Quoted 14 lines of code…
>
>
>
>                   1 day ago
>
>
>
>
>
>               jaredmooJared Moore
>
>
>
>

Fixed

Comments from Reviewable

    "operationId": "Databases_CreateOrUpdate",

Addressed by updating documentation of the operation

In reply to: 254890360 [](ancestors = 254890360)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:143 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

        "properties": {

We don't use it right now

In reply to: 249329257 [](ancestors = 249329257)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2922 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

    "requestedServiceObjectiveId": {

Added enum

In reply to: 254901642 [](ancestors = 254901642,254891285,254885356)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2342 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

    "requestedServiceObjectiveName": {

They are the same, this one is requested, other is current.

In reply to: 254886811 [](ancestors = 254886811)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2347 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

    "serviceLevelObjective": {

#Resolved

In reply to: 254886470 [](ancestors = 254886470)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2351 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

    },

#Resolved

In reply to: 254947528 [](ancestors = 254947528)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:1651 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

    "edition": {

@akromm

In reply to: 254889055 [](ancestors = 254889055,254888657)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2319 in f17c0d1. [](commit_id = f17c0d11178737536131dd09b89ff7f92ae25d21, deletion_comment = False)

        "properties": {

Talked in person, decided we don't need it yet. #Resolved

In reply to: 255234164 [](ancestors = 255234164,249329257)

Refers to: arm-sql/2014-04-01/swagger/sql.core.json:2922 in c27d9ac. [](commit_id = c27d9ac361b7a31382dcacaa522134185766427d, deletion_comment = False)

The PR has merge conflicts. Please pull from upstream master and make sure to resolve the conflicts.
You can install Tortoise Git UI client. It is very useful in resolving conflicts. After pulling from upstream master, simply right click in the file explorer Tortoise Git-> Resolve conflicts. It will either auto resolve or ask you to resolve with a nice UI support.

I will be breaking this down into smaller PRs when I get time.


Add xms-ids for multiple RPs (maintenance~marketplaceordering)

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[ ] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[x] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
If updating an existing version, please select the specific langauge SDKs and CLIs that must be refreshed after the swagger is published.

[ ] SDK of .NET (need service team to ensure code readiness)
[ ] SDK of Python
[ ] SDK of Java
[ ] SDK of Js
[ ] SDK of Go
[ ] PowerShell
[ ] CLI
[ ] Terraform
[x] No refresh required for updates in this PR

Contribution checklist:

[x] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[ ] Check this box if any of the following apply to the PR so that label "WaitForARMFeedback" will be added automatically to begin ARM API Review. Failure to comply may result in delays to the manifest.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you are using OpenAPIHub to initialize the PR for adding a new version. More details, refer to the wiki.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If any of the following scenarios apply to the PR, request approval from the Breaking Change Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in a stable version
[ ] Removing properties in a stable version
[ ] Removing API version(s) in a stable version
[ ] Updating API in a stable or public preview version with Breaking Change Validation errors
[ ] Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

This PR is to add the missing x-ms-identifiers metadata: http://azure.github.io/autorest/extensions/#x-ms-identifiers for the following RPs:

Maintenance
Managednetwork
Managedservices
Managementgroups
Managementpartner
Maps
Mariadb
Marketplace
Marketplacenotifications
Marketplaceordering

Also @ArcturusZhang who had looked into similar failures of SDK zaure-sdk-for-go check a couple days ago. @ArcturusZhang any idea who could be looking at the azure-sdk-for-python failures?

@msyyc
PS: why we mark T1 codegen as required, but not T2?

Hi @dankorenMicrosoft @MiriBerezin we Change Analysis team is adding missing x-ms-identifiers metadata: http://azure.github.io/autorest/extensions/#x-ms-identifiers for all RPs and I wonder if you or someone on your team could help review the changes of the Marketplace RP. I am reaching out as I saw you were the last people made changes to that file and apologize if you are not the right person for this code review.

Hi, track1 is not maintained anymore and please ignore the related failure.

I'm From Maps RP, LGTM, but do we have any new linting rule to enforce these new properties?


More fixes towards 14 1 0

This is a collection of fixes for issues that I found while working on the IOCipher integration.  Full comments are in the commit messages.

Also, https://github.com/n8fr8/gibberbot/commit/0bbb9082b1b1dd56e8509c8bfd9913e80e87bd7b should not be included since eighthave@df3cc88

Well, my changes still make sense, but aren't necessary, true. Ultimately, if a message has a mime-type for media, there will be nothing to linkify.

We might be able to linkify media in the message using Linkify, that's why I stuck it where I did.  But I haven't yet looked into the details yet.  For example, perhaps it would linkify a QR Code image.  But maybe that's just silly.


Modal examples don't work right

Improve documentation
Link
https://ui.supabase.com/components/sidepanel
Describe the problem
When you click on the Modal examples I think they take as a reference the top of the page, and as you scroll down you can see the modals
Describe the improvement
Fix the code
Additional context

thanks @rodbs
the whole website is out of date. All of the examples need updating to use the alpha pre release version.


Minimal viable trinity-beacon node

Abstract
This issue tries to document the current state of the trintiy-beacon command and lays out steps to turn it into an actual minimal viable beacon node. This was sparked by @ChihChengLiang starting this document which means it is also trying to address questions outlined there.
Background: How does the beacon node implementation fit into the code base?
In general, the relation between the beacon node and the existing Ethereum 1 client is very loose at this point.

The trinity-beacon command is an entirely different command from trinity and invokes its own bootstrapping routine at trinity/main_beacon.py. Bootstrapping code does share some common utility functions.

The long term vision for Trinity is to be mostly a composition of plugins with many essential parts just being written as builtin, pre-loaded core plugins (while other use defined plugins can be loaded at runtime)

We can see this already working out with the collection of current plugins.
https://github.com/ethereum/trinity/blob/7436d9c3fdef5fbeef5d7a1123c8ed2ecd16e033/trinity/plugins/registry.py#L49-L68
The plugin infrastructure also takes care of spawning isolated processes for each plugin that derives from BaseIsolatedPlugin and enables inter-process communication through a central EventBus.

Trinity provides a TrinityConfig that should be agnostic of anything that is specific to the Ethereum1 or beacon node. These specific configs need to go into the BeaconAppConfig which will be available in the context of the beacon node (similar to the Eth1AppConfig being available in the context of the Ethereum 1 node).

Currently, the beacon node uses all base plugins (not the Ethereum 1 specific plugins) which includes things like the Peer Discovery as well as the JSON-RPC API.

Does the beacon node need a Node class?
Probably not. The Node / FullNode / LightNode classes are currently used in the Ethereum 1 node to run what is currently known as the networking process. This process does currently host the PeerPool, the server that answers request from other peers as well as the syncing code.
We are actively migrating away from that architecture towards an architecture where the responsibilities are further broken down into different processes that are assembled through plugins.
Ideally, we'll soon have something that looks like this:

Peer Pool Plugin (own process) -> manages peers, relays incoming messages on the EventBus

Sync Plugin (own process) -> runs the sync, interacts with the PeerPool via the EventBus (probably hidden a convenience API) (see comment)

Chainserver Plugin (own process) -> serve other clients,  interacts with the PeerPool via the EventBus

When that happens, we won't have a hardcoded networking process anymore and instead have three processes all bootstrapped through plugins just like the discovery and JSON-RPC are already today.
Most likely, this does also mean there won't be a Node class anymore as these plugins will use their more focused abstractions.
Sooo?
For the beacon node, there are two possible ways from here:

Accelerate the re-architecture process by jumping on #213 to get to that new architecture sooner (I'm on it, but I encourage others to also experiment in the same direction if they want to)

Otherwise, just create a new plugin BeaconNodePlugin that just bootstraps a PeerPool, server, syncer in more or less the same way as the networking process currently does. This would be temporary code that would then later be broken down into smaller plugins as soon as we have a PeerPool that supports being run in its dedicated process and integrates natively with the EventBus. (see comment)

Where do we stand with the database process?
The beacon node does have its database process bootstrapped but we still need to handle the initialization of the database when its empty.
Do we need a database_mode?
The database_mode is used in the Ethereum 1 node to differentiate between light and full nodes that use an incompatible database. The beacon node should be free to handle database access in its own way however it is needed. This is why the database_mode is a property on the Eth1AppConfig and not on the TrinityConfig.
I believe in later phase, the beacon node would at least manage one database for the beacon chain and n databases for the shard. I currently do not know if there needs to be something like a different database for light nodes?

IIUC, to use the new architecture, the best timing to implement Sync Plugin and Chainserver Plugin is when Peer Pool Plugin is completed?

@ChihChengLiang that is correct but I wouldn't want you being blocked on that. So, I would propose that you just move forward with the Peer Pool, Chain Server and Sync all running in one process similar to the current architecture of the Ethereum 1 node.
To do that, you can either:

Implement a Node class similar to the one currently being used for Ethereum 1 and then just kick that off in a BaseIsolatedPlugin.

Just forget the Node class and throw together a BaseIsolatedPlugin that bootstraps the PeerPool, Server and Syncer in an ad hoc way (comparable to how it is assembled under tests)

Then, later, when we have a PeerPool that can run in its own isolated plugin with an EventBus-based API consumable from other processes, we can migrate to use that and to move chain server and sync into distinct plugins as well.

Thank you very much for this detailed write up. It cleared up a couple of things for me.

I currently do not know if there needs to be something like a different database for light nodes?

I'd say we should ignore light nodes for a long time.

I believe the current beacon node implementation is already past this and we can close this issue.


Arxivce 1950 obj change use surrogate key

creating a seperate cloud function to test rather than directly replacing the code


Declutter bug: symbols not getting associated with sentences in paper 2002.04138

Some symbols very early in the paper are successfully getting associated with sentences. Others are not. It appears that all symbols starting at subsection 2.1 "Fundamental Axioms for Interaction Values" are not getting associated with sentences.
This appears to be an issue with detecting the sentences that equation tokens belong to---the entities-sentences.csv file output by the find-sentences-for-equation-tokens is very short given the huge number of tokens in the paper. There may be a bug in the code for detecting sentences that contain equation tokens.

Should be fixed in 24e56fd2d15b4167583af616637a56fdf75e53b5


benchmarks: Fix the BitCount benchmark

*) Increase the iteration count, so that we cat a time value > 100 for better accuracy
*) Use getInt to prevent constant propgation and loop hoisting

@swift-ci Please smoke test and merge


Using whole host url as input including scheme: compatibility with parse-server

Hi,
With the parse server transition, it will be better to not hardcode or enforce the https scheme as the host url but let the user set the host dynamically (https is tedious to setup in a local server).
I will also suggest that we should just remove the VERSION field in protocol module.

It should be fixed on master.


Add data sampling class

Close #10

Add data sampling class
Add tests for data sampling
TODO: Add docs for data sampling
Simplify existing code

TODO: Maybe rename to StateSampling to make more explicit what the outcome is? (one RL environment state = Grid state)


The computaion of neg_reward is wrong

This code uses batch-averaged (sample_rouge - baseline rouge), but it don't make sense in math and this item should be sample-wise because what we really want to maximize is this:

Check https://github.com/ymfa/seq2seq-summarizer/issues/7 . The negative sign is included in the LogP so the author has reversed it in the reward.


Resolve JSON.CONTENT as object or array

Background
Currently the following are supported and resolved i.e. JSON.FILE
i.e. ${JSON.FILE:path/to/actual/json/content.json}" gets resolved during runtime.
{
    "scenarioName": "POST via file body - reuse payload",
    "steps": [
        {
            "name": "create_emp",
            "url": "/home/api/employee",
            "operation": "POST",
            "request": {
                "body" : "${JSON.FILE:integration_test_files/filebody/bodyonly/request_body.json}"
            },
            "assertions": {
                "status": 201
            }
        }
    ]
}

AC1:
Simillar way:
Support and resolve JSON.CONTENT (this time not from a file but from this scenario step(s))
i.e.
${JSON.CONTENT:<json_path_to_a_json_object_or_array in this scenario file>} gets resolved during runtime.
where example of,
json object :
{
   "foo": "bar"
}

json array :
[
   {
      "foo1": "bar1"
   },
   {
      "foo2": "bar2"
   }
]

and also,
[1,2,3]

or

["string1", "string2"]

etc

Example Usecase Scenario
{
    "scenarioName": "GET Employee Office Address",
    "steps": [
        {
            "name": "get_emp",
            "url": "/home/api/employee/EMP101",
            "operation": "GET",
            "request": {
                "x-req-token": "11-22-3abe"
            },
            "assertions": {
                "status": 200,
                "body":{
                    "employee":{
                        "id":"EMP101",
                        "type":"CONTRACT",
                        "address":{
                            "type":"office",
                            "line1":"99 London Newton St",
                            "postcode":"LON3 9XR"
                        }
                    }
                }
            }
        },
        {
            "name": "create_new_emp",
            "url": "/home/api/employee",
            "operation": "GET",
            "request": {
                "body" : {
                    "fullName":"Bob Luis",
                    "empLogin":"bolu_lon",
                    "type":"CONTRACT",
                    "address":"${JSON.CONTENT:$.get_emp.response.body.employee.address}"
                }
            },
            "assertions": {
                "status": 201,
                "body":{
                    "employee":{
                        "id":"EMP102",  // <--- New employee with a new ID
                        "type":"CONTRACT",
                        "address":{     // <--- Same office address as of earlier employee
                            "type":"office",
                            "line1":"99 London Newton St",
                            "postcode":"LON3 9XR"
                        }
                    }
                }
            }
        }
    ]
}

Suggestion:
Try to reuse the code implemented for JSON.FILE for ser-de logic.

Can you assign me this task

Can you assign me this task

Done.
Thanks for picking this.
Can you please share or link the PR here or to this issue when you're ready?
ETA:
Around 2 weeks. 20th Nov 2023.

Can you assign me this task

Hello @kpidiba, Hope you're well. It seems like there is not lot going on wrt the PR we are aiming by 20th Nov.
Forked repo: https://github.com/kpidiba/zerocode
It could be you are busy with your day job or so which is not a problem.
So could you please unassign to yourself and assign to @furkannzmnn to get this issue fixed?
Thanks for understand

Can you assign me this task

Thanks for your interest!
Now you can proceed please @furkannzmnn.

Hi @authorjapps would you please assign this issue to me if it is available.

Hi @authorjapps would you please assign this issue to me if it is available.

Done @vivektiwary , thanks for your interest.
Do you have an ETA for this?

I think i would be able to do it in 2 weeks. I already went through
JSON.FILE implementation.
Thank you
Vivek
On Mon, 4 Dec 2023 at 10:35 PM authorjapps @.***> wrote:

Hi @authorjapps https://github.com/authorjapps would you please assign
this issue to me if it is available.
Done @vivektiwary https://github.com/vivektiwary , thanks for your
interest.
Do you have an ETA for this?
—
Reply to this email directly, view it on GitHub
https://github.com/authorjapps/zerocode/issues/601#issuecomment-1839089360,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABGACIRQFKFNTNEEK5RGK7TYHX7HBAVCNFSM6AAAAAA6X5N76KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZZGA4DSMZWGA
.
You are receiving this because you were mentioned.Message ID:
@.***>

PR https://github.com/authorjapps/zerocode/pull/612 has been merged.
This feature is available in 1.3.36 (and higher releases) once released.
Documentation is et to be created/updated for this.
Visit here and update/contribute to the Doc for this: https://zerocode-tdd-docs.pages.dev (and raise a PR once done)


[wip] Settle payments

this adds a method to change balances/liquidity in the OracleLightningNetwork
I added a setter to the OracleChannel Class for liquidity
Tests still need to be done, but first feedback welcome, as there are several solutions.

basic example now works. still a very verbose code, so that checks regarding the uncertainty network can be made with the log file


fix: unrug app when missing chain adapter

Description
Types things what they actually are at runtime.

Removes non-null assertions, as they are invalid
Makes AssetsById a PartialRecord
Handles all cases where an Asset lookup by dynamic key can be undefined

Unrugs local state when Osmosis (or any other) ChainAdapter is missing and there are accounts available.
This might save us pain down the road.
Notice

[x] Have you followed the guidelines in our Contributing guide?
[x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Pull Request Type

[ ] :bug: Bug fix (Non-breaking Change: Fixes an issue)
[x] :hammer_and_wrench: Chore (Non-breaking Change: Doc updates, pkg upgrades, typos, etc..)
[ ] :nail_care: New Feature (Breaking/Non-breaking Change)

Issue (if applicable)
N/A
Risk
Medium. Touches a lot of thing - we'll want a good old regression check.
Testing
In most cases the app should function exactly as it did before.
Engineering
Is Osmosis was enabled, and then turned off, we should no longer have an unusable app.
Operations
☝
Screenshots (if applicable)
N/A

@gomesalexandre should be good to go, ser.


chore(deps): upgrade dev dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-dev-dependencies-k8s-26-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 3292

Questions ?
Please refer to the Backport tool documentation


Study SHAP to see if there are interesting Shapley approximation algos

https://github.com/slundberg/shap

I  read paper 6 which refer to this paper  If I understand right they propose a computation by stratification similar to what is proposed in this other paper

paper 5 suggest using monte-carlo sampling


Ask for code release time

I'm sorry to bother you,but I want to know when will the code plan be released?

Thank you very much.


add plugin to check signature of plugins

Addresses https://ucsc-cgl.atlassian.net/browse/SEAB-1534

I would like to emphasize that a plugin signature failure will cause a build to fail; there does not seem to be a way to only issue a warning

@wshands didn't look in detail, what could cause a plugin signature?
A man in the middle attack with someone pretending to be a legit Maven artifactory?

I would like to emphasize that a plugin signature failure will cause a build to fail; there does not seem to be a way to only issue a warning

That seems like the right behavior.


Polyline fixes

After much adieu, I have a PR that I've tested on Chrome/iOS/Android/IE and appears to solve the following issues reliably:

IE clockwise issues
mobile touch issues (closing polylines/polygons)
a few other issues that i noticed as i was knocking down these issues, such as distance tooltip throwing errors on mobile views

description of code changes

enables a flag, uses a timeout to briefly disable duplicate polyline touchevents
binds touch/click events more discriminately based on the browser
disables distance tooltip which was throwing errors on touch devices
additional touchHandled/clickHandled flags for safety
misc cleanup, more intelligent logic in some places

these changes result in a much more cross-platform stable version of leaflet-draw, and so I hope that despite the lack of tests, it can be merged, as I may not get a chance to write tests anytime soon.  i tried to tag in all of the issues i believe that it probably fixes.

@z0d14c It's pretty close, I squashed it over in https://github.com/Leaflet/Leaflet.draw/pull/661 - I have some notes for a few little quirks in that PR.

Any progress with this?

This has been merged with https://github.com/Leaflet/Leaflet.draw/pull/661


Tooltips & x-axis mode v2.2.0

Hi,
1 - Tooltips  arrow default position is "right" and we can see the arrow blink from the right to its calculated position during fade in and fade out animation.

2 - For bar chart, x-axis mode (thanks for this option @Mesonyx) depends of "stacked" param
if stacked is true, mode = label

if stacked is false, mode = single

So i'm wondering if "x-axis" shouldn't be an option instead of a "mode". It will give the liberty to have x-axis + single or x-axis + label.
3 - For line chart, when pointRadius = 0 (for a clean design or when there is a lot of data points) tooltip is visible only if your cursor is on the 1px corresponding to the X.

It is possible to have something like padding left/right = space between two points / 2 for the "hover zone" of the tooltip.

The animation is definitely a bug. I thought it was already reported, but I couldn't find a duplicate issue.
That's an interesting idea for the x axis hover. Thoughts @simonbrunel @zachpanz88 @Mesonyx ?
For #3, the points have a hitRadius option that is a number that is added to the radius to determine the hit region. The default is 1. This could be changed, but it isn't what you really want. One thing I have envisioned is to have a 'nearest' hover / tooltip mode. This would always pick the nearest point and then use that. Would that be more like what you are looking for?

Nearest mode should be perfect :)

nearest mode is exactly the 50% rule between two data points ( at least for tooltips.mode: 'x-axis'). would be great to have such an option!

@frlinw the concept of the x-axis mode was I think implemented in #3400. We now have an intersect mode that configures tooltip modes to run either all the time or only when the cursor intersects an item in the chart. Will try and figure out why the flip happens though .. it could be that something is calculated during draw instead of during the update

Great work, waiting for 2.4 so :) I will test it asap

@frlinw you can do index mode and intersect: false which will find the nearest item, then display the tooltip for all things at the same index.

Fixed last remaining issue, the weird arrow during hover, in #3446


Nex 5R Where to Switch between pal and NTSC?

Where on the Nex 5R to Switch between Pal and NTSC Video Setting??
OpenMemories Tweak is installed and enable Pal/NTSC Selector is enable.

I don't think it is possible in the 5R at least not with the current settings enabled in the app. There may be a way to do it but it hasn't been done yet.


nodePackages.ionic: init at 3.9.2

Motivation for this change
add ionic cli for cross-platform app development with the ionic framework: http://ionicframework.com/getting-started/
nix-env -f . -iA nodePackages.ionic nodePackages.cordova
ionic start myApp tabs

Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] Linux

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Fits CONTRIBUTING.md.

Thanks!


API not giving all videos data

When I try to fetch a user's all videos, the users have 2900 videos but API is not giving all videos.
Sometimes it gives only 250 videos and sometimes it gives only 550 and sometimes it gives nothing.
I tried many times by changing the IP but the same thing happens.
I think API is failing in getting all videos of users.
I run the all-videos.php in your API example folder

Alrighty, I can look into it. It's possible that they are limiting the requests. It may require me placing a sleep on the method.
I'll find some time tonight to test it.
Thanks.

Thank you @RobertGrubb for your response!

Have not forgot about this, however broken video downloads took priority over this. Will circle back.

This comes down to the main issue that we're facing while scraping TikTok, which is captcha being returned.
Once we get that portion fixed, all tickets regarding that will be solved. Closing this as this issue is the main focus.


OLS-1317: Allow attaching YAML / YAML filtered for ConfigMaps

Override failing test not related to this change:
/override "Red Hat Konflux / ols-enterprise-contract / lightspeed-console"


Fix merge_rsem_gene process in main.nf

merge_rsem_gene was not throwing an error message when only 1 sample as input of the pipeline => now fixed  by changing ${rsem_res_gene.get(0)} into ${rsem_res_gene[0]}.
I've also corrected the name of the output file of that same process as the column of RSEM output we keep is the "expected count" column and not the "TPM" one.
I think it was confusing to call the output "rsem_tpm_gene.txt" as this wasn't "transcripts per million"

I've just noticed I changed the name of the RSEM output file without correcting the documentation...

It looks like all the tests failed with Process get_software_versions terminated with an error exit status (127) and
Command error:
  Loading required package: limma

I don't understand why this step would fail as I made changes in a step a lot further down the pipeline...


Improve readme example code

Based on this article that covered use of this library the following changes from it's summary could be applied to the current readme:

mipidsi documentation should include required use statements in its example code, as well as linking out to required crates
mipidsi documentation should not reference the no-longer-used Rgb666 color type

These changes would improve beginner journey.

I can have a look, one problem is that a "real" example  case would need to use some sort of specific embedded-hal implementation to get the SPI for example. That would bring a whole tree of dependencies in, and would be generally useless as an example because it'd only work for people with that setup anyhow.

mipidsi documentation should not reference the no-longer-used Rgb666 color type

What do you mean with this exactly? Rgb666 is a valid color, it's just RawU18 that will go away AFAIK


SQL code editor scrolling not working as expected on firefox

With mouse hovering within the editor pane, rather than scrolling only the editor pane, the entire page scrolls
See intercom conversation for more details:
https://app.intercom.io/a/apps/nnz2jx08/inbox/inbox/conversation/23254447805#part_id=comment-23254447805-3562652816

just checked, can't reproduce. @BenBirt @dwl285 I believe its fixed after some change from George


watch instead of loop check nested cluster

What this PR does / why we need it:
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #137

with workaround from #201 at least I can create a nested cluster follow the dev guide with the new PR..

/lgtm
/approve


Disable fzf --preview for binary files

Category

[x] fzf binary
[ ] fzf-tmux script
[ ] Key bindings
[ ] Completion
[ ] Vim
[ ] Neovim
[ ] Etc.

OS

[x] Linux
[ ] Mac OS X
[ ] Windows
[ ] Windows Subsystem for Linux
[ ] Etc.

Shell

[x] bash
[ ] zsh
[ ] fish

I've following in my ~/.bashrc:
export FZF_DEFAULT_OPTS='
	--height 40% --multi --layout=reverse --border
	--bind ctrl-f:page-down,ctrl-b:page-up
	--preview "head -100 {}"
'

But this also includes preview for binary/non-text files. So, I tried the following:
export FZF_DEFAULT_OPTS='
	--height 40% --multi --layout=reverse --border
	--bind ctrl-f:page-down,ctrl-b:page-up
	--preview "if file {} | grep -i 'text'; then head -100 {}; fi"
'

This doesn't include the external command for binary files. But the preview window remains active i.e. the border remains.
How can I disable preview, i.e. remove the preview border for binary files?
PS I know about keybinding to toggle preview window. But I want it to toggle automatically depending on whether the currently selected line is a binary file or text file.

toggle automatically depending on whether the currently selected line is a binary file or text file

It's not supported. You might want to print a message saying that it's a binary file (e.g. https://github.com/junegunn/fzf.vim/blob/master/bin/preview.sh#L25-L28)
Having said that, setting context-sensitive preview option in FZF_DEFAULT_OPTS is a very bad idea.
seq 100 | fzf
ls -al | fzf
ps -ef | fzf
cat /usr/share/dict/words | fzf
history | fzf

You get the point.

@junegunn Thanks


Feature 3.4:  Hot backup delete API (single server)

This implements the delete operation.  A sample API call to delete the FAILSAFE directory is:

wget --body-data '{"directory": "FAILSAFE"}'  --method DELETE http://<IP_ADDRESS>:8529/_admin/hotbackup/create

The lock api is stubbed out and available, though not functional:

wget --post-data '' http://<IP_ADDRESS>:8529/_admin/hotbackup/lock

 wget --body-data '' --method DELETE  http://<IP_ADDRESS>:8529/_admin/hotbackup/lock

All, I believe this PR is ready for a final review.


How to deal with relative urls for resources?

Once you minify css and js all the relative urls go to on one file. And many resources e.g. images get linked wrong from your css or js.  How do you deal with relative urls for resources? Or u have to edit all urls used in css and js files to be absolute? I have had to manually do that.

Well you should place the build folder in the same folder as your cs/js folders... you get something like this:
assets/
|-css/
|-js/
|-img/
|-fonts/
|-build/
Then you reference in your css, the fonts and images as being ../fonts/font.eof and ../img/image.png and the folder structure holds.


fix: add support for The Events Calendar

The Events Calendar
does not use local media files. However, it was looking for a filename
'silence' which was confused by the cache-buster.

Related conversion on .org forums at https://wordpress.org/support/topic/incompatibility-with-the-events-calendar-2/

Hello @donbowman
We are going to include this feature to our next release. Do you still use this compatibility? Can you please provide some details about configuration where do you use it.
Thank you in advance

yes i still use it.
there's no specific config, if you have the events calendar installed it fails w/o this change, more or less right away.