Decide how to deal with heapless-v0.8

heapless types appear in the public interface, so upgrading it would be a breaking change.
I am not certain, but I've considered doing the following:

adding heapless-v0_7 and heapless-v0_8 features, using those versions specifically, maybe in some module-d scope, for things like to_vec, to_vec_cobs
keeping heapless as an alias for heapless-v0_7, but marking it as deprecated.

Open to feedback if people think there is a preferrable solution.
Originally posted by @jamesmunns in https://github.com/jamesmunns/postcard/issues/115#issuecomment-1877214683

Hi James,
Latest postcard has a dependency on "heapless 0.7.17" which depends on atomic-polyfill.  I'm trying to upgrade my dep tree to move to portable_atomic supports AtomicU64. Do you think you can help with this please?
Cheers

Hi,
In my view the best approach would be to get rid of other crates in the public API, as you proposed in #128 . However, I assume that releasing 2.0 might take a while, so as a temporary solution doing what you proposed here (having the 2 features flags) sounds reasonable.
In one of my projects I've used heapless 0.8 for quite a while before deciding on using postcard, which means that I either need to do a breaking downgrade now or implement my own heapless (0.8) compatibility for postcard. I guess I could also depend on postcard master, but I'm not really keen on doing so.


Prevent particles from spawning automatically no longer works

Contact Details
<EMAIL_ADDRESS>What do you need?
Hello, I saw your codepen to generate confetti on click: https://codepen.io/matteobruni/pen/wvPvOVg and it's exactly what I try to achieve: in your example, no particles are generating except the ones from the user clicks.
If I try to update the CDN version of tsparticles to 2.0+, it spawns as you can see here: (https://codepen.io/Kalane/pen/GRBVVwq) some particles automatically. I tried to play with all the options we have (autoplay, particles.number.value, ...) but I couldn't make it works.
Thanks a lot!
tsParticles Version
2.0.0 to 2.9.1
Which library are you using?
Vanilla (tsparticles), React.js (react-particles, react-tsparticles, react-particles-js)
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Thanks for the report, I didn't notice this. I've found the first version with this issue, it's the 2.3.0, this will help me find the issue. I'm preparing 2.9.2 and I will include this fix as well.

Thanks a lot for the quick answer!

2.9.2 released with the fix

It seems the problem is the same with the Sea Anemone Preset. The animation starts right away.
Could this be related?

It seems the problem is the same with the Sea Anemone Preset. The animation starts right away.
Could this be related?

Sea Anemone confit is made for starting it at the load, if you want to start it on click you can load it on your event. Otherwise you have to customize the configuration

Sea Anemone config is made for starting it at the load, if you want to start it on click you can load it on your event. Otherwise you have to customize the configuration

Got it, thank you very much!


add node version 6.9 to the CI environment test matrices

Apparently the current CI tests for node version 6 pull down node version 6.10.x , but we've come to the realization last night / this morning that there is an ABI breakage between the V8 headers used between those versions of node.
Let's try to avoid this in the future by also testing node version 6.9 explicitly on both Travis and Appveyor
closes #661

Seems reasonable. However, this should not have occurred in the first place. The ABI is not allowed to break in a minor update of node.js.

As per the discussion in #661 , now I also see that arbitrarily testing for node 6.9.x doesn't win us anything. This breakage was entirely due to electron being non-compliant. Let's not merge this.


vimPlugins.competitest-nvim: init at 2024-01-23

Description of changes
added vim plugin CompetiTest.nvim
https://github.com/xeluxee/competitest.nvim

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

results of nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"
1 package built:
vimPlugins.competitest-nvim

1 package built:
vimPlugins.competitest-nvim


How to change data of a screens exit animation during runtime

How would I add custom data to exit animation of a screen during runtime? like args. For my usecase, I want screen to move towards specific dynamic object while exiting

I have implemented an example of that case in a demo. Please take a look at it.


Document programming with "file stores" (aka object stores or blob stores)

This PR adds a new incomplete document (although it is moderately far along) that

provides a conceptual overview to the two file stores (web and data) that we provide
discusses how to program them from @nimbella/sdk-* in the languages we currently support for that
summarizes the nim command support for them

I've read half way through.

I've read half way through.

I am going to complete the document before merging and will request approval at that time.   I merged the command-docs PR and rebased this one so I can finish up the last section with some xrefs.


use _compute_permutation in distribute_fc2

NOTE: I have not tested the case where atom_list != range(num_pos) since it doesn't seem to me that I can easily test it using input files.  Maybe there should be a unit test that calls phonon.produce_force_constants(calculate_full_force_constants=False)?
Please let me know if this is an issue.
(Just as an aside, my main method of testing my changes are to do bit by bit comparisons of band.yaml.  I make sure to preserve the order of all meaningful floating point operations so that the file produced in the end is identical; I do this simply because it makes it easier to find bugs)

Here's some numbers, doing --writefc on these input files:

before:

total runtime: 217.664s
distribute_force_constants: 63.720s (30% of total)

after:

total runtime: 162.016
distribute_force_constants: 0.491s (negligible!)

Oops, need to fix my declarations to be pre-C99 compatible

Maybe there should be a unit test that calls phonon.produce_force_constants(calculate_full_force_constants=False)?

No. That was made before I realized the existence of unittest. I am an old programmer. But I think it is trivial.
At the first step, force constants are calculated between atoms in primitive cell and atoms in supercell. Therefore the force constants matrix is unnecessary to be filled fully due to translational symmetry. 'fully' means those elements for atoms between supercells. However by numerical noise, the translational symmetry can be broken, and in phonopy there is an option to force the translational symmetry, which requires 'full' force constants matrix. When using a good force calculator, the degree of broken symmetry is negligible often, so we need the former force constants, which corresponds to phonon.produce_force_constants(calculate_full_force_constants=False).
Therefore if we test it, what has to be tested is whether the set of atomic indices contains all atomic indices of atoms in primitive cell or not.

Oops, need to fix my declarations to be pre-C99 compatible

Yes, unfortunately this is necessary to support old Microsoft Visual C(++), which is still used to make conda package with python-2.7.

Your performance gain of distribute_force_constants is considerable. Recent my update is just to improving performance using openMP concurrency. Because your algorithm is quite fast, so my recent update can be ignored.

I don't know well how to use github conversation, so I write it an ugly way below.
https://github.com/atztogo/phonopy/blob/ce40b082bfa6e3a3c7b22dcd48d8abd2e4d89a31/phonopy/harmonic/force_constants.py#L834
    def sort_by_lattice_distance(fracs):
        carts = np.dot(fracs - np.rint(fracs), lattice.T)
        perm = np.argsort(np.sum(carts**2, axis=1))
        sorted_fracs = fracs[perm]
        return perm, sorted_fracs

This sorted_fracs is passed to c code. So I think it's better to write
        sorted_fracs = np.array(fracs[perm], dtype='intc', order='C')

This is a kind of charm. Sometimes numpy just resorting the index order only but not for data in memory. If I am sure this is not the case always, I don't need to do this. order='C' is also redundant, since it seems it is the default parameter. But for multidimensional numpy array, I do this for sure to make the data in memory correct order. The numpy behaviour may depend on its version, too, which I am afraid of. But as far as I do this, I didn't have troubles.
Related to this, I have a small comment on my habit.
https://github.com/ExpHP/phonopy/blob/ba95ab7a003ce13874aee25bf250fc54865559a8/phonopy/harmonic/force_constants.py#L842
out = []

If the array size of out is known before the following loop. I prefer to declare the array before
out = np.zeros((nsym, npos), dtype='intc', order='C')

to help the readability and to make me relax to expect collect memory order.

// in C- code comments is not allowed for old C style compatibility, unfortunately...

Overall, I am ready to merge.

Do you want to remove _get_sym_mappings_from_permutations before merging or after?
Can I modify the numpy array declarations that I wrote above?

By the way, do you hate pointer arithmetic? I grew up with old computers and some machine language, touching memory directly is very usual, but not a good convention.

This is a kind of charm. Sometimes numpy just resorting the index order only but not for data in memory. If I am sure this is not the case always, I don't need to do this. order='C' is also redundant, since it seems it is the default parameter. But for multidimensional numpy array, I do this for sure to make the data in memory correct order.

Oops! Yes, I forgot to consider this, but yes, this is very important here since the C code uses PyArray_DATA directly.  (in other projects I've had the C code call PyArray_ContiguousFromAny, but that only solves part of the problem; it's still possible to forget other important things like dtype=intc)

Do you want to remove _get_sym_mappings_from_permutations before merging or after?

I'll work on it after.

Can I modify the numpy array declarations that I wrote above?

Of course!

If the array size of out is known before the following loop. I prefer to declare the array before to help the readability and to make me relax to expect collect memory order.

Thanks for explaining this reasoning.  I'll try to keep this in mind in the future.
Really, the only reason for my habit of using list and append is to avoid having another index variable (since I'm afraid I might make a typo and use the wrong index somewhere).

By the way, do you hate pointer arithmetic? I grew up with old computers and some machine language, touching memory directly is very usual, but not a good convention.

Heh, what makes you say that?  I definitely don't enjoy it, since there's a lot more room to make mistakes when you have to manually write out strides and offsets all the time. (e.g. an easy accident is to write i * num_rot + j * 9 when you meant i * num_rot * 9 + j * 9).  That said, I'm not exactly alien to it, either. :)

Merged to the master branch. Your code is much well written by me.  Thanks.
On pointer arithmetic, you are correct. I use a lot of it in phono3py. I have to be careful not to make mistake in strides. What I would mean was about this line,
permutation = &permutations[sym_index * num_pos];

I normally write
permutation = permutations + sym_index * num_pos;

As you write in the last line, I have to remember that the point arithmetic is type-dependent with respect to real (though virtual) memory address.

I don't know well how to use github conversation, so I write it an ugly way below.

Some tips: The way I do those comments on the code lines is that I go to the files tab and click on a line of code; it will let you write a comment.  This feature is often useful for requesting changes to PRs.
Also, if you write your code blocks using ```python or ```c instead of by indenting 4 spaces, you get syntax highlighting.
**indented 4 spaces**

    def f(x):
        pass

**better highlighting** (replace ' by `)

'''python
def f(x):
    pass
'''

indented 4 spaces
def f(x):
    pass

better highlighting
def f(x):
    pass

Thanks!

I go to the Changes tab and click on a line of code

What a nice feature!


gocd-server: 16.9.0-4001 -> 19.3.0-8959

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build gocd-server
Pinging the maintainers @grahamc and @swarren83

@GrahamcOfBorg test gocd-server

Any news?

Duplicate of #61309.

@flokli  gocd-agent != gocd-server

oops :-)

@GrahamcOfBorg build gocd-server

@GrahamcOfBorg eval


Add label "API Review Scoping" in GH issues of API Review meeting type

As mentioned in https://github.com/Azure/azure-sdk-tools/issues/3858

@maririos do we need to add it if the issue already exists?

It should always exist, so we need to add it for new issues, and for existing issues make sure it is there

@JonathanCrd is figuring out the permissions for the Azure SDK bot into our testing repository

@JonathanCrd Jonathan Cardenas FTE is figuring out the permissions for the Azure SDK bot into our testing repository

Now we have 2 new testing repos in the bot account, so we should not be blocked by permissions anymore :)

I have modified the current flows involved in the GH Issue creation, I added an optional parameter that expects a list of labels to be included on the issues. I also modified the GitHub - Close - issue flow to add a label when reopening an issue (if label is given).
This way, this is not a breaking change for the apps, they should be working as normal.
Next step:

[ ] Adding the Label parameter in the API Reviews App (Blocked until we finished working on the Daylight Savings bug)


Documented example of using a database router

As ...
Patti - Platform Admin
I want ...
A documented solution for implementing a database router
So that ...
Nautobot can route RO SQL queries to a read replica and RW SQL queries to a write instance
I know this is done when...

Documented support for database routers
ability to route RO and RW queries to separate db instances

Optional - Feature groups this request pertains to.

[ ] Automation
[ ] Circuits
[ ] DCIM
[ ] IPAM
[ ] Misc (including Data Sources)
[ ] Organization
[ ] Plugins (and other Extensibility)
[ ] Security (Secrets, etc)
[ ] Image Management
[ ] UI/UX
[X] Documentation
[ ] Other (not directly a platform feature)

Database Changes
None
External Dependencies
None

This works today but #4393 is a caveat, setting CHANGELOG_RETENTION statically in nautobot_config solves the issue with recursion but negates the ability to configure that setting via the UI in constance config

Do we really need to document this, or can we just make a reference to the official Django documentation on the topic?

Because we have some hard coded dependencies and the implications of certain configuration types is different enough to warrant us documenting.

Because we have some hard coded dependencies and the implications of certain configuration types is different enough to warrant us documenting.

Are you speaking of the Dolt integration? I'm not aware of other hard-coded dependencies you're referencing. Any context you can provide would super helpful! Thank you.

DATABASES setting MUST have a default database. We aren't supporting separate databases & models that live in separate databases. We currently have a recursive issue with CHANGELOG_RETENTION + constants config when attempting to implement a DB router.

DATABASES setting MUST have a default database. We aren't supporting separate databases & models that live in separate databases. We currently have a recursive issue with CHANGELOG_RETENTION + constants config when attempting to implement a DB router.

Oh boy, I'd love to understand that issue! Anything you can share to help get to the bottom of it?


llama70b one card to infer device map with max memory limitation

What does this PR do?

Fixes # (issue)
Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[ ] Did you make sure to update the documentation with your changes?
[ ] Did you write any new necessary tests?

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

LGTM! Can you also share a command that I can execute to test it please? You will also need to run
pip install -U ruff
make style

from the root of the repo to make the code style check pass.

@regisss fixed the formation. The disk_offload is tested nightly


[Dribbblish] Sidebar folders invisible

Describe the bug
Folders aren't showing up in the sidebar. I have tried both the automatic install and the manual. I have also tried reinstalling spicetify. Am also using fullAppDisplay and popupLyrics, do not know if the bug could be due to some interaction.
To Reproduce
Steps to reproduce the behavior:

Open Spotify
Playlist folder names visible for a fraction of a section before...
They disappear, showing what is visible in the screenshot below.

Expected behavior
I expect folder names to be visible.
Screenshots
If applicable, add screenshots to help explain your problem.

Logs

DevTools failed to load source map: Could not load content for https://xpui.app.spotify.com/vendor~xpui.js.map: Connection error: net::ERR_NAME_NOT_RESOLVED
DevTools failed to load source map: Could not load content for https://xpui.app.spotify.com/xpui.js.map: Connection error: net::ERR_NAME_NOT_RESOLVED
3fluent.js:77 Uncaught TypeError: Cannot read properties of undefined (reading 'Translations')
    at fluent.js:77
(anonymous) @ fluent.js:77
setInterval (async)
fluent @ fluent.js:76
(anonymous) @ fluent.js:100
vendor~xpui.js:29 MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 change listeners added.  My apologies!


Regression CentralPackageVersions/2.0.29+ no longer works by default in projects with nuproj extension

Note: the repro provided here is a minified repro from a real world use case.
Migrating a CoreXT nuproj (not to be confused with the public package nuget.org/nuproj) project to retail toolset. To do so, we use Sdk=Microsoft.Build.NoTargets when the .nuproj is being compiled under retail toolset.
RCA
It appears that Microsoft.Build.CentralPackageVersions is trying to detect when it's used with nuget.org/nuproj based on the project file extension. However, that's not a reliable way to detect project type.
Recommended fix: condition on the NuProjPath property. Note that since property order is difficult to maintain with Sdk style projects, it's probably best not to check this setting until an early build target.
Repro Steps

git clone https://github.com/aaronla-ms/bug-repros.git
cd bug-repros\repro-cpv-regression
repro.cmd

expected
both restores pass
actual
restore under 2.0.29 and later fails,

$ repro
Microsoft.Build.CentralPackageVersions/2.0.26
Determining projects to restore...
Restored C:\repos\aaronla-ms\bug-repros\repro-cpv-regression\repro-cpv-regression.nuproj (in 340 ms).
Microsoft.Build.CentralPackageVersions/2.0.29
Determining projects to restore...
C:\repos\aaronla-ms\bug-repros\repro-cpv-regression\repro-cpv-regression2.nuproj : error NU1604: Project dependency Newtonsoft.Json does not contain an inclusive lower bound. Include a lower bound in the dependency version to ensure consistent restore results.
C:\repos\aaronla-ms\bug-repros\repro-cpv-regression\repro-cpv-regression2.nuproj : warning NU1701: Package 'Newtonsoft.Json 3.5.8' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
Failed to restore C:\repos\aaronla-ms\bug-repros\repro-cpv-regression\repro-cpv-regression2.nuproj (in 368 ms).

I think NuProj has established itself as the implementation of .nuproj projects.  If you want your NoTargets projects to load in Visual Studio, they'll need to have a .csproj or .msbuildproj extension anyway.
I think you should do one of the following:

Change the file extension of your projects so they load in Visual Studio and you get CentralPackageVersions
Set EnableCentralPackageVersions to true in your project/Directory.Build.props

We can't check properties during build because a build isn't run in Visual Studio for legacy projects so everything must be set up correctly during evaluation.

Ok. We don't really have any need to update this package for now, so I guess we'll stay on 2.0.26 for now.

Yeah, I just confirmed that our proprietary corext/nuproj package makes reliance on project extensions as well:

  <_NuProjProjectReference Include="@(ProjectReference)" Condition="'%(ProjectReference.Extension)' == '.nuproj' AND '%(ProjectReference.ReferenceOutputAssembly)' != 'false'" />

Will stick with 2.0.26 for now until either issue is fixed or we've moved fully to retail toolset.

Thankfully we can override this detection. Will add this to our Directory.Build.targets before we bring in CPV:
<!-- workaround issue https://github.com/microsoft/MSBuildSdks/issues/250 -->
<PropertyGroup>
  <EnableCentralPackageVersions>true</EnableCentralPackageVersions>
</PropertyGroup>

Closing as by design due to https://github.com/microsoft/MSBuildSdks/pull/84.  Workaround is to enable it manually or use a different file extension.


Full hits on string is not included in result with the Levenshtein-driver

Hello
First of all - thank you for creating this package. It's a really good addition to Laravel.
I found an issue with the Levenshtein-driver, but keep in mind that I might be mistaken. It might still be worth looking into.
The problem is - when you use the Levenshtein-driver - that the matches that has 0 in "distance" from the search string is not included in the query. This is due to the following code:
src/SearchDrivers/BaseSearchDriver.php:128 (version 2.0.19)
return $this->query
            ->orderBy($this->relevanceFieldName, 'desc')
            ->having($this->relevanceFieldName, '>', 0);

The having-clause causes the full match to be exlcuded in this case since the distance (stored in the relevance-column) is 0. In my opinion the having-clause should only be applied to the other drivers except the Levenshtein-driver. Is this something that maybe should be changed? And is this project open for PR's?
Kind regards,
Robert S.

Please see Please see https://github.com/TomLingham/Laravel-Searchy/issues/117


Fail to convert iApp-created VS to AS3

Bug description
Journeys reports 'JSON.parse: unexpected character at line 1 column 1 of the JSON data' for the configuration created by f5.http.v1.2.0 iApp, where Application Service Name is: app1-template
To Reproduce
Steps to reproduce the behavior:

Upload f5.http.v1.2.0 iApp template to the BigIP system
Create an Application Service named app1-template.
Dump an UCS and upload it to Journeys at the Application Service Migration section
In the Applications steps, unfold tenant_2 and app1-template.app_1
you should see the error

Expected behavior
Journeys should parse the configuration without an error
Screenshots

Context

Journeys version: f5devcentral/f5-bigip-journeys-app:v3.0.1
BigIP 15.1.3

Filed internal Jira SOLUTIONS-4225.


Move console commands

Right now (for legacy reasons) console commands live in the Tempest\Console\Commands namespace that can be moved elsewhere (e.g., MigrateUpCommand).
We should move these to a Console directory under their respective components. At that point, we need to decide whether it makes most sense to actually require the console component in the composer file or move attributes to a global namespace or do something entirely different.

@brendt - I'd welcome your feedback here.

it makes most sense to actually require the console component in the composer file or move attributes to a global namespace or do something entirely different.

I don't fully get this point, I'm afraid. But I get the problem with namespace collisions. I want to avoid having a Commands namespace, because that's confusing with the commandbus implementation.
I think we'd need something like this? Tempest\Framework\ConsoleCommands\… Would that be possible?

@brendt - my recommendation was to move them to the namespace they are related to. For example, migration commands might be in Tempest\Database\Console.
The only problem I can see with this is that you could include those commands without requiring the console package as a dependency for that package, but we still have to have the attributes somehow. Do we move attributes to a common namespace similar to how Laravel has a Contracts namespace?

Well this is one of those things that had me worried about too many components :p
But, no problem, I think it's pretty clear to me — my vision, at least — component packages should be truly standalone (tempest/database shouldn't have a dependency on tempest/console). That also means though that this package cannot contain any console commands. It's the framework package that brings everything together, and thus adds commands for many components. I'd like to follow that structure for now. We can always move console commands to a package "down the line" without it being a breaking change, but we cannot move them "up" anymore. So let's keep console commands in "the framework" for now, and see how far we can get with that.
Does that sound reasonable?

@brendt that sounds totally reasonable and probably better practice. Does it make sense to keep all these framework specific things confined to a namespace like Laravel does with the Foundation namespace?

Yes, I thought to call that namespace Framework

All framework specific commands are now in the Framework namespace


ValueError: Unsupported reduction type None

I've faced with this issue and still stuck at this.
Can you help to fix? Thanks a lot
  File "/home/ocr/anaconda3/envs/pika/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ocr/anaconda3/envs/pika/lib/python3.6/site-packages/torch_baidu_ctc/__init__.py", line 167, in forward
    blank=self.blank,
  File "/home/ocr/anaconda3/envs/pika/lib/python3.6/site-packages/torch_baidu_ctc/__init__.py", line 117, in ctc_loss
    raise ValueError("Unsupported reduction type {!r}".format(reduction))
ValueError: Unsupported reduction type None

Fixed with 45356ef3e90f6cdf5a280db514f075d023eb93c5. I'm not generating new packages for PyTorch 1.0, though. I will create a new version with PyTorch 1.1, the new stable version.


Image Captioning example in keras Approach?

I have used the keras example code of Image Captioning in that I have used the VGG pretrained model for extracting image features(4096) and for text part I have done indexing to the unique words and post zero padding according the max caption length(which is equal the length of biggest sentence in data) and for next words I created a numpy array of (number of example, vocabulary size) vocabulary size is equal to number of unique words in data. The next words is a 1's 0's matrix 1 means present of word in the sentence 0 absence.
for predicting what should be given exactly in partial caption and in next words if we consider "cat sat on mat".
And in training data I am appending "START" and "END" tokens in the start and end of training captions.
here is my code:
correct me if I am doing wrong in code
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation, Flatten
from keras.layers.convolutional import Convolution2D, MaxPooling2D, ZeroPadding2D
from keras.optimizers import SGD, Adadelta, Adagrad
from keras.utils import np_utils, generic_utils
from keras.callbacks import EarlyStopping
from keras.layers.advanced_activations import PReLU, LeakyReLU
from keras.layers import Embedding,GRU,TimeDistributedDense,RepeatVector,Merge
from keras.preprocessing.text import one_hot
from keras.preprocessing import sequence
import cv2
import numpy as np

max_caption_len = 21
vocab_size = 43
def VGG_16(weights_path=None):
    model = Sequential()
    model.add(ZeroPadding2D((1,1),input_shape=(3,224,224)))
    model.add(Convolution2D(64, 3, 3, activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(64, 3, 3, activation='relu'))
    model.add(MaxPooling2D((2,2), strides=(2,2)))

    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(128, 3, 3, activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(128, 3, 3, activation='relu'))
    model.add(MaxPooling2D((2,2), strides=(2,2)))

    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(256, 3, 3, activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(256, 3, 3, activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(256, 3, 3, activation='relu'))
    model.add(MaxPooling2D((2,2), strides=(2,2)))

    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(512, 3, 3, activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(512, 3, 3, activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(512, 3, 3, activation='relu'))
    model.add(MaxPooling2D((2,2), strides=(2,2)))

    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(512, 3, 3, activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(512, 3, 3, activation='relu'))
    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(512, 3, 3, activation='relu'))
    model.add(MaxPooling2D((2,2), strides=(2,2)))

    model.add(Flatten())
    model.add(Dense(4096, activation='relu'))
    model.add(Dropout(0.5))
    model.add(Dense(4096, activation='relu'))
    model.add(Dropout(0.5))
    model.add(Dense(1000, activation='softmax'))

    if weights_path:
        model.load_weights(weights_path)

    #Remove the last two layers to get the 4096D activations
    model.layers.pop()
    model.layers.pop()

    return model
print "VGG loading"
image_model = VGG_16('vgg16_weights.h5')
image_model.trainable = False
print "VGG loaded"
# let's load the weights from a save file.
# image_model.load_weights('weight_file.h5')

# next, let's define a RNN model that encodes sequences of words
# into sequences of 128-dimensional word vectors.
print "Text model loading"
language_model = Sequential()
language_model.add(Embedding(vocab_size, 256, input_length=max_caption_len))
language_model.add(GRU(output_dim=128, return_sequences=True))
language_model.add(TimeDistributedDense(128))
print "Text model loaded"
# let's repeat the image vector to turn it into a sequence.
print "Repeat model loading"
image_model.add(RepeatVector(max_caption_len))
print "Repeat model loaded"
# the output of both models will be tensors of shape (samples, max_caption_len, 128).
# let's concatenate these 2 vector sequences.
print "Merging"
model = Sequential()
model.add(Merge([image_model, language_model], mode='concat', concat_axis=-1))
# let's encode this vector sequence into a single vector
model.add(GRU(256, return_sequences=False))
# which will be used to compute a probability
# distribution over what the next word in the caption should be!
model.add(Dense(vocab_size))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy', optimizer='rmsprop')
print "Merged"
# "images" is a numpy float array of shape (nb_samples, nb_channels=3, width, height).
# "captions" is a numpy integer array of shape (nb_samples, max_caption_len)
# containing word index sequences representing partial captions.
# "next_words" is a numpy float array of shape (nb_samples, vocab_size)
# containing a categorical encoding (0s and 1s) of the next word in the corresponding
# partial caption.
print "Data preprocessig"
Texts = ["START A girl is stretched out in shallow water END",
        "START The two people stand by a body of water and in front of bushes in fall END",
        "START A blonde horse and a blonde girl in a black sweatshirt are staring at a fire in a barrel END",
        "START Children sit and watch the fish moving in the pond END",
        "START A fisherman fishes at the bank of a foggy river END"]

Images = ["667626_18933d713e.jpg",
         "3637013_c675de7705.jpg",
         "10815824_2997e03d76.jpg",
         "12830823_87d2654e31.jpg",
         "17273391_55cfc7d3d4.jpg"]
images = []
for image in Images:
    img = cv2.imread(image)
    img.resize((3,224,224))
    images.append(img)
images = np.asarray(images)

words = [txt.split() for txt in Texts]
unique = []
for word in words:
    unique.extend(word)
unique = list(set(unique))
word_index = {}
index_word = {}
for i,word in enumerate(unique):
    word_index[word] = i
    index_word[i] = word

partial_captions = []
for text in Texts:
    one = [word_index[txt] for txt in text.split()]
    partial_captions.append(one)

partial_captions = sequence.pad_sequences(partial_captions, maxlen=max_caption_len,padding='post')
next_words = np.zeros((5,vocab_size))
for i,text in enumerate(Texts):
    text = text.split()
    x = [word_index[txt] for txt in text]
    x = np.asarray(x)
    next_words[i,x] = 1

print "Data preprocessing done"
model.fit([images, partial_captions], next_words, batch_size=1, nb_epoch=5)

for this task i am taking only 5 example to know the working of this model and also I want to know whether my approach is right or wrong. Afterwords I am going to use flickr8k dataset for the same

@bhaveshoswal  what is the dataset which you are using ?

@bhaveshoswal  @shashankg7 when i run this code i get a error "All input arrays and the target array must have the same number of samples."  Can you please help on the same ?

You have take any five images and five questions on it as i have taken in Texts and Images variable then code will run and make all images size to (224,224)

@bhaveshoswal  hi thanks i was able to resolve it. did u tried ur code on flickr data ?
in that there are 5 captions for one image , how are you trying to handle that?  have u completed training on flickr data ?

I Tried on 5 images and their 5 captions from flicker data

@bhaveshoswal @deepnarainsingh if you are still trying to implement this type of model in Keras (without the ability to finetune the ConvNet), I have a working implementation in this repository.

@bhaveshoswal thanks for your sharing, how to handle real flickr data? are you used data_generator? please explain me, i need that. regards.

@kucingit3m not using data_generator just reshape the image size to (224,224) and do embedding on caption for five caption repeat the image vector five times for it in training data hope thats helps you.

@bhaveshoswal thanks! thats help me. but why my loss model always getting up every epoch, did you ever experienced that?

@bhaveshoswal  thanks for the explanation. I have given the input just as you explained above but my loss kept on increasing for every epoch. Did you face such issue while training ?
Also could you also explain how to give the test data. I mean there is only a image present for which we need to get the caption. So how do we give the data for testing ?
Thanks in advance.

@elliottd thanks for your implementation

@kucingit3m that's same with me

@indra215 you have to give two inputs first image of size (224,224) and other partial caption for example [START A] according to the image you give

@bhaveshoswal  thanks for the reply. I've followed your example code above (on a much larger dataset) where in you used the entire caption in the partial_captions and in the next_words, you gave the 0 and 1 encoding of the words present in the partial_captions. But where is this partial caption meaning coming in the data ? I mean you are giving the entire caption in the partial_captions.
Did you successfully train this model and generated captions on some real data ? If so could you please help me in the code on how to give the input data for training the model.
Also how to give the data while testing on a new image where we don't have any partial_caption there.
Thank you in advance.

@bhaveshoswal I have trained the model as you have described above without any problems. But I am facing the problem of loading the model using load_model() after training it. My issue can be seen in detail here: https://github.com/fchollet/keras/issues/5442 . Have you had a similar problem after training the model? If yes then how did you solve it?
Thank you in advance.

Hi, it seems the image captioning example has gone. @bhaveshoswal Did you finish your codes?

where is the image caption code? I just git cloned the keras code, and there is nothing in there for captioning.

Hi @fchollet , could you please help by adding the Image Caption Generation code. Your help would be highly appreciated.

@junyongyou @bhomass
This might be of your help https://github.com/anuragmishracse/caption_generator .

The latest version of the captioning example can be found here. It's an older commit (8 March 2017) of the keras repo.

Why was it removed from the set of examples?
On Mon, Jun 26, 2017 at 12:11 PM, Pavlos<EMAIL_ADDRESS>wrote:

The latest version of the captioning example can be found here
https://github.com/fchollet/keras/blob/0b2c044d48a335e97ffef4b6c76031fa12627ec9/docs/templates/getting-started/sequential-model-guide.md#architecture-for-learning-image-captions-with-a-convnet-and-a-gated-recurrent-unit.
It's an older commit (8 March 2017) of the keras repo.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/2295#issuecomment-311005030,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFdLCHeNfetGEjS0RIuQM3mMFCFcn9TAks5sH3XPgaJpZM4IGGhT
.

Not sure... It works with keras 2 with minor modifications so I don't see a practical reason. Maybe the application was just too niche?

Hello, if someone is interested you can find an image captioning in keras 2.0 here

thanks!

OSError: Unable to open file (Unable to open file: name = 'vgg16_weights.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)
getting this error after writing the convolution layers.any new updated code for this example. please send provide the link for practise

@SJameer You can use vgg16 in keras.applications, here: https://keras.io/applications/#vgg16

@anuragmishracse - how much time does it take for the language model ?
My program is running for past 2 hrs on a GPU, still no result .....

@anuragmishracse - how much time does it take for the language model ?
My program is running for past 2 hrs on a GPU, still no result .....

@anuragmishracse - how much time does it take for the language model ?
My program is running for past 2 hrs on a GPU, still no result .....


[SDP-1209] Validate TENANT_XLM_BOOTSTRAP_AMOUNT on server startup

What
Validate TENANT_XLM_BOOTSTRAP_AMOUNT on server startup
Why
If a server is misconfigured, it needs to be surfaced on server launch, rather than leaving the error to be triggered and debugged by a host at a later time.
Address https://stellarorg.atlassian.net/browse/SDP-1209.
Checklist
PR Structure

[x] This PR has a reasonably narrow scope (if not, break it down into smaller PRs).
[x] This PR title and description are clear enough for anyone to review it.
[x] This PR does not mix refactoring changes with feature changes (split into two PRs otherwise).

Thoroughness

[x] This PR adds tests for the new functionality or fixes.
[x] This PR contains the link to the Jira ticket it addresses.

Configs and Secrets

[x] No new CONFIG variables are required -OR- the new required ones were added to the helmchart's values.yaml file.
[x] No new CONFIG variables are required -OR- the new required ones were added to the deployments (pr-preview, dev, demo, prd).
[x] No new SECRETS variables are required -OR- the new required ones were mentioned in the helmchart's values.yaml file.
[x] No new SECRETS variables are required -OR- the new required ones were added to the deployments (pr-preview secrets, dev secrets, demo secrets, prd secrets).

Release

[x] This is not a breaking change.
[x] This is ready for production.. If your PR is not ready for production, please consider opening additional complementary PRs using this one as the base. Only merge this into develop or main after it's ready for production!

Deployment

[x] Does the deployment work after merging?

stellar-disbursement-platform-backend-preview is available here:SDP:      https://sdp-backend-pr291.previews.kube001.services.stellar-ops.com/healthAP:       https://sdp-ap-pr291.previews.kube001.services.stellar-ops.com/healthFrontend: https://sdp-backend-dashboard-pr291.previews.kube001.services.stellar-ops.com

stellar-disbursement-platform-backend-preview is available here:SDP:      https://sdp-backend-pr291.previews.kube001.services.stellar-ops.com/healthAP:       https://sdp-ap-pr291.previews.kube001.services.stellar-ops.com/healthFrontend: https://sdp-backend-dashboard-pr291.previews.kube001.services.stellar-ops.com

stellar-disbursement-platform-backend-preview is available here:SDP:      https://sdp-backend-pr291.previews.kube001.services.stellar-ops.com/healthAP:       https://sdp-ap-pr291.previews.kube001.services.stellar-ops.com/healthFrontend: https://sdp-backend-dashboard-pr291.previews.kube001.services.stellar-ops.com


test: Destroy path

Description
Undeploy the stack - verify that ALL resources are deleted (check in resource controller)
Context: https://github.com/terraform-ibm-modules/stack-retrieval-augmented-generation/issues/17
New or affected modules

By submitting this issue, you agree to follow our Code of Conduct

There any two approaches to undeploy the stack and delete all the resources:-

Delete the project and it will undeploy all the resources associated with the project before deleting.
Undeploy the stack one by one through the UI.


validate data before sending it to the node

we need to validate the user has entered a reasonable data before sending it to the node for example
if the user entered cpu = 100000 this should fail from the begining
also validate that all names are in lower case

verified


标点预测服务报错

[2023-04-20 11:00:04,503] [   ERROR] - Call punctuation http://xx.xxx.xx.xx:8190/paddlespeech/text occurs error
[2023-04-20 11:00:04,503] [   ERROR] - HTTPConnectionPool(host='<IP_ADDRESS>', port=8080): Max retries exceeded with url: http://<IP_ADDRESS>:8190/paddlespeech/text (Caused by ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response')))
测试
[   ERROR] - Failed to Text punctuation.

<IP_ADDRESS>可以ping通


Updated for Scrapy v1.0 API changes

works in production without raising deprecation warnings
Request Queue test fails, appears to be related to Scrapy v1 (?)

Test output: https://gist.github.com/fraserharris/df51b43184c8c35620cd

Thanks for the PR.
E       assert [] == [call('http://testdomain/pat...omain/path?wr=1&wa=1'), ...]
E         Right contains more items, first extra item: call('http://testdomain/path?wr=0')
E         Use -v to get the full diff

Same here. Don't have time to dig more.


[WIP] initial guni open vault logic, extracted common componets

Guni UI implementation

Changes 👷♀

- extracted common components
- added guni support
How to test 🧪

- test on UI using hardhat
Definition of done ✔

[ ] Acceptance criteria for each issue met
[ ] Unit tests written where needed and passing
[ ] End-to-end tests for happy path
[ ] Documentation updated
[ ] Project builds without errors
[ ] Non-functional requirements met
[ ] Code reviewed and functionality tested by the reviewer (locally, Heroku, etc.)
[ ] Feature verified and accepted by Product Owner
[ ] Project deployed to production environment

Not relevant anymore. We will try to approach refactor in different way.


(UI Bug) Deployment progress modal looks chaotic when there are six or more tiers

If you are reporting a problem with Token Wizard, please include the following information:
Which network did you use? (Mainnet, Kovan, Rinkeby, etc.)
Rinkeby
If you were able to create it, what is the URL of your crowdsale?
https://wizard.poa.network/invest?addr=0x1409a40C3bf91FD6909b902eBC37f9C657610bb4&networkID=4
Do you have screenshots showing the problem?
6 tiers

10 tiers

Do you see errors in the dev console? If yes, please include a screenshot
To open the dev console in Google Chrome, press F12, or go to View -> Developer -> Developer Tools, and open the Console tab
If you see errors, please right click on them and "Save as..". Zip saved file and attach it to the Issue.
No

Duplicate of https://github.com/poanetwork/token-wizard/issues/518


Add Event.getAsMap() function to return the Event as a Map<Object, Object>

Signed-off-by: Taylor Gray<EMAIL_ADDRESS>Description

Adds a function getAsMap() to the Event interface
Implements this function in JacksonEvent and returns the Event as a Map<Object, Object> using  ObjectMapper

Check List

[x] New functionality includes testing.
[x] New functionality has been documented.

[x] New functionality has javadoc added

[x] Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #848 (a82c2d2) into main (ac9e452) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##               main     #848   +/-   ##
=========================================
  Coverage     92.21%   92.21%
- Complexity      659      660    +1
=========================================
  Files            80       80
  Lines          1939     1940    +1
  Branches        164      164
=========================================
+ Hits           1788     1789    +1
  Misses          110      110
  Partials         41       41

Impacted Files
Coverage Δ

...m/amazon/dataprepper/model/event/JacksonEvent.java
99.20% <100.00%> (+<0.01%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ac9e452...a82c2d2. Read the comment docs.


Streaming response returns 'StuffDocumentsChain' not the stream

When calling StreamingResponse.from_chain, stream is not returned, instead 'StuffDocumentsChain' str is returned.
This code works as intended when calling normally without streaming.
Motorhead memory was omitted to simplify code.
Scenario
@app.get("/chat", response_model=ChatResponse)
async def chat_endpoint(
    sessionID: str, question: str, model: str, aimodel: str, engine: str
):
    chain, inputs = await answerQuestion(
        sessionID, question, model, aimodel
    )

    return StreamingResponse.from_chain(chain, inputs, media_type="text/event-stream")

async def answerQuestion(sessionID: str, question: str, model: str, ai_model: str):
    key = (
        os.environ.get("OPEN_API_KEY_GPT4")
        if ai_model == "gpt-4"
        else os.environ.get("OPEN_API_KEY")
    )

    namespace = "05a77398-8e25-49de-9ba1-3338366f2386"
    vectorstore = load_pinecone_existing_index(namespace, model)
    llm = OpenAI(
        openai_api_key=key,
        temperature=0.125,
        model_name=ai_model,
        verbose=True,
        streaming=True,
    )

    retriever = SelfQueryRetriever.from_llm(
        llm,
        vectorstore,
        document_content_description,
        metadata_field_info,
        verbose=True,
    )

    template = """Some template...

    {context}

    {chat_history}
    Zákazník: {human_input}
    Chatbot:"""

    docs = retriever.get_relevant_documents(question)

    llm = ChatOpenAI(
        model_name=ai_model,
        streaming=True,
        verbose=True,
        temperature=0.125,
        max_tokens=600,
        openai_api_key=key,
    )

    prompt = PromptTemplate(
        input_variables=["chat_history", "human_input", "context"], template=template
    )

    chain = load_qa_chain(
        llm=llm,
        document_variable_name="context",
        chain_type="stuff",
        prompt=prompt,
        verbose=True,
    )

    inputs = {
        "input_documents": docs,
        "human_input": question,
    }

    return chain, inputs

Actual result
'StuffDocumentsChain'

Expected result
Stream is returned. For example case, from Read.me it works
Acceptance criteria
Endoint should return stream.

@gabriel-ecegi ah! i understand the problem here. this particular chain type is not yet supported and throws a KeyError which gets streamed. There's an issue related to this: https://github.com/ajndkr/lanarky/issues/35. I plan on adding a docs page for this project ASAP.
In the meantime, I will improve the error message as the above is indeed very confusing.

Thank you, I will be looking forward to this feature.
So if I understand correctly, I can make it work by adding
`@register_streaming_callback("StuffDocumentsChain")
class AsyncStuffDocumentsChainStreamingCallback(AsyncStreamingResponseCallback):
"""AsyncStreamingResponseCallback handler for StuffDocumentsChain."""
pass

`
I have added it, now it does not throw any arror, but also does not return anything.

yup! but it looks like the callback doesn't work for StuffDocumentsChain. can you try by inheriting AsyncBaseRetrievalQAStreamingCallback instead of AsyncStreamingResponseCallback?
you can do the following:
from lanarky.callbacks import AsyncRetrievalQAStreamingCallback

@register_streaming_callback("StuffDocumentsChain")
class AsyncStuffDocumentsChainStreamingCallback(AsyncStreamingResponseCallback):
    """AsyncStreamingResponseCallback handler for StuffDocumentsChain."""

    pass

let me know if it works!

Thank you, it works now.


WIP: [#133533671]  Upgrade rds-broker with new dependencies

#133533671 Update RDS broker dependencies.
What?
Include changes from https://github.com/alphagov/paas-rds-broker/pull/39
WIP: Update after PR is merged
Dependencies

paas-rds-broker PR (must be reviewed, merged and update this one) https://github.com/alphagov/paas-rds-broker/pull/39
paas-cf PR: TBD

Update  the paas-cf after this is merged and built in build ci.
Who?
Anyone but @keymon

Bumped and merged.


conda-forge feedstock

Tasks
We need a conda-forge feedstock from PyPi

Packages are now available on conda-forge
https://anaconda.org/conda-forge/spyder-remote-client
https://anaconda.org/conda-forge/spyder-remote-server


Postgis 0.11

Creates a PostGIS data reader to convert road networks located in PostGIS directly into GraphHopper's native network format. Please see the README.postgis for more information.

Thanks a lot, this is a valuable addition to the GraphHopper ecosystem! Probably many users have waited for it. Please do not hesitate to post it on our forum too.
Still, I think, this should be better reside in a separate repository as e.g. we won't be able to include it in web-bundle per default (too many dependencies that most of the users won't need, would be pulled in) and also maintenance needs to be guaranteed once included.
With a similar reasoning we are also planning to move reader-shp out of the repository as this is a similar module and no one from the community seems to be interested (and we core devs do not use it).
Please try to do this in a separate module like we are doing it for map-matching and our navigation module. E.g. if you still like to use the same graphhopper.sh script file then this should already work using the --jar option.
Wiring the modules together to get a running web server is trivial as this example in the map matching module shows.
Please let us know if you need assistance in this process and if changes to the GH core are required. Will let it open until we are sure it works for you :)

To be honest, I don't understand why someone needs a PostGIS import for GraphHopper.
This pull requests adds a PostGIS reader but it can only read PostGIS tables which have been imported from Geofabrik's data free shape files. Geofabrik's free shape files are not designed to be used for routing applications. As you have observed, roads are not split at intersections.
It is faster to use the OSM raw data import of GraphHopper (avoids database related overhead). I can imagine one benefit only – a reduced memory consumption because the expensive caching of node locations happens on a Geofabrik server where the shape file is produced. However, someone who suffers this issue will not become happy if he/she tries to read a whole planet from a PostGIS database. Not to mention that Geofabrik's free shape files are not available for continents or the whole world.
I think that it is not a good idea to merge code into master which relies on a proprietary product of a company which can be replaced by a free and open source solution which is already supported by GraphHopper. I agree with @karussell that it should be in a separate project using GraphHopper.
@mbasa  Or is there any use case which cannot be solved without a PostGIS import? I am happy to give suggest alternative solutions.

@Nakaner I see one clear benefit here: the ability to enrich/change data (osm tags) in postgis easily which can be used in the street network.

@Nakaner I see one clear benefit here

Good idea. But this can/should happen in a separate repository too as readers are pluggable.

Yep, I agree @karussell - separate repository makes sense for this.

This came about when we had a project that needed the very detailed Digital Road Map (DRM) data of Japan for GraphHopper usage. The data was already in PostGIS and converting the ~20 million road data into OSM was very painful due to the way the OSM data is structured. We also had to use the DRM's attributes when determining the cost in the route creation.
So the benefit of a PostGIS reader for us is:

ability to use non-OSM data (i.e. Navteq) easily with GraphHopper
ability to have dynamic cost for route creation (i.e. Traffic data)

I blogged about this here:
https://georepublic.info/en/blog/2018/graphhopper-with-postgis-data-reader/
And I agree with everybody that this can be a separate repository. This was beneficial for us and it was hoped that it can be useful for others as well.

Thanks for the details. Please open a new issue if you stumble over problems when externalizing this.

Thanks for the great work @mbasa. I am currently working on reading data from PostGIS and create a GH graph from this. I took the liberty to transform @mbasa’s fork into a GraphHopper module, update to the latest version of GraphHopper and extending some of the functions, fix some minor issues, etc.
You can find the current state here, contributions are welcome :slight_smile: :
https://github.com/boldtrn/graphhopper-postgis


Create consistent icons rules for cupcake site

User Story

As a Cupcake website content creator

I need to: clear consistent icon labels on the cupcake site

so that I can: correctly navigate throughout the site.

Additional information
I am working on icon rules to bring consistency to the resource cards on the cupcake site.
Acceptance criteria

[ ] Reviewed and approved by Wonil
[ ] Discussed with Carbon team

Currently we don't have much consistency with our icons on the cupcake site. Almost all the resource acrd icons are launch.

After auditing the Carbon site, it looks like their icons are consistent either. Almost all icons are 'launch' and only one 'download' for data viz.

Here is my suggestion –
rightArrow icon: links within the cupcake site
launch icon: Leaving the cupcake site
download icon: anything that downloads from Sketch cloud
disable icon: any card that is disabled


30 create new user form user can provide personal data

fixes #30

approved

@Tetsux3 please review this again, i think you might have missed something


Need a tip:   pugi::xml_parse_result result = doc.load_file(filename.c_str() relative path is not working

What is missing?
Original path on windows: "D:/dev/projects/Debug/Resources/maps/map_start.xml"
full path is working:
doc.load_file("D:/dev/projects/Debug/Resources/maps/map_start.xml")
relative path is not working !!!!
doc.load_file("maps/map_start.xml")
This kind of reative path is working (why not above?):
auto parentItems = SpriteBatchNode::create("graphics/items.png", 100);

Because engine-x has default search path to Resources on windows

pugixml no serach path, so the relative should based on working ditectory, not Resources

thanks


Spike: Back End work with Onix for Billing

Related to #1192, this ticket represents the back end work that is necessary to record the CO data, and establishes where we store it.

Draft of information/summary/decisions to date: https://docs.google.com/document/d/1GpKa2DwHB3noPym5-AfBAOCiFXpk5J3mgKWpqy3bcIo

Moved the write-up to https://docs.google.com/document/d/1MVagpXQMLtR15fKzom11nw3lR-i4NWImk5H2iERK-t0 which is a shared document between us and Onix. Conversations will be ongoing. This concludes the spike of initial planning with Onix.


fix save as

The safe file dialog did not return path with file extension in the build.
#861mdcbhu

Task linked: CU-861mdcbhu Fix: Scenes can sometimes not be saved in the build


A restore does not move namespaces back to the original project in the cluster

On master-head - commit id: 731b33e1a776, backup restore operator tag: v0.0.1-rc9

In a HA setup, Deploy a custom cluster --> create a project --> create a namespace --> deploy a workload in this namespace.
Take a backup b1 (install backup restore operator in the
Move the namespace to a different project say test2
restore from b1.
The namespace is still seen in project test2

Expected:
Namespace be moved back to the original project

@sowmyav27 although the namespace denoting a project is created in the management cluster, the actual kubernetes namespaces created in the project live in the downstream user cluster.
So if you create project test1 in your custom cluster, and create a namespace ns1 in it, the namespace denoting test1 will be in your local cluster, but the namespace ns1 is in the downstream user cluster. Backup-restore-operator only performs actions on the cluster it is installed on, which is the local cluster.
As you know the label field.cattle.io/projectId on a namespace determines which project it is in. Restore doesn't actually change this label because the namespace ns1 is in the downstream cluster.
The same use case should work if the projects were in the local cluster. Not sure if this is a common use case, of creating projects in the mgmt cluster, but if it is, then the same set of steps work correctly in the mgmt cluster

@mrajashree This will not work on a local cluster unless you give in the particular namespace details in the resourceset to backup. As per offline discussion this will be documented.


docker integration driver ignores preemption option during mount command

When attempting to mount a volume via docker integration driver, there is no check in place to allow preemptive mount option to function correctly. This is the result of scrubbing unavailable volumes from the list of volumes before processing attach/mount command.
https://github.com/codedellemc/libstorage/blob/master/drivers/integration/docker/docker.go#L173

Fix looks good


Add lein-cljfmt and apply default rules to codes - Draft

Run lein cljfmt fix <target> on

src
test
docs
examples

@anthony-khong
This is for discussion only.
Please take time to review and let me know if this is something we can use.
I am trying to learn more about the rules that allow the tools to be format to your preference.
You can learn more about the rules here
Also more documentation is available here

Codecov Report

Merging #254 into develop will not change coverage.
The diff coverage is 100.000%.

@@             Coverage Diff             @@
##            develop       #254   +/-   ##
===========================================
  Coverage   100.000%   100.000%
===========================================
  Files            34         34
  Lines          2658       2658
===========================================
  Hits           2658       2658

Impacted Files
Coverage Δ

src/clojure/zero_one/geni/core.clj
100.000% <ø> (ø)

src/clojure/zero_one/geni/ml/classification.clj
100.000% <ø> (ø)

src/clojure/zero_one/geni/ml/clustering.clj
100.000% <ø> (ø)

src/clojure/zero_one/geni/ml/evaluation.clj
100.000% <ø> (ø)

src/clojure/zero_one/geni/ml/feature.clj
100.000% <ø> (ø)

src/clojure/zero_one/geni/ml/fpm.clj
100.000% <ø> (ø)

src/clojure/zero_one/geni/ml/recommendation.clj
100.000% <ø> (ø)

src/clojure/zero_one/geni/rdd/unmangle.clj
100.000% <ø> (ø)

src/clojure/zero_one/geni/spark_context.clj
100.000% <ø> (ø)

src/clojure/zero_one/geni/storage.clj
100.000% <ø> (ø)

... and 22 more

Will close the PR, as we will take it in small steps.
See PR #255 going forward.


New Tab page UI for choosing the profile to open from a grid list of profile icons

Description of the new feature/enhancement
New Tab UI for choosing the profile to open
At this moment, opening Windows Terminal (without specifically choosing a profile) would open a window with the default profile. Some power-users such as myself may need to open a different profile based on our immediate need. This would require me to open a second tab with the correct profile, then close the tab with the default one. (Or we'd have to create profile shortcuts in our Desktop or Start Menu.)
Modern web browsers have a built-in New Tab page which often shows frequent shortcuts or recently visited pages to be helpful to the user in deciding where they would like to go. A simplified version of such a page could be added to Windows Terminal, that simply displays a list of all the profiles. Choosing one could open it in the new tab that's already created and waiting for a profile.

In Settings, the Default Profile selection can have an option called "Show the New Tab page." Consequently, the New Tab button on the tab bar would always show the New Tab page instead of opening a profile. This further increases the usefulness of this feature by giving the same convenient UI to pick a profile. (This may negate the need for the drop-down which could be removed or made optional.)
Note: The checkbox, if checked, would set the chosen profile to the Default Profile selection in Settings. Subsequently, all new tabs and windows will open in that profile until the user returns to the Settings and resets the selection to "Show the New Tab page." (The checkbox would be unchecked again.)

Thanks for the suggestion! This is actually already being tracked by another issue on our repo - please refer to #3586 for more discussion.
/dup #3586
(curiously enough, this was requested earlier this week over in #10338 as well)

@zadjii-msft Thank you! And sorry, I tried searching but couldn't find anything similar. Thanks for pointing out the duplicates.


Low quality display

I have Xiaomi Redmi 4 Pro and I get low quality display

What is the result of:
scrcpy -v

(especially the SDL version)
What operating system do you use?
Could you provide a screenshot in 1:1 (pixel-perfect) (press Ctrl+g), please?

scrcpy v1.0
dependencies:

SDL 2.0.7
libavcodec 57.107.100
libavformat 57.83.100
libavutil 55.78.100

OS: Windows 10
This is the part that fits my screen:

OK, so it seems the local resize is not filtered correctly.

SDL 2.0.7

Very recent, great.
Could you try again with this change, please?
--- a/app/src/screen.c
+++ b/app/src/screen.c
@@ -19,7 +19,7 @@ SDL_bool sdl_init_and_configure(void) {
     atexit(SDL_Quit);

     // Bilinear resizing
-    if (!SDL_SetHint(SDL_HINT_RENDER_SCALE_QUALITY, "1")) {
+    if (!SDL_SetHint(SDL_HINT_RENDER_SCALE_QUALITY, "2")) {
         LOGW("Could not enable bilinear filtering");
     }

is there a chance to include that in a built windows package?
I'm having trouble getting gradle via ninja

I have no access to a Windows machine right now.
You don't need to build the server (so no need for gradle), just use the prebuilt server.

I have the same issue and tried rebuilding from source with the bilinear filtering patch above. It still looks grainy in resize to fit mode (ctrl+x).

I just had this issue.
Using the -b flag and playing with the bitrate fixed it for me.
For me on OnePlus 6 18-20 was best bitrate
scrcpy.exe -b 20

Right now I have scaling set to 200 % because it is needed on 4k display. So I tried to set scaling to 100 % and restarted application. Now it looks much better. Will this information help you fix the issue?

I actually solved it myself. scrcpy.exe > Properties > Compatibility > Change high DPI settings > Override high DPI scaling behavior > Scaling performed by: Application.
It was the buggy Windows 10 that could not do it properly. Hope ths will help someone else.

I would also like to thank you for a great software.

I experience the same issue here, since I upgraded to >= 1.9. I did not experience any issue in 1.8 and below.

If you use v1.9 and replace SDL2.dll by the one from v1.8, does it work better?

I am using OSX 🙃

@acadet Ah yes, of course.
I'm not sure this is the same problem as this issue. Is the quality better if you press Ctrl+g (pixel-perfect)? (I guess not, contrary to https://github.com/Genymobile/scrcpy/issues/40#issuecomment-372114617)
Your problem is more likely a hidpi setting (each pixel takes 2×2 pixel).

Ctrl + g did not help, unfortunately.

So it's definitely a hidpi issue.
Is there a setting similar to https://github.com/Genymobile/scrcpy/issues/40#issuecomment-424466723 on macOS?

Not on an app basis, only for my entire display.

@rullskidan :
I actually solved it myself. scrcpy.exe > Properties > Compatibility > Change high DPI settings > Override high DPI scaling behavior > Scaling performed by: Application.
It was the buggy Windows 10 that could not do it properly. Hope ths will help someone else.

Yes, definitely! Thanks! Can the Windows section of the README be extended with this tip? (or else the FAQ)
I was horrified by the output quality on a Windows 10 PC compared to a Mac. I resized to pixel-perfect and the image was crisp and sharp, so I knew it wasn't scrcpy at fault.
After setting "Override high DPI scaling behavior. Scaling performed by:" to "application" (in the Compatibility tab), the quality is comparable again to what I was used to from Mac.
It's a step that cannot be skipped on Windows 10.

Can the Windows section of the README be extended with this tip? (or else the FAQ)

I just updated the FAQ.

Hi, I have a OnePlus 6 and I have tried everything listed above as well as in the FAQ. I'm still facing this issue. It's certainly not because of the low DPI setting on my phone, because I even tried changing the display size to Large, Very Large, but that doesn't fix the problem.
Please let me know, if there's anything I can do to fix this.

What is the native definition of your device? (the size printed after "Initial texture size" in scrcpy console)

1 file pushed. 2.1 MB/s (26202 bytes in 0.012s)
INFO: Initial texture: 1080x2280
This is the default on running scrcpy.exe

If you press Ctrl+G (1:1 pixel), how is the quality?

Actually, when it's totally zoomed in by Ctrl+G, and I can only see less than half of my screen, it's clear.
But when I resume it to full screen by (Win + ↑), it's the same quality as before.

OK, so it's just the downscaling.
I guess it's just that the text is rendered with some high-def grid target, downscaling just breaks the antialiasing hints or something. (https://gankra.github.io/blah/text-hates-you/)
Could you post a capture after you pressed Ctrl+G? (or even just adb exec-out screencap -p > file.png)
Then try to resize it with some image editor (like Gimp) to the same definition (the screen part on your first screenshot is about 630×300).

This is the screenshot after I pressed Ctrl + G. Was only able to capture half of it using Snipping Tool.
Sorry, I tried executing the command but the file it generated couldn't open correctly.

If I linear-resize it to 300px width (like your previous screenshots), I get this:

It's not that bad. I'm interested in a full-size screenshot of your New York Time app :)
You could also try to record, then play it in VLC for example:
scrcpy -r fullsize.mp4   # then resize while playing
scrcpy -m640 -r small.mp4

How is the quality?

Okay, so after recording it through VLC, both the videos have the same quality of text apparently. But, when I cropped the video inside VLC, the text was clear and readable again. I can't for the life of me figure out the problem.
Have uploaded both on my

Hmm, your small.mp4 is not small, it's 1080×2280. Did you really pass -m640 when recording it?

You're right. This one is not clear. Updated the link again, sorry.

One more test: what if you play the 1080x2280 in VLC, but resizing the window so that it has a small width (~300px)?

Yes, this is totally clear. Can read everything clearly. Do you have any lead?

GPU's Bilinear filtering / anisotropic filtering only works for scaling up. You need trilinear filtering (mipmapping) to get good quality.
I don't know how to enable that in SDL. But now that FFMPEG is used, how about scaling with libswscale?

@cjxgm Compare linear downscaling with nearest downscaling: https://github.com/Genymobile/scrcpy/issues/40#issuecomment-590293741
It seems @Sumat2222 does not even have linear filtering.
@Sumat2222
diff --git a/app/src/scrcpy.c b/app/src/scrcpy.c
index 17be1ed..fa6ee86 100644
--- a/app/src/scrcpy.c
+++ b/app/src/scrcpy.c
@@ -61,7 +61,7 @@ sdl_init_and_configure(bool display) {
     }

     // Use the best available scale quality
-    if (!SDL_SetHint(SDL_HINT_RENDER_SCALE_QUALITY, "2")) {
+    if (!SDL_SetHint(SDL_HINT_RENDER_SCALE_QUALITY, "1")) {
         LOGW("Could not enable bilinear filtering");
     }

Does it improve the result?
You could also try to use another render driver:
index 17be1ed..4224c00 100644
--- a/app/src/scrcpy.c
+++ b/app/src/scrcpy.c
@@ -65,6 +65,10 @@ sdl_init_and_configure(bool display) {
         LOGW("Could not enable bilinear filtering");
     }

+    if (!SDL_SetHint(SDL_HINT_RENDER_DRIVER, "opengl")) {
+        LOGW("Could not set render driver");
+    }
+
 #ifdef SCRCPY_SDL_HAS_HINT_MOUSE_FOCUS_CLICKTHROUGH
     // Handle a click to gain focus as any other click
     if (!SDL_SetHint(SDL_HINT_MOUSE_FOCUS_CLICKTHROUGH, "1")) {

Compare linear downscaling with nearest downscaling: #40 (comment)

That's expected behavior. When downscaling using OpenGL or D3D, linear filtering is mostly the same as nearest filtering. True linear filtering (what GIMP and other CPU algorithms do) requires averaging pixels in the covering area (average many pixels), which is expensive to do on GPU. OpenGL / D3D on the otherhand, do linear mixing between nearest samples (average 2 pixels only), thus producing results similar to nearest filtering.

@cjxgm Oh, you're right. I thought the problem was only on some computers on Windows, but in fact the reason was I did not try with such a low definition (from 1440x3120 to 296x640).
Btw, I also reproduce in VLC (if I play the recorded high-def video but resize the window to a small size).
However, I'm not sure it makes sense to generate mipmaps for one-time frames at 60 fps :/

Speed-wise, generating mipmaps at 60 fps is not that bad. We actually do that in our in-house game engine. And many modern game engines generate mipmaps for the screen buffer in their post-processing pipelines. GPUs are fast for generating mipmaps, you can always measure and compare.
libswscale (part of FFmpeg) can also be used for scaling, which also has good quality and is heavily optimized.

I don't know how to enable that in SDL.

It seems we need to use OpenGL directly (calling glGenerateMipmap(GL_TEXTURE_2D)), after the SDL texture has been bound with SDL_GL_BindTexture.
But that means that we must force OpenGL (SDL will use Direct3D on Windows by default). I don't know well the implications of this on multi-platform compatibility (especially Windows). I want scrcpy to be compatible on all platforms, and I don't want to write/maintain several implementations of the rendering part (Direct3D, OpenGL, …).
On the other hand, if that's possible and works everywhere (set aside the GLSL differences between OpenGL versions), that would be great: I could write my vertex + fragment shaders and have better control on drawing. And I could do the YUV to RGB conversion in the fragment shader directly (it seems it is done in software in SDL_UpdateTextureYUV).

Hi, nothing seems to be able to fix this problem for me. I guess, I will have to use as is.
Anyway, thanks for making such a great application 👍  You're a legend!

Before reaching consensus on how to implement it, I'd like to get my hands dirty for some quick hacks.
I've tried to implement both mipmapping and libswscale approach, tune their parameters, and get these results:

Well, this page may not be a good test candidate, but it is what it is.
My phone is 1080p. The result window is 225p. Tested on Linux.
As expected:

"original bilinear" looks like nearest filtered.
"libswscale" looks much better. Bicubic is clearer than bilinear.

Mipmapping + trilinear filtering alone makes everything blurry. (sort of expected.)
Notch up 1-layer of mipmap makes the result pretty close to bicubic (well, not all the time).

I'm posting the patch for "mipmap trilinear bias" here, so that people can try it out if they really can't stand the image quality (like me). The code is ugly because it's a quick hack.
diff --git a/app/src/screen.c b/app/src/screen.c
index beb1075..e7848c5 100644
--- a/app/src/screen.c
+++ b/app/src/screen.c
@@ -15,6 +15,18 @@

 #define DISPLAY_MARGINS 96

+// just-enough GL to enable trilinear filtering
+typedef unsigned int gl_enum;
+typedef int gl_int;
+typedef float gl_float;
+static void (*gl_generate_mipmap)(gl_enum target) = NULL;
+static void (*gl_tex_parameteri)(gl_enum target, gl_enum pname, gl_int param) = NULL;
+static void (*gl_tex_parameterf)(gl_enum target, gl_enum pname, gl_float param) = NULL;
+static const gl_enum gl_texture_2d = 0x0DE1;
+static const gl_enum gl_texture_min_filter = 0x2801;
+static const gl_enum gl_linear_mipmap_linear = 0x2703;
+static const gl_enum gl_texture_lod_bias = 0x8501;
+
 // get the window size in a struct size
 static struct size
 get_window_size(SDL_Window *window) {
@@ -186,6 +198,8 @@ screen_init_rendering(struct screen *screen, const char *window_title,
         window_flags |= SDL_WINDOW_BORDERLESS;
     }

+    SDL_SetHint(SDL_HINT_RENDER_DRIVER, "opengl");
+
     int x = window_x != -1 ? window_x : (int) SDL_WINDOWPOS_UNDEFINED;
     int y = window_y != -1 ? window_y : (int) SDL_WINDOWPOS_UNDEFINED;
     screen->window = SDL_CreateWindow(window_title, x, y,
@@ -230,6 +244,13 @@ screen_init_rendering(struct screen *screen, const char *window_title,

     screen->windowed_window_size = window_size;

+    gl_generate_mipmap = SDL_GL_GetProcAddress("glGenerateMipmap");
+    gl_tex_parameteri = SDL_GL_GetProcAddress("glTexParameteri");
+    gl_tex_parameterf = SDL_GL_GetProcAddress("glTexParameterf");
+    if (!gl_generate_mipmap || !gl_tex_parameteri || !gl_tex_parameterf) {
+        LOGW("Trilinear filtering disabled. OpenGL 3.0+ required.");
+    }
+
     return true;
 }

@@ -296,6 +317,14 @@ update_texture(struct screen *screen, const AVFrame *frame) {
             frame->data[0], frame->linesize[0],
             frame->data[1], frame->linesize[1],
             frame->data[2], frame->linesize[2]);
+
+    if (gl_generate_mipmap && gl_tex_parameteri && gl_tex_parameterf) {
+        SDL_GL_BindTexture(screen->texture, NULL, NULL);
+        gl_generate_mipmap(gl_texture_2d);
+        gl_tex_parameteri(gl_texture_2d, gl_texture_min_filter, gl_linear_mipmap_linear);
+        gl_tex_parameterf(gl_texture_2d, gl_texture_lod_bias, -1.0f);
+        SDL_GL_UnbindTexture(screen->texture);
+    }
 }

 bool

@rom1v Thanks alot for this pretty awesome tool and thanks to  @cjxgm for the awesome solution, build again , and everything looks fine...🙂,
💗  this tool....

@cjxgm This improves the rendering when there is a big downscaling on the client side.
But in theory, it would be better to properly downscale before encoding. It would reduce the bitrate and would be "recordable" (for now recording with -m600 for example records what the device sends, with a linear/nearest downscale).
Do you know if there is a possibility to downscale before encoding, on the device?

@cjxgm I realize I never responded to https://github.com/Genymobile/scrcpy/issues/40#issuecomment-591917787
Firstly, thank you for your work :+1:
On the screenshots, libswscale bicubic is clearly the best (but it's in software).
IMO, mipmap trilinear bias looks pretty good.
However, I applied your patch on current master, and I cannot get good results with my 1440x3120 device, downscaled to 600. On the left, without your patch, on the right, with your patch:

Even with lod_bias -1.0f I don't see any difference on my machine. (I checked, the code is correctly called.)
Did I miss something?

@cjxgm I realize I never responded to https://github.com/Genymobile/scrcpy/issues/40#issuecomment-591917787
Firstly, thank you for your work :+1: Results are great :tada:
On your screenshots, libswscale bicubic is clearly the best (but it's in software), but mipmap trilinear bias looks pretty good (I personally prefer lod_bias=-1).
Given the good results, and since the performance looks ok, I think that it should be the default for downscaling in scrcpy (with an option --no-mipmaps to get the previous behavior).
I suggest adding an option --render-driver (just forwarded to the SDL hint), use mipmaps when supported (OpenGL and supported version) and --no-mipmaps to disable.
What do you think?

@cjxgm Would you have time to add this feature and submit a PR (on dev branch)?
If you don't have time (or motivation), please tell me, I'll try to do it :wink: I would like a better downscaling in the next release :)

Please test PR #1284 :wink:

Trilinear filtering is enabled for OpenGL in v1.13. :tada:
Big thanks to @cjxgm.

The scrcpy GUI hang on Windows 10 if I use --render-driver=opengl but works if I use --render-driver=opengl --no-mipmaps. By hang I mean no screen interaction and closing the window does not work.
scrcpy 1.13

dependencies:
 - SDL 2.0.12
 - libavcodec 58.54.100
 - libavformat 58.29.100
 - libavutil 56.31.100

[server] INFO: Device: OnePlus ONEPLUS A6003 (Android 10)
INFO: Renderer: opengl
INFO: OpenGL version: 4.6.0 NVIDIA 442.19
INFO: Trilinear filtering enabled
INFO: Initial texture: 1080x2280

Oh that's bad 😞 I guess it depends on your drivers.
Even if it defeats the ourpose of the feature, you could try with a lower definition:
scrcpy -m640 --render-driver=opengl

scrcpy -m640 --render-driver=opengl

I tried with -m640 and it still hangs. My GPU is nvidia GTX 1050 TI.

scrcpy -m640 --render-driver=opengl

I tried with -m640 and it still hangs. My GPU is nvidia GTX 1050 TI.

hello did you solve this?
and did your pc using integrated graphic card?
if you using integrated graphic card and gtx 1050 you can force scrcpy to use your opengl on your dedicated graphic card using nvidia control panel.
its solve my ugly rendered graphics on my host pc

if you using integrated graphic card and gtx 1050 you can force scrcpy to use your opengl on your dedicated graphic card (on your case 1050) using nvidia control panel.

It is a desktop GPU. I'm not aware that desktops have switchable GPU like Optimus. On desktops, you can only select which GPU that will be used on the BIOS.

It works now with scrcpy 1.4 (64 bit)
INFO: OpenGL shaders: ENABLED
INFO: Created renderer: opengl
INFO: Renderer: opengl
INFO: OpenGL version: 4.6.0 NVIDIA 442.19
INFO: Trilinear filtering enabled
INFO: Initial texture: 1080x2280

It works now with scrcpy 1.14 (64 bit)
INFO: OpenGL shaders: ENABLED
INFO: Created renderer: opengl
INFO: Renderer: opengl
INFO: OpenGL version: 4.6.0 NVIDIA 442.19
INFO: Trilinear filtering enabled
INFO: Initial texture: 1080x2280

may i ask a noob question you just type that on the black window that appears you press enter and then it gets better? or you have to type it somewhere else

@komantis It's a parameter to pass when you launch scrcpy. See https://github.com/Genymobile/scrcpy/issues/385#issuecomment-451903274

In Windows 10, using version 1.15.1, opengl driver works with the 64 bit version but hangs with the 32 bit version.
The output when running the 32 bit version:
>  scrcpy.exe -S --render-driver opengl -V debug
INFO: scrcpy 1.15 <https://github.com/Genymobile/scrcpy>
DEBUG: Using server (portable): C:\Apps\Android\scrcpy\scrcpy-server
C:\Apps\Android\scrcpy\scrcpy-server: 1 file pushed, 0 skipped. 17.7 MB/s (33618 bytes in 0.002s)
DEBUG: Screensaver enabled
[server] INFO: Device: OnePlus ONEPLUS A6003 (Android 10)
DEBUG: Starting stream thread
DEBUG: Starting controller thread
DEBUG: Starting receiver thread
INFO: Renderer: opengl
INFO: OpenGL version: 4.6.0 NVIDIA 432.00
INFO: Trilinear filtering enabled
INFO: Initial texture: 1080x2280
[server] INFO: Device screen turned off

opengl driver works with the 64 bit version but hangs with the 32 bit version.

And without --render-driver opengl, both 32 bits and 64 bits work?

And without --render-driver opengl, both 32 bits and 64 bits work?

Yes both versions work without --render-driver openg.
Unrelated to the driver problem, with -S the initial display is a black screen rather than the login screen, and right clicking the mouse results in [server] INFO: Device screen turned off message in the console rather than making the screen turns on. I have to send a power key event using adb shell input keyevent KEYCODE_POWER to actually turn on the screen.

I have to send a power key event using adb shell input keyevent KEYCODE_POWER to actually turn on the screen.

That's basically what is done on start (by other means).

actually even without using the -S option the initial screen display is black.

Always? Does MOD+p work (Alt+p by default)?

I didn't experience this with version 1.14 before.

Could you test again with v1.14? (then retest with v1.15.1)

Always? Does MOD+p work (Alt+p by default)?

Always. Alt+p works.

Could you test again with v1.14? (then retest with v1.15.1)

tested with 1.14 -> screen on
tested with 1.15.1 -> screen black

Tracking events with getevent shows that 1.14 sends some events when it is started but not when its windows is closed while 1.15.1 sends some events when the scrcpy window is closed but nothing when it is started.
1.14> scrcpy -V debug

# getevent -t
130|OnePlus6:/ # getevent -t
add device 1: /dev/input/event6
  name:     "uinput-tri-state-key"
add device 2: /dev/input/event5
  name:     "sdm845-tavil-snd-card Button Jack"
add device 3: /dev/input/event4
  name:     "sdm845-tavil-snd-card Headset Jack"
add device 4: /dev/input/event0
  name:     "qpnp_pon"
add device 5: /dev/input/event1
  name:     "gf_input"
could not get driver version for /dev/input/mice, Not a typewriter
add device 6: /dev/input/event2
  name:     "gpio-keys"
add device 7: /dev/input/event3
  name:     "synaptics"
[<PHONE_NUMBER>24] /dev/input/event3: 0003 002f 00000000
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 00000489
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>24] /dev/input/event3: 0003 002f 00000001
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 0000048a
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>24] /dev/input/event3: 0003 002f 00000002
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 0000048b
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>24] /dev/input/event3: 0003 002f 00000003
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 0000048c
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>24] /dev/input/event3: 0003 002f 00000004
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 0000048d
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>24] /dev/input/event3: 0003 002f 00000005
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 0000048e
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>24] /dev/input/event3: 0003 002f 00000006
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 0000048f
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>24] /dev/input/event3: 0003 002f 00000007
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 00000490
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>24] /dev/input/event3: 0003 002f 00000008
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 00000491
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>24] /dev/input/event3: 0003 002f 00000009
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 00000492
[<PHONE_NUMBER>24] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>24] /dev/input/event3: 0000 0000 00000000

>>> scrcpy runs

1.15.1> scrcpy -V debug

# getevent -t

130|OnePlus6:/ # getevent -t
add device 1: /dev/input/event6
  name:     "uinput-tri-state-key"
add device 2: /dev/input/event5
  name:     "sdm845-tavil-snd-card Button Jack"
add device 3: /dev/input/event4
  name:     "sdm845-tavil-snd-card Headset Jack"
add device 4: /dev/input/event0
  name:     "qpnp_pon"
add device 5: /dev/input/event1
  name:     "gf_input"
could not get driver version for /dev/input/mice, Not a typewriter
add device 6: /dev/input/event2
  name:     "gpio-keys"
add device 7: /dev/input/event3
  name:     "synaptics"

>>> scrcpy runs
>>> scrcpy window closed

[<PHONE_NUMBER>80] /dev/input/event3: 0003 002f 00000000
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 0000049d
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>80] /dev/input/event3: 0003 002f 00000001
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 0000049e
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>80] /dev/input/event3: 0003 002f 00000002
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 0000049f
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>80] /dev/input/event3: 0003 002f 00000003
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 000004a0
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>80] /dev/input/event3: 0003 002f 00000004
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 000004a1
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>80] /dev/input/event3: 0003 002f 00000005
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 000004a2
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>80] /dev/input/event3: 0003 002f 00000006
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 000004a3
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>80] /dev/input/event3: 0003 002f 00000007
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 000004a4
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>80] /dev/input/event3: 0003 002f 00000008
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 000004a5
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>80] /dev/input/event3: 0003 002f 00000009
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 000004a6
[<PHONE_NUMBER>80] /dev/input/event3: 0003 0039 ffffffff
[<PHONE_NUMBER>80] /dev/input/event3: 0000 0000 00000000

(could you please open a new issue with the content of that post?)

I think using KEYCODE_WAKEUP as in this commit https://github.com/Genymobile/scrcpy/commit/322f1512ea806611eb37f508cd952bbbc050f853 does not work, at least in my device.
Running adb shell input keyevent KEYCODE_WAKEUP also does not turn on the screen.

Hi, I'm having a problem with the quality of the display, I've tried putting in commands but none of them work and I don't know why

@MayaArche https://github.com/Genymobile/scrcpy/issues/385#issuecomment-451903274

Thank you so much, sorry for my stupidity.

@cjxgm I guess we should implement the libswscale approach as an option too, at least for Windows and macOS (OpenGL mipmapping is only available on Linux on scrcpy concretely).
I did a quick test, the performance seems ok (but to work properly the changes are quite intrusive, in particular, we must manually resize on window resize, create a texture on target size change instead of frame size change (but only if this option is enabled), reinit SwsContext when necessary, etc.).

use --encoder parameter and encoder available
example: --encoder OMX.google.h264.encoder

use --encoder parameter and encoder available
example: --encoder OMX.google.h264.encoder

I am working on supporting swscale filters.
Here is a WIP branch you could test: swscale.
scrcpy --window-height=400 --scale-filter=none
scrcpy --window-height=400 --scale-filter=trilinear  # mipmap, which is the current default if supported

# swscale filters:
scrcpy --window-height=400 --scale-filter=bilinear
scrcpy --window-height=400 --scale-filter=bicubic
scrcpy --window-height=400 --scale-filter=x
scrcpy --window-height=400 --scale-filter=point
scrcpy --window-height=400 --scale-filter=area
scrcpy --window-height=400 --scale-filter=bicublin
scrcpy --window-height=400 --scale-filter=gauss
scrcpy --window-height=400 --scale-filter=sinc
scrcpy --window-height=400 --scale-filter=lanczos
scrcpy --window-height=400 --scale-filter=spline

Thank you for your feedbacks :wink:
cc @cjxgm

However, for perfect responsiveness, the swscaling should be executed from a separate thread. I need to change that.

Is there a binary build for the swscale branch? I'm on Windows so installing mingw64 is not trivial.

@char101 Sorry, there isn't, and it requires to update the Windows build script to add a dependency to libswscale, so I will do that later once I get a proper implementation with scaling out of the main thread.

Thanks. By the way, would it be possible to use adb shell wm size and adb shell wm density to make the phone resolution match the app window size? That would give the best output by eliminating any scaling.

Asynchronous swscaling brings too much complexity. In bulk (probably not exhaustive):

the window size might be changed between the request and the result
the swscale target size depends on the window size...
but the window size depends on the new frame size (if it changes, on rotation for example)
the window should not be resized immediately on frame size change, but the new frame must be computed before the swscale and applied after
all the hacky state to make fullscreen/maximized on all platforms must now be handled from two separate threads
the swscale VS non-swscale paths are quite different (one handles the resize to be done on the CPU, the other on the GPU)

In any case, swscaling adds a lot of latency (and consumes a lot of CPU). For my 1440x3120 device, bicubic swscaling takes between 20ms and 60ms (depending on the target window size). Bilinear swscaling a bit less, but still a lot.
I'm not sure it's worth it. The complexity would necessarily add bugs for things which works correctly now, and make the whole thing difficult to maintain.
An alternative is to keep a synchronous version (like here, but it is not finished). The complexity is acceptable, but dozens of milliseconds on the UI thread is less acceptable.
The real performant solution would be to enable mipmapping (which is sufficient and solves the problem) for renderers other than OpenGL. But SDL does not provide this possibility, and writing and maintaining shaders manually for all platforms is not conceivable (not enough time/resources).

By the way, would it be possible to use adb shell wm size and adb shell wm density to make the phone resolution match the app window size?

In theory, yes, and the performance would be even better (with a smaller device size). However, in practice, on my devices, it does not work well, because it breaks the UI of apps (and even the system, the navigation bar is quite broken, the home screen and lock screen are a bit broken too…).

I actually solved it myself. scrcpy.exe > Properties > Compatibility > Change high DPI settings > Override high DPI scaling behavior > Scaling performed by: Application.
It was the buggy Windows 10 that could not do it properly. Hope ths will help someone else.

This worked perfectly for me. My Android display is no longer pixelated when I set Windows 11 DPI compatibility to application! Thank you!

I am not sure what happened, but on the latest Windows 11 dev preview build (22523.1000) with the latest NVIDIA driver 511.09, the text on the screen appears pixelated again. This is happening even though I set the Scaling performed by option to Application, as I posted above.
If I press ALT + g for "pixel perfect" mode then it looks a lot better, but the window is too big for my 4k monitor.
Screenshot of Pixelization

Scaling Option Setting

Pixel Perfect Mode (looks sharp)

What's the device size ("Initial texture" in the console) and the window size?
What if you start with your device in landscape mode?

What's the device size ("Initial texture" in the console) and the window size?

scrcpy 1.21 <https://github.com/Genymobile/scrcpy>
C:\Users\TrevorSullivan\scoop\apps\scrcpy\current\scrcpy-server: 1 file pushed, 0 skipped. 48.9 MB/s (40067 bytes in 0.001s)
[server] INFO: Device: Google Pixel 4 XL (Android 12)
INFO: Renderer: direct3d
INFO: Initial texture: 1440x3040

What if you start with your device in landscape orientation?

The quality of the text and graphics looks a lot better in landscape mode.
scrcpy --lock-video-orientation=1

1440x3040

OK, so it's just that the device screen is rendered on a surface 3040 pixels height, and downscaled by a significant factor to your monitor height.
There are two causes of bad quality related to this issue:

hidpi scaling wrongly configured (it will be correct by default on 1.22 https://github.com/Genymobile/scrcpy/commit/ad11c5babb0ff89647dd606392ed7907a20277c7)
display at a size way smaller than the device screen (your issue)

For the second issue, mipmapping reduces the problem: https://github.com/Genymobile/scrcpy/pull/1284
But it works only on OpenGL (SDL only exposes what's necessary to enable mipmapping on OpenGL), so it works on Linux.
You could enable OpenGL renderer on Windows, but drivers are often slow/buggy:
scrcpy --render-driver=opengl

https://github.com/Genymobile/scrcpy/blob/master/FAQ.md#the-quality-is-low

Yeah I already tried the OpenGL renderer, based on the document you linked to. It is still rather low quality text though.
Both methods are kinda low quality compared to the landscape mode or ALT+G pixel perfect mode, under the default Direct3D renderer.
OpenGL
scrcpy --serial [2605:59c0:c4e0:901:xxxxxxxxxxxxxx]:43017 --render-driver=opengl

OpenGL (no mipmaps)
scrcpy --serial [2605:59c0:c4e0:901:xxxxxxxxxxxxxx]:43017 --render-driver=opengl --no-mipmaps

Could you please test this binary (replace in v1.21): https://github.com/Genymobile/scrcpy/issues/2865#issuecomment-997225952

Could you please test this binary (replace in v1.21): #2865 (comment)

I downloaded it and enabled the Application-based DPI scaling on the binary. It's still pixelated.

@pcgeek86 Use the official v1.21 release (not scoop or other), then replace scrcpy.exe by the one provided in the comment.

It's magically fixed itself somehow. I have high quality text again without any special parameters, and without doing pixel-perfect.

Thank you!


Keep file names

Is there a way to keep the original filenames of the uploaded files?  Right now it is forcing a UUID.

Hi @jungalwala, sorry for the late reply. I was busy with other projects. I will fix it in the following days. Thanks!

Hi @jungalwala, I've released the new version which can keep the original filename or set it to a new customized filename.
Thanks!


Investigate using buildah to run docker commands in the pipelines

Running certain docker commands inside a docker container requires privileged security roles and use of docker sockets. Both of them are not ideal.  Investigate if we can use buildah instead of avoid the use of docker sockets & elevated permissions.

Appsody team is working on this.  We will have to make a tweak to the pipelines to absorb those changes.

Replaced appsody-docker with appsody-buildah in the build tasks and it works now.