docs.botframework.com links are dead

Many links on this page now redirect to https://docs.microsoft.com/en-us/azure/bot-service

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2698526b-6251-3832-85e6-e6ba41b74909
Version Independent ID: f2109d38-c3cd-06b3-7c51-fefcf1367fb9
Content: Conduct audio calls - Bot Service
Content Source: articles/nodejs/bot-builder-nodejs-conduct-audio-calls.md
Service: bot-service
Product: bot-framework
GitHub Login: @v-ducvo
Microsoft Alias: v-ducvo

Hi Eric,
Thanks for your feedback. You are correct that currently any link that previously pointed to the v3 SDK documentation currently redirects to https://docs.microsoft.com/en-us/azure/bot-service/?view=azure-bot-service-4.0  I have asked that we provide a backup version of this documentation and if that becomes available, will correct these and similar links to point to that location instead. I will let you know the outcome of this request.

Hi Eric, any news on this? As I see it, there is currently no documentation of any SDK online (neither v3 nor v4), which basically makes the SDK useless?!

Hi any updates on this?
We are struggling to find the SDK references to v3.

Thanks Eric!
I came on here to let you know these were now available and you just pointed out their availability!
Thanks for your feedback.  Closing this now as a resolved issue.


Add helper to display item count information

Similar to what will_paginate has:

I always found this type of information handy and have included it in my Rails projects.  I didn't see this option available for this plugin (sorry, not sure if that is the right term) and added a simple version of the functionality myself.
Would you accept a PR for this?

You provide the %Scrivener.Page{} which has that information easily accessible so I'm not following whats to be gained by offering a helper to do that? Additionally, the copy would need to be swappable and I wouldn't want to introduce a gettext dependency. What would your helper do exactly?

What would your helper do exactly?

The bottom part, "Displaying x 1-20, of 59 in total"

You provide the %Scrivener.Page{} which has that information easily accessible

It provides page_number, page_size, total_entries, total_pages, right?  Does it provide the indexes of the entries on the current page?
Here is the code if it's helpful for anyone else.  I'm new to elixir so I'm probably doing something fundamentally wrong and I need to cover some edge cases:
 def pagination_info(page, type) do
  total_entries = page.total_entries

  first_index = find_first_index(page)
  if first_index <= total_entries do
    content_tag :p, class: "pagination-info" do
      pagination_info_content(page, type)
    end
  end
end

defp pagination_info_content(page, type) do
  page_size = page.page_size
  total_entries = page.total_entries
  first_index = find_first_index(page)
  last_index = find_last_index(page)

  if page_size >= total_entries do
    "Displaying all #{type}"
  else
    start = "Displaying "
    indexes =
      if page_size > 1 and first_index != last_index do
        "#{first_index} - #{last_index}"
      else
        first_index
      end

    indexes = content_tag :strong, indexes
    total_entries = content_tag(:strong, total_entries)

    [start, type, " ", indexes, " of ", total_entries, " in total"]
  end
end

defp find_first_index(page) do
  (page.page_number - 1) * page.page_size + 1
end

defp find_last_index(page) do
  page_number = page.page_number
  page_size = page.page_size
  total_entries = page.total_entries

  index = if page_size > 1, do: page_number * page_size
  cond do
    index > total_entries -> total_entries
    true -> index
  end
end

Use like so:
<%= pagination_info @page, "Widgets" %>

I think this should be a helper in your application since that text you have in there will need to be configurable and internationalized w/ gettext. If you really feel it something others could use, feel free to create your own package on hex.


Hover effect on empty blocks of calender

It seems that Hover effect on empty calendar blocks aren't disabled.
For example hover effect should be only on dates not on empty blocks of dates.
Expected behaviour with hover :

Current behaviour of hover on empty blocks:

Hint

This problem can be fixed by modifying javascript code when we are generating empty  elements for week day fill up issue.

@jahir-raihan hey i am beginner web developer learning javascript will i be able to work on this ?

Sure, try your best. I will be there to assist you on the way.

can i try this or is it assigned???

@Axxi3 Try your best, no one is assigned.

@devbytemehedi hey man i fixed the code i am giving a pull request please review the changes.


Hbc::DSL#license is deprecated!

General troubleshooting steps

[x] I have retried my command with --force and the issue is still present.
[x] I have checked the instructions for reporting bugs (or making requests) before opening the issue.

[x] None of the templates was appropriate for my issue, or I’m not sure.

[x] I ran brew update-reset && brew update and retried my command.
[x] I ran brew doctor, fixed as many issues as possible and retried my command.
[x] I understand that if I ignore these instructions, my issue may be closed without review.

Description of issue
Cannot install - Hbc::DSL#license is deprecated!
Output of your command with --verbose --debug
$ brew cask install --force --verbose --debug wch-ch34x-usb-serial-driver
Warning: Calling Hbc::DSL#license is deprecated!
There is no replacement.
/usr/local/Homebrew/Library/Taps/mengbo/homebrew-ch340g-ch34g-ch34x-mac-os-x-driver/Casks/wch-ch34x-usb-serial-driver.rb:9:in `block in load'
Please report this to the mengbo/ch340g-ch34g-ch34x-mac-os-x-driver tap!
Error: Calling <<-EOS.undent is disabled!
Use <<~EOS instead.
/usr/local/Homebrew/Library/Taps/mengbo/homebrew-ch340g-ch34g-ch34x-mac-os-x-driver/Casks/wch-ch34x-usb-serial-driver.rb:23:in block in load' Please report this to the mengbo/ch340g-ch34g-ch34x-mac-os-x-driver tap! Or, even better, submit a PR to fix it! Follow the instructions here:   https://github.com/caskroom/homebrew-cask#reporting-bugs /usr/local/Homebrew/Library/Homebrew/utils.rb:113:in odeprecated'
/usr/local/Homebrew/Library/Homebrew/utils.rb:121:in odisabled' /usr/local/Homebrew/Library/Homebrew/compat/extend/string.rb:3:in undent'
/usr/local/Homebrew/Library/Taps/mengbo/homebrew-ch340g-ch34g-ch34x-mac-os-x-driver/Casks/wch-ch34x-usb-serial-driver.rb:23:in block in load' /usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cask.rb:23:in instance_eval'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cask.rb:23:in initialize' /usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cask_loader.rb:31:in new'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cask_loader.rb:31:in cask' /usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cask_loader.rb:67:in cask'
/usr/local/Homebrew/Library/Homebrew/compat/hbc/cask_loader.rb:10:in cask' /usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cask_loader.rb:113:in cask'
/usr/local/Homebrew/Library/Taps/mengbo/homebrew-ch340g-ch34g-ch34x-mac-os-x-driver/Casks/wch-ch34x-usb-serial-driver.rb:1:in load' /usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cask_loader.rb:57:in instance_eval'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cask_loader.rb:57:in load' /usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cask_loader.rb:170:in load'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/abstract_command.rb:50:in block in casks' /usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/abstract_command.rb:50:in map'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/abstract_command.rb:50:in casks' /usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/install.rb:14:in run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/abstract_command.rb:35:in run' /usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:98:in run_command'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:168:in run' /usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:132:in run'
/usr/local/Homebrew/Library/Homebrew/cmd/cask.rb:8:in cask' /usr/local/Homebrew/Library/Homebrew/brew.rb:101:in '
Error: Kernel.exit
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:179:in exit' /usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:179:in rescue in run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:156:in run' /usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:132:in run'
/usr/local/Homebrew/Library/Homebrew/cmd/cask.rb:8:in cask' /usr/local/Homebrew/Library/Homebrew/brew.rb:101:in '
$

#### Output of `brew cask doctor`

$ brew cask doctor
==> Homebrew-Cask Version
Homebrew-Cask 1.6.0-2-g4682108
caskroom/homebrew-cask (git revision 14f44; last commit 2018-04-10)
==> macOS
10.13.4
==> SIP
Enabled
==> Java
N/A
==> Homebrew-Cask Install Location

==> Homebrew-Cask Staging Location
/usr/local/Caskroom
==> Homebrew-Cask Cached Downloads
~/Library/Caches/Homebrew/Cask
==> Homebrew-Cask Taps:
/usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask (3956 casks)
/usr/local/Homebrew/Library/Taps/mengbo/homebrew-ch340g-ch34g-ch34x-mac-os-x-driver (1 cask)
==> Contents of $LOAD_PATH
/usr/local/Homebrew/Library/Homebrew/cask/lib
/usr/local/Homebrew/Library/Homebrew
/Library/Ruby/Gems/2.3.0/gems/did_you_mean-1.0.0/lib
/Library/Ruby/Site/2.3.0
/Library/Ruby/Site/2.3.0/x86_64-darwin17
/Library/Ruby/Site/2.3.0/universal-darwin17
/Library/Ruby/Site
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/vendor_ruby/2.3.0
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/vendor_ruby/2.3.0/x86_64-darwin17
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/vendor_ruby/2.3.0/universal-darwin17
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/vendor_ruby
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/x86_64-darwin17
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/universal-darwin17
==> Environment Variables
LC_ALL="en_US.UTF-8"
PATH="/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/Homebrew/Library/Homebrew/shims/scm"
SHELL="/bin/bash"

#### Output of `brew tap`

$ brew tap
caskroom/cask
homebrew/core
mengbo/ch340g-ch34g-ch34x-mac-os-x-driver
$

Please always use fenced code blocks as they were. Removing them makes the issue difficult to read.
The issue is in mengbo/ch340g-ch34g-ch34x-mac-os-x-driver, not here. They need to remove the license line.


Add document for sticky consumer

Fixes #5682
Motivation
Add document for sticky consumer
Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (no)
The default values of configurations: (no)
The wire protocol: (no)
The rest endpoints: (no)
The admin cli options: (no)
Anything that affects deployment: (no)

@Anonymitaet Please help take a look.

Looking forward we have sticky consumer docs for other client libraries, such as Go, Python, C++, and so on. I'm glad to help as well.

Looking forward we have sticky consumer docs for other client libraries, such as Go, Python, C++, and so on. I'm glad to help as well.

Thanks, we have not support sticky consumer for other language client yet.

@codelipenghui could you please update the doc? Thank you

@codelipenghui  could you please update the doc? Thank you

@codelipenghui thanks for correcting all occurrences for Key_Shared.

run java8 tests

run java8 test

run java8 tests

run java8 tests
2019-11-28\T\06:46:08.382 [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 70.408 s <<< FAILURE! - in org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryChangeLedgerPathTest
2019-11-28\T\06:46:08.383 [ERROR] testChangeZKPath(org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryChangeLedgerPathTest)  Time elapsed: 0.151 s  <<< FAILURE!
java.io.IOException: Failed to initialize metadata client driver
	at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:453)
	at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:334)
	at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl$DefaultBkFactory.<init>(ManagedLedgerFactoryImpl.java:187)
	at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl.<init>(ManagedLedgerFactoryImpl.java:126)
	at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl.<init>(ManagedLedgerFactoryImpl.java:108)
	at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryChangeLedgerPathTest.testChangeZKPath(ManagedLedgerFactoryChangeLedgerPathTest.java:54)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:54)
	at org.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:44)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.bookkeeper.meta.exceptions.MetadataException: Failed to create zookeeper client to localhost:22439
	at org.apache.bookkeeper.meta.zk.ZKMetadataDriverBase.initialize(ZKMetadataDriverBase.java:217)
	at org.apache.bookkeeper.meta.zk.ZKMetadataClientDriver.initialize(ZKMetadataClientDriver.java:62)
	at org.apache.bookkeeper.client.BookKeeper.<init>(BookKeeper.java:443)
	... 17 more
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
	at org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase.waitForConnection(ZooKeeperWatcherBase.java:159)
	at org.apache.bookkeeper.zookeeper.ZooKeeperClient$Builder.build(ZooKeeperClient.java:257)
	at org.apache.bookkeeper.meta.zk.ZKMetadataDriverBase.initialize(ZKMetadataDriverBase.java:197)
	... 19 more

run java8 tests

run java8 tests

run java8 tests

run java8 tests

run cpp tests


Better docs combined

Thanks @tifa365! I am closing your PR because I've integrated your changes into the original PR, where I've continued work on this.

Thanks @tifa365! I am closing your PR because I've integrated your changes into the original PR, where I've continued work on this.


Minimap appearance and clickable areas are off

When I use the minimap on Chrome there is progressive mismatch between the ASCII map and the clickable rooms along the x axis. It gets to the point that the leftmost edge of the unconference lobby takes me to the theatre, and I can't get the unconference lobby until I'm at about the o/n on 'unconference'.
Travis tried on the main build and didn't see it, so, who even knows. Low priority since you can use the list as a backup.

Are you talking about the full map or the mini-map (which I think now should just open the map instead of letting you select things?)
What would be really helpful is if you can open up the Inspector, find the clickable areas (they're each individual divs that should be aligned) and taking a screenshot of a misaligned one

I wonder does it possibly just count the width of a character as too wide? It gets worse through the x axis and it seems like right from the start, the width is too big even though I believe the width character count is correct.

Cool, this is solvable.
We're not explicitly specifying a font for the map, and since it's a <pre> our global preset gets overriden by default to whatever the system monospace font is, which varies from OS to OS.
Manually specifying a font, and then fixing the sizing for that font, should do it. IBM Mono doesn't look great for this, so if I have time I'll find something else.

This definitely happened again in 2021 (I lampshaded it in Oracle). I mentioned it in our Discord at the time but not here, mostly because we were working pretty close to the wire.

To my instincts, this feels like a somewhat easy win for someone who can find the relevant CSS for the map modal, but willing to hear otherwise if need be.

As an implementation detail, this is going to be a fair bit more complex than just CSS. The clickable areas are generated via manual JS math that calculates grid offsets based on presumed font sizes of each individual character. Either it's as simple as saying "whoops, we said each character was N pixels wide but it's actually N+1", or this is a deep dark pit of sadness.
This ties into #371, but one potential solution would be to move the entire map over to a single static image using an old-school image map. We would still need manual pixel offsets to programmatically insert user counts (and potentially room names, I don't remember if we do that programmatically or in the art), but that might enable a simpler workflow. This map isn't designed to be accessible (it's why we provide the list of rooms as a fallback) so I don't think we'd necessarily lose much from having the text not be highlightable.


Upgrade Tensorflow to v1.0.0-rc2

Install guide

issues:
ERROR: /root/tensorflow/tensorflow/core/kernels/BUILD:1921:1: C++ compilation of rule '//tensorflow/core/kernels:svd_op' failed: gcc failed:_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -Wl,-z,-relro,-z,now -B/opt/rh/devtoolset-2/root/usr/bin -B/usr/bin -Wunused-but-set-pld.lib.shell.BadExitStatusException: Process exited with status 4.
gcc: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,

Workaround, increase Swap size:
dd if=/dev/zero of=/opt/swapfile bs=1024 count=5120000
mkswap /opt/swapfile -f
swapon /opt/swapfile

issue:
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.


Fix branch references in the R3 branch of the catalogue repo

/assign @radoslawc @efiacor @arora-sagar

/lgtm


Daily only gives hero xp if it succeeds

Somehow I don't think you were supposed to get xp on your hero if the daily failed.

I didn't know that was a thing, thank you


create a chart repository

[x] create a chart repository: https://helm.sh/docs/topics/chart_repository/
[x] use github pages in the same repo
[x] make the first release
[x] automatise the chart release in CI

done via https://github.com/bitnik/binder-launches/pull/7, https://github.com/bitnik/binder-launches/pull/8 and https://github.com/bitnik/binder-launches/pull/9


Creating MBP version of OSC

Now ready for review!
Implemented MBP version of the operational space controller.
Overall it is identical to the RBT version. The main differences are the different names for the same MBP and RBT method and the fact that MBP uses Context for computing kinematic methods instead of RBT's KinematicsCache.
Although not yet implemented, some optimizations that might be useful include storing the plant contexts instead of recreating them for each QP, and looking into different QP solvers (perhaps using multiple solvers in parallel and returning the first solution?)

Is this awaiting review? I think @yminchen should provide the code review, and I'll just give it a brief look when it's ready.

Once the drake-update branch is merged with master, this should be ready for review.

@yminchen Ready for review.

@yangwill Could you update your PR post (the first comment box), and also mention what the changes are aside from the RBT to MBP change? It will help me to review the PR faster and also make it clearer for whoever read this PR in the future. Thanks!

Ready for final review

I'm a little concerned about the file/class names here. Should we move the RBP version to /attic/ and then this won't need the _mbp prefix?


Union source data from multiple QuickBooks instances

Are you a current Fivetran customer?
Michael Maltese, Solution Engineer, Sigma Computing
What change(s) does this PR introduce?
Add ability to union from multiple data sources
Did you update the CHANGELOG?

[x] Yes

Does this PR introduce a breaking change?

[x] No
Backwards compatible

Did you update the dbt_project.yml files with the version upgrade (please leverage standard semantic versioning)? (In both your main project and integration_tests)

[x] Yes

Is this PR in response to a previously created Bug or Feature Request

[x] Yes, Feature #18

How did you test the PR changes?

[x] Local

Select which warehouse(s) were used to test the PR

[x] Snowflake

Provide an emoji that best describes your current mood
:bear:

Based off of https://github.com/fivetran/dbt_xero_source/pull/11

@ligfx this is an amazing feature addition to the package! I also really appreciate how you took the time in scaffolding the changes from our Xero integration here as well. 💯
At first glance these changes look great. I will take time over the next few weeks doing a deeper review of this PR. Additionally, we will want to update the dbt_quickbooks package to account for the union feature as well. Since there are quite a few joins that take place in that package I imagine the work there will be quite extensive and may take more time.
My team can plan for the work in the modeling package over the next few months. Otherwise, if you would like to attempt a PR on the modeling package I would be happy to assist in any way I can 😄. Otherwise, I will continue to share updates on the progress of this PR and the future PR here.
Again I want to thank you so much for putting the time into opening this PR. It is because of contributors like yourself that these packages continue to be improved upon! I look forward for when we get to integrate this into our package! 🏅

Awesome, thanks @fivetran-joemarkiewicz . I've started taking a peek at the dbt_quickbooks package too, though as you mentioned that's much more extensive.
This also rebases reasonably cleanly on top of the casting PR (just some whitespace errors and the changelog). I can update this PR once that one gets merged in.

Rebased on top of the latest main now that the casting fixes have landed.

@ligfx I wanted to chime back in here as we are looking to incorporate this union feature into the next release of the QuickBooks package!
We have integrated a few pieces already within our working branch, but want to include a number of your contributions. In order to do this I will need to edit your branch to address a number of merge conflicts. Is this branch by chance being used in production at all? If not, would you mind if I made a few edits to allow for this to be successfully merged into our updates branch and accept your contributions?

Hi Joe,
Glad to hear it! Please, go ahead and make any modifications necessary.
On Thu, Jan 19, 2023 at 6:05 PM Joe Markiewicz @.***>
wrote:

@ligfx https://github.com/ligfx I wanted to chime back in here as we
are looking to incorporate this union feature into the next release of the
QuickBooks package!
We have integrated a few pieces already within our working branch, but
want to include a number of your contributions. In order to do this I will
need to edit your branch to address a number of merge conflicts. Is this
branch by chance being used in production at all? If not, would you mind if
I made a few edits to allow for this to be successfully merged into our
updates branch and accept your contributions?
—
Reply to this email directly, view it on GitHub
https://github.com/fivetran/dbt_quickbooks_source/pull/26#issuecomment-1397723701,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAERBLN6A3Y47HIXFSBZ4CLWTHCBXANCNFSM5SULZSGQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Michael Maltese
(209) 404-5755
linkedin.com/in/mmaltese
New York, NY

Thanks again for all your help @ligfx!
We have applied your updates to our dbt_quickbooks_source package in our latest release. Definitely take a look at the new package and see if you're able to union your source data now! Thanks for all your contributions!


Trouble installing microtime dependency

Hi,
Congratulations on and thank you for the release of elm-markup v3! I've played around with the examples and it looks great :smiley:.
I'm unable to install this CLI tool, as microtime won't install. However, I'm not sure it's necessary anyway? As far as I can tell (from a quick search), it's only used in timeCheckpoint in runners/node/lib.js, and that function doesn't seem to be being called or exported?
More info on the failed install: despite this issue from the microtime repo, neither npm i<EMAIL_ADDRESS>nor npm i<EMAIL_ADDRESS>will install. I'm running node v12.3.1 and npm v6.9.0 on macOS 10.14.5.

Ah, interesting!  Yeah, I think that can be removed. I was doing some ad-hoc benchmarking at some point, but not anymore.

I just published a patch version with that removed.  Thanks for the well described issue!


Changed SearchAfter field type to FieldValue instead of String

Description

Changed SearchAfter of SearchRequest type to FieldValue instead of String
Written in the same way as other models receive various types

Issues Resolved
https://github.com/opensearch-project/opensearch-java/issues/755
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

I think this PR looks good, it's a breaking change for 3.0. For a backport to 2.x your plan LGTM.
How about we start introducing an UPGRADING.md that describes all things that change in a way that is backwards incompatible? @channel-dante Care to add an UPGRADING.md in the same format as https://github.com/opensearch-project/opensearch-py/blob/main/UPGRADING.md? And rebase.
Thank you for hanging in here with us!

@dblock Can you please confirm about this pr?

Merged, thanks @channel-dante. Want to do a manual backport for a backwards compatible version?

@dblock I didn't understand what kind of work it was.

Nevermind, I re-read the comments on bcw above. Thanks.


Remap key to mousewheel

Is it possible to remap a key to a mousewheel event?
I'm running this config file:
devices:
- input_name: 'Dell KB216 Wired Keyboard'
  input_phys: 'usb-0000:00:14.0-4/input0'
  output_name: remap-kbd
  remappings:
    KEY_W:
    - code: REL_WHEEL
      type: EV_REL
      repeat: true
      value: -1

And running with sudo python evdevremapkeys/evdevremapkeys.py -f config.yaml but it doesn't seem to work...

@cjh9 Take a look at evsieve.


{'event': 'error', 'data': {'message': 'An unexpected error occurred.', 'code': 18}}

when I run the code by command below:
#python3  ./downloader.py
it  shows me:
{'event': 'error', 'data': {'message': 'An unexpected error occurred.', 'code': 18}}
code line 18 is AUTH_TOKEN,I copy the AUTH_TOKEN right. I`m confused about this error.
please help me out ,many thks :0

same error receiving.

Same issue how to resolve.

Yesterday I could not download, although Authentication token and Recapcha was corrected. May the security of website has been changed.

Yes the code no work anymore

Is it such a coincidence? I just finished the PIL module problem. so pitiful am I :(

I'm also having the same error code as well :( Any solutions or fixes available?
TIA @evmer

Same issue here :-/ any solution?

Same here! Thank you for looking at it!

i'm having this issue a well btw.

@evmer  please fix it.

@jajosheni  @owohai  @evmer
please Dears can you fix this because its need to update i think.

Just an update. It seems like the issue is a ReCaptcha problem, since now I get this error:
{'event': 'error', 'data': {'message': 'Failed to validate recaptcha token', 'code': 6}}
Dunno how helpful this is. I've double-checked and I'm sure the ReCaptcha token I used was accurate, so I'm not really sure what the issue is.

Just an update. It seems like the issue is a ReCaptcha problem, since now I get this error: {'event': 'error', 'data': {'message': 'Failed to validate recaptcha token', 'code': 6}}
Dunno how helpful this is. I've double-checked and I'm sure the ReCaptcha token I used was accurate, so I'm not really sure what the issue is.

The error keep been {'event': 'error', 'data': {'message': 'An unexpected error occurred.', 'code': 18}}

Same problem... I hope there is a fix, thanks!

same problem

Same problem.....waiting for a fix......

Same problem here.

Same here

same issue here... any ideas? I was unable to find any information on that error number '18'...

if you run python3 in verbose mode:
python3 -v downloader.py
..you'll see that when it reaches that error, it appears to be due to a problem with the plugin of PyPDF2:
`import 'PyPDF2._version' # <_frozen_importlib_external.SourceFileLoader object at 0x10cafe0d0>
/usr/local/lib/python3.11/site-packages/PyPDF2/pycache/papersizes.cpython-311.pyc matches /usr/local/lib/python3.11/site-packages/PyPDF2/papersizes.py
code object from '/usr/local/lib/python3.11/site-packages/PyPDF2/pycache/papersizes.cpython-311.pyc'
import 'PyPDF2.papersizes' # <_frozen_importlib_external.SourceFileLoader object at 0x10cafe1d0>
import 'PyPDF2' # <_frozen_importlib_external.SourceFileLoader object at 0x10c5b5d10>
{'event': 'error', 'data': {'message': 'An unexpected error occurred.', 'code': 18}}
`
I'm not handy with Python, but it appears to me that we'd need to swap it with an updated PDF module [or a different one] for Python?  Looking through the docs on PyPDF2, I think it's breaking because perlego adjusted some of their responsive code for different devices ; but that's a speculation.

I have found out that the problem is caused when the script tries to load book pages: specifically when this line or similiar ones are called ws.send(json.dumps({"action":"loadPage","data":{"authToken": AUTH_TOKEN, "pageId": str(next_page), "bookType": book_format, "windowWidth":1792, "mergedChapterPartIndex":merged_chapter_part_idx}})). The action of loading page i think is changed on the server, because until the authentication everything works but I think they changed the info we need to send or they crypted, because I have analized book-delivery packet with devoloper options on chrome and I saw that the request sends ({"action":"loadPage","data":{some long string}}, I think it's in base64 but I can't decode. Actually it sends three diffrent request same to the last one that all starts with the same 9 characters, and based on the response one is for fonts, one is the content, and one I didn't understand. So we need to figure out what they want and we can get back this great script, but I'm a noob so I don't know how to decrypt the string or what they need

Ditto...
{'event': 'error', 'data': {'message': 'An unexpected error occurred.', 'code': 18}}

I have found out that the problem is caused when the script tries to load book pages: specifically when this line or similiar ones are called ws.send(json.dumps({"action":"loadPage","data":{"authToken": AUTH_TOKEN, "pageId": str(next_page), "bookType": book_format, "windowWidth":1792, "mergedChapterPartIndex":merged_chapter_part_idx}})). The action of loading page i think is changed on the server, because until the authentication everything works but I think they changed the info we need to send or they crypted, because I have analized book-delivery packet with devoloper options on chrome and I saw that the request sends ({"action":"loadPage","data":{some long string}}, I think it's in base64 but I can't decode. Actually it sends three diffrent request same to the last one that all starts with the same 9 characters, and based on the response one is for fonts, one is the content, and one I didn't understand. So we need to figure out what they want and we can get back this great script, but I'm a noob so I don't know how to decrypt the string or what they need

OK I think it's impossible to decrypt the data cause is crypted with CryptoJS3 using EvpKDF so we need the cipher and the salt to decrypt but I think it's server-side so it's impossible. But I'm a super noob so if anyone finds something else or knows how to retrive the cipher would be great cause this script was very useful, and saved me a lot of time compared to my download script

I looked at it some more and I really don't think that's what it is, and it wouldn't be impossible because the browser has to render it [eventually] ; it'd be only impossible if we couldn't see it via Chrome etc. and we were trying to somehow intercept a transmission.  Instead, the script looks to "piggy back" on an established connection and simply automate something that could be done by hand with copy & paste - it really doesn't do anything more than that.  Being that that technique is tedious, it was hopeful that this script could eliminate some of the time burden.  Another approach would be to use a browser-automation tool to do it, but therein lies a lot of work to set that up and sharing it isn't as convenient as it is with Python or another script.
Rerun this in -v or verbose mode with python and trace it back and you'll see that it gets stuck when this calls upon PyPDF to render the window (the page width 1792, for example, I don't know where the original dev of this script came up with that).  It appears to me like it's breaking in the presentation layer, not the data layer, because if you modify the script to simply get the output without doing anything, it's capable of connecting to that websocket (it is for me at least).  So there's something that's changed with Perlego's presentation layer, likely some sort of responsiveness upgrade - that appears to have broken this.
I have a question though since I've let my Perlego account lapse.  Perlego claims that the "app" allows for many of it's books to be downloaded offline in PDF form.  Has anyone tried an iPhone or Android emulator (or even a real phone or device) and tried to just use a standard DMCA removal tool (like Epubor Ultimate) to get them that way?  It'd involve downloading the file, copying it (easier to do from an Android) to a computer, and then removing the DMCA from that file.  This seems like this would work; I don't believe they'd be using a non-standard PDF encryption since the whole point of PDF is it's ability to do that.. otherwise, the would have come up with their own bizarre proprietary format (like Amazon and it's AZW3 or somesuch)

Hey jasonCbraatz and to all the others who contributed to this issue
I believe as Jason mentioned: Perlego’s presentation layer has changed
radically and from what I’ve seen it’s even harder to copy the contents of
what’s stored within
A possible fix would be probably a browser  extension that that’s
screenshots of each page but the main reason why people use this is due to
its ability to quite literally copy the contents and dump it in a text form
(or at least that’s why I liked the script in the first place)
I’ll be brushing through the code for the coming weeks to find possible
fixes but other than that, if anyone could chip in and give us clues and
other things that may help I’ll be really grateful :)
On Sun, 21 May 2023 at 12:00 AM jasonCbraatz @.***>
wrote:

I looked at it some more and I really don't think that's what it is, and
it wouldn't be impossible because the browser has to render it
[eventually] ; it'd be only impossible if we couldn't see it via Chrome
etc. and we were trying to somehow intercept a transmission. Instead, the
script looks to "piggy back" on an established connection and simply
automate something that could be done by hand with copy & paste - it really
doesn't do anything more than that. Being that that technique is tedious,
it was hopeful that this script could eliminate some of the time burden.
Another approach would be to use a browser-automation tool to do it, but
therein lies a lot of work to set that up and sharing it isn't as
convenient as it is with Python or another script.
Rerun this in -v or verbose mode with python and trace it back and you'll
see that it gets stuck when this calls upon PyPDF to render the window (the
page width 1792, for example, I don't know where the original dev of this
script came up with that). It appears to me like it's breaking in the
presentation layer, not the data layer, because if you modify the script to
simply get the output without doing anything, it's capable of connecting to
that websocket (it is for me at least). So there's something that's changed
with Perlego's presentation layer, likely some sort of responsiveness
upgrade - that appears to have broken this.
I have a question though since I've let my Perlego account lapse. Perlego
claims that the "app" allows for many of it's books to be downloaded
offline in PDF form. Has anyone tried an iPhone or Android emulator (or
even a real phone or device) and tried to just use a standard DMCA removal
tool (like Epubor Ultimate) to get them that way? It'd involve downloading
the file, copying it (easier to do from an Android) to a computer, and then
removing the DMCA from that file. This seems like this would work; I don't
believe they'd be using a non-standard PDF encryption since the whole point
of PDF is it's ability to do that.. otherwise, the would have come up with
their own bizarre proprietary format (like Amazon and it's AZW3 or somesuch)
—
Reply to this email directly, view it on GitHub
https://github.com/evmer/perlego-downloader/issues/67#issuecomment-1556002560,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHGQVR2FHURVJ3CZ6O2HB5LXHEWFLANCNFSM6AAAAAAXP6VOTU
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Dr. Meer
P.O.Box 149
Riyadh, KSA 11323

I looked at it some more and I really don't think that's what it is, and it wouldn't be impossible because the browser has to render it [eventually] ; it'd be only impossible if we couldn't see it via Chrome etc. and we were trying to somehow intercept a transmission. Instead, the script looks to "piggy back" on an established connection and simply automate something that could be done by hand with copy & paste - it really doesn't do anything more than that. Being that that technique is tedious, it was hopeful that this script could eliminate some of the time burden. Another approach would be to use a browser-automation tool to do it, but therein lies a lot of work to set that up and sharing it isn't as convenient as it is with Python or another script.
Rerun this in -v or verbose mode with python and trace it back and you'll see that it gets stuck when this calls upon PyPDF to render the window (the page width 1792, for example, I don't know where the original dev of this script came up with that). It appears to me like it's breaking in the presentation layer, not the data layer, because if you modify the script to simply get the output without doing anything, it's capable of connecting to that websocket (it is for me at least). So there's something that's changed with Perlego's presentation layer, likely some sort of responsiveness upgrade - that appears to have broken this.
I have a "theory" on what could be a workaround/another way to do this but I can't test it since I've let my Perlego account lapse. Perlego claims that it's "app" allows for many of it's books to be downloaded offline in PDF form. Has anyone tried an iPhone or Android emulator (or even a real phone or device), downloaded a book, and then tried to just use a standard DMCA removal tool (like Epubor Ultimate) to get them that way? It'd involve downloading the file, copying it (easier to do from an Android) to a computer, and then removing the DMCA from that file. This seems like this would work; I don't believe they'd be using a non-standard PDF encryption since the whole point of PDF is it's ability to do that.. otherwise, the would have come up with their own bizarre proprietary format (like Amazon and it's AZW3 or somesuch)

No as I said and also you can verify if you are able to create a perlego account using chrome developer option now to render the page u need to send a request where the data is encrypted as I said, before as it's written in the source code of the script we had to send the Authtoken the pageid and so on, if you want to confirm that just look at the video explanation that is mentioned in the readme of this project, u can see in the chrome devoloper option under network tab that the request loadpage is changed from "action":"loadPage","data":{"authToken": AUTH_TOKEN, "pageId": 1, "bookType": pdf, "windowWidth":1792, "mergedChapterPartIndex": 1as showed in the video to  "action":"loadPage","data":{"long string" } as you can see on your chrome, also if you run one by one the commands of the script you can see that when you use the command ws.send(json.dumps({"action":"loadPage","data":{"authToken": AUTH_TOKEN, "pageId": str(next_page), "bookType": book_format, "windowWidth":1792, "mergedChapterPartIndex":merged_chapter_part_idx}})) the response is the same as the issue here, so definitely it's this the problem.
The error of pypdf maybe is caused from the fact that it was deprecated a month ago, but it's Very easy to fix just switch to the new library.
I've also tried to use the mobile app but for pdf u download images and htmls that are encrypted and I wasn't able to decrypt, I also analyzed the traffic and also there they encrypted the request.
As I said I created my own script to download pages using selenium, but it takes like 10/15 minutes for a 400 pages book and the pdf creation it's sometimes broken but I'm trying to fix it.
I don't think we will be able to get the script back as it was.

@mattiapergola mind to share the selenium script? ty

sometime, would love to get the take by @evmer - the originator of the code ; seeing the above, any other ideas that haven't been explored thus far?

I think I found out how to reverse the encryption. I need someone to help me finalized it, as I am just a beginner in web development.

@Oredna I'm happy to help, did you end up using a different library?

@Oredna I'm happy to help, did you end up using a different library?

Please provide me with some communication chanel I shout contact to get in touch with you.

@Oredna thanks for the reply, if you guys want we can try to collaborate and make the script back to work. If you want join https://join.slack.com/t/makeperlegodo-urb3544/shared_invite/zt-1w32hpjf9-Cg4vslya1menaLewHQkhtA

@Oredna I'm happy to help, did you end up using a different library?

Did you solve the error?

@Oredna I'm happy to help, did you end up using a different library?

Did the solve the problem?

@Oredna I'm happy to help, did you end up using a different library?

Did you solve the problem?

Anyone solved the problem?

is there an alternative code to download ebooks from perlego?

no fix yet? :(

@NukosX Ok sorry.

I changed the ({"action":"loadPage","data":{"authToken": AUTH_TOKEN, "pageId": str(next_page), "bookType": book_format, "windowWidth":1792, "mergedChapterPartIndex":merged_chapter_part_idx}})) to ({"action":"loadPage","data":inputvariable}))
The inputvarible is an input variable that I enter the value of the loadpage data that appears in the console. The code worked but it kept downloading the same page/s.

Might be the below key to be considered to insert in the code:
{"action":"loadPage","data":""}

also have error code 18, dear author of this code it would be wonderful if you can help us unlock, I'm doing research and need to access these books' pdf for notes taking, thank you!

also have error code 18, dear author of this code it would be wonderful if you can help us unlock, I'm doing research and need to access these books' pdf for notes taking, thank you!

Perhaps you should dive in and fix it yourself. This is FREE SOFTWARE that is COMMUNITY SUPPORTED. Join the community and pitch in. Otherwise, keep your criticisms to yourself.

looks like perlego altered the way it fetch data from the server. now for each chapter in the event
{
    "action": "loadPage",
    "data": "longString"
}

it sends a long string, after some work, i found that this string/token does not expire and often same string/token works for several book
you can filter out this event loadPage and copy data values for each chapter.

s.
place these strings/tokens in TOKEN_LIST list , modified in this MR :
https://github.com/evmer/perlego-downloader/pull/76
this is sometimes a tiresome process, specially for books with lots of chapters but it works.
i was unable to find the source of this string/token so this is the best I've got so far. Feel free to improve this .
Happy reading and Happy Coding :)

looks like perlego altered the way it fetch data from the server. now for each chapter in the event
{
    "action": "loadPage",
    "data": "longString"
}

it sends a long string, after some work, i found that this string/token does not expire and often same string/token works for several book
you can filter out this event loadPage and copy data values for each chapter.
 s.
place these strings/tokens in TOKEN_LIST list , modified in this PR : #76
this is sometimes a tiresome process, specially for books with lots of chapters but it works. i was unable to find the source of this string/token so this is the best I've got so far. Feel free to improve this .
Happy reading and Happy Coding :)

Have you tried this? the values of load page data changes every time when new page/chapter is opened. Tried to run the code but error remained same.

looks like perlego altered the way it fetch data from the server. now for each chapter in the event
{
    "action": "loadPage",
    "data": "longString"
}

it sends a long string, after some work, i found that this string/token does not expire and often same string/token works for several book
you can filter out this event loadPage and copy data values for each chapter.
 s.
place these strings/tokens in TOKEN_LIST list , modified in this PR : #76
this is sometimes a tiresome process, specially for books with lots of chapters but it works. i was unable to find the source of this string/token so this is the best I've got so far. Feel free to improve this .
Happy reading and Happy Coding :)

Have you tried this? the values of load page data changes every time when new page/chapter is opened. Tried to run the code but error remained same. First one needs to understand the changes implemented by the API. Seems approach to be changed as well then code it to do the task.

Yes, the code works, i am using it.you need to copy data values for all pages/chapters and copy them in the TOKEN_LIST array . on refresh or reloading the same page, the token changes but the previous one also works

Yes, the code works, i am using it.you need to copy data values for all pages/chapters and copy them in the TOKEN_LIST array . on refresh or reloading the same page, the token changes but the previous one also works

so i need if there are 300 pages in a book then i have to copy data values of every page one by one?

Yes, the code works, i am using it.you need to copy data values for all pages/chapters and copy them in the TOKEN_LIST array . on refresh or reloading the same page, the token changes but the previous one also works

so i need if there are 300 pages in a book then i have to copy data values of every page one by one?

i believe you can find some better way to copy 300 values, and it works with chapters not pages.
i filter the event with loadPage and copy all the values at once, you can write some script to get the values but that's up for improvement . i couldn't find any other solution to the issue

looks like perlego altered the way it fetch data from the server. now for each chapter in the event
{
    "action": "loadPage",
    "data": "longString"
}

it sends a long string, after some work, i found that this string/token does not expire and often same string/token works for several book
you can filter out this event loadPage and copy data values for each chapter.
 s.
place these strings/tokens in TOKEN_LIST list , modified in this PR : #76
this is sometimes a tiresome process, specially for books with lots of chapters but it works. i was unable to find the source of this string/token so this is the best I've got so far. Feel free to improve this .
Happy reading and Happy Coding :)

Have you tried this? the values of load page data changes every time when new page/chapter is opened. Tried to run the code but error remained same. First one needs to understand the changes implemented by the API. Seems approach to be changed as well then code it to do the task.

Yes, the code works, i am using it.you need to copy data values for all pages/chapters and copy them in the TOKEN_LIST array . on refresh or reloading the same page, the token changes but the previous one also works

By placing the different tokens of the chapters, I get the following error when executing:

download error: Connection timed out

It works, but unfortunately, is not scalable. I see more viable to just take a SS for every page, join and transform them to pdf.

looks like perlego altered the way it fetch data from the server. now for each chapter in the event
{
    "action": "loadPage",
    "data": "longString"
}

it sends a long string, after some work, i found that this string/token does not expire and often same string/token works for several book
you can filter out this event loadPage and copy data values for each chapter.
 s.
place these strings/tokens in TOKEN_LIST list , modified in this PR : #76
this is sometimes a tiresome process, specially for books with lots of chapters but it works. i was unable to find the source of this string/token so this is the best I've got so far. Feel free to improve this .
Happy reading and Happy Coding :)

Have you tried this? the values of load page data changes every time when new page/chapter is opened. Tried to run the code but error remained same. First one needs to understand the changes implemented by the API. Seems approach to be changed as well then code it to do the task.

Yes, the code works, i am using it.you need to copy data values for all pages/chapters and copy them in the TOKEN_LIST array . on refresh or reloading the same page, the token changes but the previous one also works

Hi can you guide me where to put that code. I am bit confuse

looks like perlego altered the way it fetch data from the server. now for each chapter in the event
{
    "action": "loadPage",
    "data": "longString"
}

it sends a long string, after some work, i found that this string/token does not expire and often same string/token works for several book
you can filter out this event loadPage and copy data values for each chapter.
 s.
place these strings/tokens in TOKEN_LIST list , modified in this PR : #76
this is sometimes a tiresome process, specially for books with lots of chapters but it works. i was unable to find the source of this string/token so this is the best I've got so far. Feel free to improve this .
Happy reading and Happy Coding :)

Have you tried this? the values of load page data changes every time when new page/chapter is opened. Tried to run the code but error remained same. First one needs to understand the changes implemented by the API. Seems approach to be changed as well then code it to do the task.

Yes, the code works, i am using it.you need to copy data values for all pages/chapters and copy them in the TOKEN_LIST array . on refresh or reloading the same page, the token changes but the previous one also works

Hi can you guide me where to put that code. I am bit confuse?

sure @dhakalanamol . you need to pull code from this PR
https://github.com/evmer/perlego-downloader/issues/67#issuecomment-1698710927
and put the tokens in this list TOKEN_LIST

you only need to add tokens for different chapters and not pages, code will do the rest

@usama455  i updated all lines according to your file and remove lines which you point out .
can you tell me please where is this loadpage value i can see ?
i am trying to find it in firefox console but i cannot find. or any easy way to find it ? i mean any software which can inspect and find it more fast and easily ?

@usama455 i got the loadpage data value but after update file i am getting this error can you help me to fix it.
PS C:\Users\Hp\Downloads\Compressed\perlego-downloader-main\perlego-downloader-main> python downloader.py Traceback (most recent call last): File "C:\Users\Hp\Downloads\Compressed\perlego-downloader-main\perlego-downloader-main\downloader.py", line 13, in import fitz File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\fitz__init__.py", line 1, in from frontend import * File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\frontend__init__.py", line 1, in from .events import * File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\frontend\events__init__.py", line 1, in from .clipboard import * File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\frontend\events\clipboard.py", line 2, in from ..dom import Event File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\frontend\dom.py", line 439, in from . import dispatcher File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\frontend\dispatcher.py", line 15, in from . import config, server File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\frontend\server.py", line 24, in app.mount(config.STATIC_ROUTE, StaticFiles(directory=config.STATIC_DIRECTORY), name=config.STATIC_NAME) File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\starlette\staticfiles.py", line 57, in init raise RuntimeError(f"Directory '{directory}' does not exist") RuntimeError: Directory 'static/' does not exist PS C:\Users\Hp\Downloads\Compressed\perlego-downloader-main\perlego-downloader-main>

can you share your code?

@usama455 can you help me to fix error ? please see my second last comment

@usama455 i got the loadpage data value but after update file i am getting this error can you help me to fix it.
PS C:\Users\Hp\Downloads\Compressed\perlego-downloader-main\perlego-downloader-main> python downloader.py Traceback (most recent call last): File "C:\Users\Hp\Downloads\Compressed\perlego-downloader-main\perlego-downloader-main\downloader.py", line 13, in import fitz File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\fitz__init__.py", line 1, in from frontend import * File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\frontend__init__.py", line 1, in from .events import * File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\frontend\events__init__.py", line 1, in from .clipboard import * File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\frontend\events\clipboard.py", line 2, in from ..dom import Event File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\frontend\dom.py", line 439, in from . import dispatcher File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\frontend\dispatcher.py", line 15, in from . import config, server File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\frontend\server.py", line 24, in app.mount(config.STATIC_ROUTE, StaticFiles(directory=config.STATIC_DIRECTORY), name=config.STATIC_NAME) File "C:\Users\Hp\AppData\Local\Programs\Python\Python310\lib\site-packages\starlette\staticfiles.py", line 57, in init raise RuntimeError(f"Directory '{directory}' does not exist") RuntimeError: Directory 'static/' does not exist PS C:\Users\Hp\Downloads\Compressed\perlego-downloader-main\perlego-downloader-main>

can you share your code?

Same error for me, tried with some chapter token to understand the behavior but it give the exact same error. The code is the new one of @usama455 with the TOKEN_LIST array.

Try installing these dependencies;
aiofiles==23.2.1
anyio==4.0.0
appdirs==1.4.4
certifi==2023.7.22
charset-normalizer==2.1.1
ci-info==0.3.0
click==8.1.7
configobj==5.0.8
configparser==6.0.0
etelemetry==0.3.0
exceptiongroup==1.1.3
filelock==3.12.3
frontend==0.0.1
future==0.18.3
h11==0.14.0
httplib2==0.22.0
idna==3.4
importlib-metadata==6.8.0
isodate==0.6.1
itsdangerous==2.1.2
looseversion==1.3.0
lxml==4.9.3
networkx==3.1
nibabel==5.1.0
nipype==1.8.6
numpy==1.25.2
packaging==23.1
pandas==2.1.0
pathlib==1.0.1
Pillow==8.4.0
prov==2.0.0
pydot==1.4.2
pyee==8.2.2
PyMuPDF==1.23.3
PyMuPDFb==1.23.3
pyparsing==3.1.1
PyPDF2==1.28.6
pyppeteer==1.0.2
python-dateutil==2.8.2
pytils==0.4.1
pytz==2023.3.post1
pyxnat==1.6
rdflib==7.0.0
requests==2.28.1
scipy==1.11.2
simplejson==3.19.1
six==1.16.0
sniffio==1.3.0
starlette==0.31.1
tools==0.1.9
tqdm==4.66.1
traits==6.3.2
typing_extensions==4.7.1
tzdata==2023.3
urllib3==1.26.16
uvicorn==0.23.2
websocket-client==1.4.0
websockets==10.4
zipp==3.16.2
And here is my modified code for download.py
`from io import BytesIO
from PIL import Image
import asyncio
import shutil
import base64
import shutil
import time
import json
import sys
import ssl
import re
import os
import fitz
import requests
import websocket
from pyppeteer import launch
from PyPDF2 import PdfMerger
from token_list import TOKEN_LIST
AUTH_TOKEN = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjoxOTY2Mjc0NywidG9rZW4iOiJjOTY0Zjg2YS1kNDc5LTQ0MjMtYTJmYS0wOTc1OTJhODAwYzYiLCJwZXJzaXN0ZW50Ijp0cnVlLCJpYXQiOjE2OTQ0NDE3OTR9.cGUF0u6Sx_HTTkWfGgR9S_a_jiafQa1qEYDvO_qqr9V0c1opWrwEWXWvlRorXyPbowOhzH7zqiQpaiuMN5Gq6lOk96xYhn3pd75beTkBuJzhmtMuSqjLQloIOtCekgiqblQ4wFdCkMjelzK-P-2p5bZaWWXTIjEboDTR2vMeSBQ622Pir3fVOTaqpFkaXIFQCKYLTYs8VYjnHAE7VVANeKaJPJB_pj2Ngkc0K7blrRzp0WhaclWofmRNsW2Q3qej1c8EP4w835utjCtyAYHPuvdHD-X8FF2snntr4FG_33ZPN8k3M_xEZkSHlADLeXV-GQqhDEQ_kGZ1739QX9b-6k19QdvxRiMDMZIWQniQH71YZNLcmv4RJHLQM0VDi-irH3rKYGxYCX3CZ5je386SfsKz5MsaSXq0kYnte8J_RsD0rw_okGQ_IEg-V8t3hutFNuyfGBoz5uj5vL9BgCMkWg_k5dcqUBP3ZACiHT4y73EiOwFXwdoc4sPrjFu-CagzM3TnyQCRCJaICczy-JPSTXhRBCfXJDb1xueZQl-pUwW7Cg4a9zJC6v8r3BN1piocWuAjtC2W0Jt0n3z08Kmu8zk7OnWn8KuzwIEOVt6Bxu_UHJXIBWF6ZDZCSA_uYLaHmL6LeswvJw7IRAsVW1PYNfyqVkJ0GAsdOcCumoOY2DQ"
BOOK_ID = "3866026"
RECAPTCHA = "03AFcWeA5frMvhJ257qcHPYmU58aksp3CviRYMMCBx2MlWgh5EQV7EtmZtuTOCr6NB15WXQ0vffIi08PN3COOkVlw2nZBRpLsa7f52M8TVLyeRZlLoqQh9f5x6UItBKSjnEqD55-Clmsy4EIMjpsmcGCr-bLAPWjpzuP5vvZyTrfrS9zXOAthLiv_rSkI305aMKF0va-kOBB1C3yacXCcHV0TZvghAFOx7okK9gld1l2hhyKCimD09F0hZ9yN6liwwoBQeuR2w8MZi98E4lKiVxMg99grZn9V0CrDUj2cW5ShayK5u-3AEhUvxt10Rd16tnxQ4-ujUvy0L4bCLolPqUkjfcRWrE6fc1lNCAhdDD4B_Lx2U9bz8z1uYrsYk3-CE7ApgyU8CFcERIORixOy5ehFELQsrHsh_ftaIIsqOL707f9EiuCxnjZCWs__HN7i7LvMOnt_WvHFCBRWcIWFCYi0XtohZWLVl-d4bs55YlneHXE5NBmhNf4E4gZzkHBGUqQtByV5S5DJTepThL6BVFdfcn4mcYmh8KFKOYBSBNr1Rx5g2sXsln54tp1-MWPpqPXGaWaSyj8Ei"
PUPPETEER_THREADS = 50
tokenIndex = 0
def init_book_delivery():
while True:
try:
ws = websocket.create_connection("wss://api-ws.perlego.com/book-delivery/", skip_utf8_validation=True, timeout=30,  sslopt={"cert_reqs": ssl.CERT_NONE, "check_hostname": False})
except Exception as error:
print(f'init_book_delivery() error: {error}')
continue
break
time.sleep(1)

ws.send(json.dumps({"action":"initialise","data":{"authToken": AUTH_TOKEN, "reCaptchaToken": RECAPTCHA, "bookId": str(BOOK_ID)}}))

return ws

class merged_chapter:
def init(self):
self.merged_chapter_number = 1
class chapter:
def init(self):
self.page_id = 1
	self.contents = {}

download pages content
while True:
chapters = {}
contents = {}
page_id = None
ws = init_book_delivery()

init_data = {}

while True:
	try:
		data = json.loads(ws.recv())
	except Exception as error:
		print(f'download error: {error}')
		ws = init_book_delivery()
		continue

	if data['event'] == 'error':
		sys.exit(data)

	elif data['event'] == 'initialisationDataChunk':
		if page_id != None: # we're here because ws conn broke, so we can resume from last page_id
			ws.send(json.dumps({"action":"loadPage","data":{"authToken": AUTH_TOKEN, "pageId": page_id, "bookType": book_format, "windowWidth":1792, "mergedChapterPartIndex":0}}))
			merged_chapter_part_idx = 0
			# reset latest content
			contents[page_id] = {}
			for i in chapters[page_id]: contents[i] = {}
			continue

		chunk_no = data['data']['chunkNumber']
		init_data[chunk_no] = data['data']['content']

		# download all the chunks before proceeding
		if len(init_data) != data['data']['numberOfChunks']: continue

		# merge the initialisation content
		data_content = ""
		for chunk_no in sorted(init_data):
			data_content += init_data[chunk_no]

		# extract the relevant data
		data_content = json.loads(json.loads(data_content))
		book_format = data_content['bookType']
		merged_chapter_part_idx = 0

		if book_format == 'EPUB':
			bookmap = data_content['bookMap']
			for chapter_no in bookmap:
				chapters[int(chapter_no)] = []
				contents[int(chapter_no)] = {}
				for subchapter_no in bookmap[chapter_no]:
					chapters[int(chapter_no)].append(subchapter_no)
					contents[subchapter_no] = {}
		elif book_format == 'PDF':
			for i in range(1, data_content['numberOfChapters'] + 1):
				chapters[i] = []
				contents[i] = {}
		else:
			raise Exception(f'unknown book format ({book_format})!')

		ws.send(json.dumps({"action":"loadPage","data":TOKEN_LIST[0]}))

	elif 'pageChunk' in data['event']:
		page_id = int(data['data']['pageId'])

		merged_chapter_no = (int(data['data']['mergedChapterNumber']) - 1) if book_format == 'EPUB' else 0
		number_of_merged_chapters = int(data['data']['numberOfMergedChapters']) if book_format == 'EPUB' else 1

		chunk_no = int(data['data']['chunkNumber']) - 1
		number_of_chunks = int(data['data']['numberOfChunks'])

		chapter_no = page_id + merged_chapter_no + merged_chapter_part_idx

		if contents.get(chapter_no) == None:
			contents[chapter_no] = {}
			chapters[page_id].append(chapter_no)

		if contents[chapter_no] == {}:
			for i in range(number_of_chunks):
				contents[chapter_no][i] = ""

		contents[chapter_no][chunk_no] = data['data']['content']

		# check if all merged chapters have been downloaded
		if not all(contents.get(i) not in [None, {}] for i in range(page_id, page_id+number_of_merged_chapters+merged_chapter_part_idx)): continue

		# check if all chunks of all merged pages/chapters have been downloaded
		if not all( all(chunk != "" for chunk in contents[i].values() ) for i in range(page_id, page_id+number_of_merged_chapters+merged_chapter_part_idx)): continue

		# check if all pages/chapters have been downloaded
		if all(contents[i] != {} for i in [page_id]+chapters[page_id]):

			print(f"{'chapters' if book_format == 'EPUB' else 'page'} {page_id}-{page_id+number_of_merged_chapters+merged_chapter_part_idx} downloaded")
			merged_chapter_part_idx = 0
			try:
				next_page = list(chapters)[list(chapters).index(page_id) + 1]
			except IndexError:
				break
		else:
			merged_chapter_part_idx += 1
			next_page = page_id

		ws.send(json.dumps({"action":"loadPage","data":TOKEN_LIST[tokenIndex-1]}))
		tokenIndex = tokenIndex + 1
		if (tokenIndex == len(TOKEN_LIST)):
			break
break

create cache dir
cache_dir = f'{os.getcwd()}/{book_format}_{BOOK_ID}/'
try:
os.mkdir(cache_dir)
except FileExistsError:
pass
convert html files to pdf
async def html2pdf():
# start headless chrome
browser = await launch(options={
		'headless': True,
		'autoClose': False,
		'args': [
			'--no-sandbox',
			'--disable-setuid-sandbox',
			'--disable-dev-shm-usage',
			'--disable-accelerated-2d-canvas',
			'--no-first-run',
			'--no-zygote',
			'--single-process',
			'--disable-gpu',
			'--disable-web-security',
			'--webkit-print-color-adjust',
			'--disable-extensions'
		],
	},
)

async def render_page(chapter_no, semaphore):
	async with sem:
		page = await browser.newPage()
		await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36')

		# download cover separately
		if chapter_no == 0:
			r = requests.get(f"https://api.perlego.com/metadata/v2/metadata/books/{BOOK_ID}")
			cover_url = json.loads(r.text)['data']['results'][0]['cover']
			img = Image.open(BytesIO(requests.get(cover_url).content))
			img.save(f'{cache_dir}/0.pdf')
			return

		# merge chunks
		content = ""
		for chunk_no in sorted(contents[chapter_no]):
			content += contents[chapter_no][chunk_no]

		# remove useless img (mess up with pdf gen)
		if book_format == 'EPUB':
			match = re.search('<img id="trigger" data-chapterid="[0-9]*?" src="" onerror="LoadChapter\(\'[0-9]*?\'\)" />', content)
			if match:
				content = content.replace(match.group(0), '')

		# reveal hidden images
		imgs = re.findall("<img.*?>", content, re.S)
		for img in imgs:
			img_new = img.replace('opacity: 0', 'opacity: 1')
			img_new = img_new.replace('data-src', 'src')
			content = content.replace(img, img_new)

		# save page in the cache dir
		f = open(f'{cache_dir}/{chapter_no}.html', 'w', encoding='utf-8')
		f.write(content)
		f.close()

		# render html
		await page.goto(f'file://{cache_dir}/{chapter_no}.html', {"waitUntil" : ["load", "domcontentloaded", "networkidle0", "networkidle2"], "timeout": 0})

		# set pdf options
		options = {'path': f'{cache_dir}/{chapter_no}.pdf'}
		if book_format == 'PDF':
			width, height = await page.evaluate("() => { return [document.documentElement.offsetWidth + 1, document.documentElement.offsetHeight + 1]}")
			options['width'] = width
			options['height'] =  height
		elif book_format == 'EPUB':
			options['margin'] = {'top': '20', 'bottom': '20', 'left': '40', 'right': '40'}

		# build pdf
		await page.pdf(options)
		await page.close()

sem = asyncio.Semaphore(PUPPETEER_THREADS)
await asyncio.gather(*[render_page(chapter_no, sem) for chapter_no in contents if not os.path.exists(f'{cache_dir}/{chapter_no}.pdf')])

asyncio.run(html2pdf())
merge pdfs
rel = requests.get(f"https://api.perlego.com/metadata/v2/metadata/books/{BOOK_ID}")
book_title = json.loads(rel.text)['data']['results'][0]['title']
print('merging pdf pages...')
merger = PdfMerger()
contents_value = contents  # Copy the value of 'contents'
contents_type = type(contents)  # Get the type of 'contents'
keys_list = list(contents.keys())
for chapter_no in sorted(contents):
chapter_pdf_path = os.path.join(cache_dir, f'{chapter_no}.pdf')
# merger.append(f'{cache_dir}/{chapter_no}.pdf')

pdf_document = fitz.open(chapter_pdf_path)
first_page = pdf_document[0]
lines = first_page.get_text("text").strip().split('\n')
first_line = lines[0].strip()
second_line = lines[1].strip() if len(lines) > 1 else ""
isFirstLine = True
is_empty_page = True
for page_num in range(pdf_document.page_count):
	page = pdf_document[page_num]
	text = page.get_text("text")
	if text.strip():
		is_empty_page = False
		break
if not is_empty_page:
	if first_line:
		isFirstLine = False
	if isFirstLine:
		chapter_title = first_line
	elif second_line:
		chapter_title = second_line
	else:
		chapter_title = f"Chapter {chapter_no}"
	merger.append(chapter_pdf_path)

pdf_document.close()

merger.write(f"output.pdf")
merger.close()
delete cache dir
shutil.rmtree(f'{book_format}_{BOOK_ID}')`
Note that you will need to create a file that is named token_list.py that contains the following;
TOKEN_LIST = [
# INSERT TOKENS HERE. F12 -> Network -> WS -> filter: loadPage
]
NB: i know my styleguide for replying is garbage :)

`
from io import BytesIO
from PIL import Image
import asyncio
import shutil
import base64
import shutil
import time
import json
import sys
import ssl
import re
import os
import fitz
import requests
import websocket
from pyppeteer import launch
from PyPDF2 import PdfMerger
from token_list import TOKEN_LIST
AUTH_TOKEN = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjoxOTY2Mjc0NywidG9rZW4iOiJjOTY0Zjg2YS1kNDc5LTQ0MjMtYTJmYS0wOTc1OTJhODAwYzYiLCJwZXJzaXN0ZW50Ijp0cnVlLCJpYXQiOjE2OTQ0NDE3OTR9.cGUF0u6Sx_HTTkWfGgR9S_a_jiafQa1qEYDvO_qqr9V0c1opWrwEWXWvlRorXyPbowOhzH7zqiQpaiuMN5Gq6lOk96xYhn3pd75beTkBuJzhmtMuSqjLQloIOtCekgiqblQ4wFdCkMjelzK-P-2p5bZaWWXTIjEboDTR2vMeSBQ622Pir3fVOTaqpFkaXIFQCKYLTYs8VYjnHAE7VVANeKaJPJB_pj2Ngkc0K7blrRzp0WhaclWofmRNsW2Q3qej1c8EP4w835utjCtyAYHPuvdHD-X8FF2snntr4FG_33ZPN8k3M_xEZkSHlADLeXV-GQqhDEQ_kGZ1739QX9b-6k19QdvxRiMDMZIWQniQH71YZNLcmv4RJHLQM0VDi-irH3rKYGxYCX3CZ5je386SfsKz5MsaSXq0kYnte8J_RsD0rw_okGQ_IEg-V8t3hutFNuyfGBoz5uj5vL9BgCMkWg_k5dcqUBP3ZACiHT4y73EiOwFXwdoc4sPrjFu-CagzM3TnyQCRCJaICczy-JPSTXhRBCfXJDb1xueZQl-pUwW7Cg4a9zJC6v8r3BN1piocWuAjtC2W0Jt0n3z08Kmu8zk7OnWn8KuzwIEOVt6Bxu_UHJXIBWF6ZDZCSA_uYLaHmL6LeswvJw7IRAsVW1PYNfyqVkJ0GAsdOcCumoOY2DQ"
BOOK_ID = "3866026"
RECAPTCHA = "03AFcWeA5frMvhJ257qcHPYmU58aksp3CviRYMMCBx2MlWgh5EQV7EtmZtuTOCr6NB15WXQ0vffIi08PN3COOkVlw2nZBRpLsa7f52M8TVLyeRZlLoqQh9f5x6UItBKSjnEqD55-Clmsy4EIMjpsmcGCr-bLAPWjpzuP5vvZyTrfrS9zXOAthLiv_rSkI305aMKF0va-kOBB1C3yacXCcHV0TZvghAFOx7okK9gld1l2hhyKCimD09F0hZ9yN6liwwoBQeuR2w8MZi98E4lKiVxMg99grZn9V0CrDUj2cW5ShayK5u-3AEhUvxt10Rd16tnxQ4-ujUvy0L4bCLolPqUkjfcRWrE6fc1lNCAhdDD4B_Lx2U9bz8z1uYrsYk3-CE7ApgyU8CFcERIORixOy5ehFELQsrHsh_ftaIIsqOL707f9EiuCxnjZCWs__HN7i7LvMOnt_WvHFCBRWcIWFCYi0XtohZWLVl-d4bs55YlneHXE5NBmhNf4E4gZzkHBGUqQtByV5S5DJTepThL6BVFdfcn4mcYmh8KFKOYBSBNr1Rx5g2sXsln54tp1-MWPpqPXGaWaSyj8Ei"
PUPPETEER_THREADS = 50
tokenIndex = 0
def init_book_delivery():
while True:
try:
ws = websocket.create_connection("wss://api-ws.perlego.com/book-delivery/", skip_utf8_validation=True, timeout=30,  sslopt={"cert_reqs": ssl.CERT_NONE, "check_hostname": False})
except Exception as error:
print(f'init_book_delivery() error: {error}')
continue
break
time.sleep(1)

ws.send(json.dumps({"action":"initialise","data":{"authToken": AUTH_TOKEN, "reCaptchaToken": RECAPTCHA, "bookId": str(BOOK_ID)}}))

return ws

class merged_chapter:
def init(self):
self.merged_chapter_number = 1
class chapter:
def init(self):
self.page_id = 1
	self.contents = {}

download pages content
while True:
chapters = {}
contents = {}
page_id = None
ws = init_book_delivery()

init_data = {}

while True:
	try:
		data = json.loads(ws.recv())
	except Exception as error:
		print(f'download error: {error}')
		ws = init_book_delivery()
		continue

	if data['event'] == 'error':
		sys.exit(data)

	elif data['event'] == 'initialisationDataChunk':
		if page_id != None: # we're here because ws conn broke, so we can resume from last page_id
			ws.send(json.dumps({"action":"loadPage","data":{"authToken": AUTH_TOKEN, "pageId": page_id, "bookType": book_format, "windowWidth":1792, "mergedChapterPartIndex":0}}))
			merged_chapter_part_idx = 0
			# reset latest content
			contents[page_id] = {}
			for i in chapters[page_id]: contents[i] = {}
			continue

		chunk_no = data['data']['chunkNumber']
		init_data[chunk_no] = data['data']['content']

		# download all the chunks before proceeding
		if len(init_data) != data['data']['numberOfChunks']: continue

		# merge the initialisation content
		data_content = ""
		for chunk_no in sorted(init_data):
			data_content += init_data[chunk_no]

		# extract the relevant data
		data_content = json.loads(json.loads(data_content))
		book_format = data_content['bookType']
		merged_chapter_part_idx = 0

		if book_format == 'EPUB':
			bookmap = data_content['bookMap']
			for chapter_no in bookmap:
				chapters[int(chapter_no)] = []
				contents[int(chapter_no)] = {}
				for subchapter_no in bookmap[chapter_no]:
					chapters[int(chapter_no)].append(subchapter_no)
					contents[subchapter_no] = {}
		elif book_format == 'PDF':
			for i in range(1, data_content['numberOfChapters'] + 1):
				chapters[i] = []
				contents[i] = {}
		else:
			raise Exception(f'unknown book format ({book_format})!')

		ws.send(json.dumps({"action":"loadPage","data":TOKEN_LIST[0]}))

	elif 'pageChunk' in data['event']:
		page_id = int(data['data']['pageId'])

		merged_chapter_no = (int(data['data']['mergedChapterNumber']) - 1) if book_format == 'EPUB' else 0
		number_of_merged_chapters = int(data['data']['numberOfMergedChapters']) if book_format == 'EPUB' else 1

		chunk_no = int(data['data']['chunkNumber']) - 1
		number_of_chunks = int(data['data']['numberOfChunks'])

		chapter_no = page_id + merged_chapter_no + merged_chapter_part_idx

		if contents.get(chapter_no) == None:
			contents[chapter_no] = {}
			chapters[page_id].append(chapter_no)

		if contents[chapter_no] == {}:
			for i in range(number_of_chunks):
				contents[chapter_no][i] = ""

		contents[chapter_no][chunk_no] = data['data']['content']

		# check if all merged chapters have been downloaded
		if not all(contents.get(i) not in [None, {}] for i in range(page_id, page_id+number_of_merged_chapters+merged_chapter_part_idx)): continue

		# check if all chunks of all merged pages/chapters have been downloaded
		if not all( all(chunk != "" for chunk in contents[i].values() ) for i in range(page_id, page_id+number_of_merged_chapters+merged_chapter_part_idx)): continue

		# check if all pages/chapters have been downloaded
		if all(contents[i] != {} for i in [page_id]+chapters[page_id]):

			print(f"{'chapters' if book_format == 'EPUB' else 'page'} {page_id}-{page_id+number_of_merged_chapters+merged_chapter_part_idx} downloaded")
			merged_chapter_part_idx = 0
			try:
				next_page = list(chapters)[list(chapters).index(page_id) + 1]
			except IndexError:
				break
		else:
			merged_chapter_part_idx += 1
			next_page = page_id

		ws.send(json.dumps({"action":"loadPage","data":TOKEN_LIST[tokenIndex-1]}))
		tokenIndex = tokenIndex + 1
		if (tokenIndex == len(TOKEN_LIST)):
			break
break

create cache dir
cache_dir = f'{os.getcwd()}/{book_format}_{BOOK_ID}/'
try:
os.mkdir(cache_dir)
except FileExistsError:
pass
convert html files to pdf
async def html2pdf():
# start headless chrome
browser = await launch(options={
		'headless': True,
		'autoClose': False,
		'args': [
			'--no-sandbox',
			'--disable-setuid-sandbox',
			'--disable-dev-shm-usage',
			'--disable-accelerated-2d-canvas',
			'--no-first-run',
			'--no-zygote',
			'--single-process',
			'--disable-gpu',
			'--disable-web-security',
			'--webkit-print-color-adjust',
			'--disable-extensions'
		],
	},
)

async def render_page(chapter_no, semaphore):
	async with sem:
		page = await browser.newPage()
		await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36')

		# download cover separately
		if chapter_no == 0:
			r = requests.get(f"https://api.perlego.com/metadata/v2/metadata/books/{BOOK_ID}")
			cover_url = json.loads(r.text)['data']['results'][0]['cover']
			img = Image.open(BytesIO(requests.get(cover_url).content))
			img.save(f'{cache_dir}/0.pdf')
			return

		# merge chunks
		content = ""
		for chunk_no in sorted(contents[chapter_no]):
			content += contents[chapter_no][chunk_no]

		# remove useless img (mess up with pdf gen)
		if book_format == 'EPUB':
			match = re.search('<img id="trigger" data-chapterid="[0-9]*?" src="" onerror="LoadChapter\(\'[0-9]*?\'\)" />', content)
			if match:
				content = content.replace(match.group(0), '')

		# reveal hidden images
		imgs = re.findall("<img.*?>", content, re.S)
		for img in imgs:
			img_new = img.replace('opacity: 0', 'opacity: 1')
			img_new = img_new.replace('data-src', 'src')
			content = content.replace(img, img_new)

		# save page in the cache dir
		f = open(f'{cache_dir}/{chapter_no}.html', 'w', encoding='utf-8')
		f.write(content)
		f.close()

		# render html
		await page.goto(f'file://{cache_dir}/{chapter_no}.html', {"waitUntil" : ["load", "domcontentloaded", "networkidle0", "networkidle2"], "timeout": 0})

		# set pdf options
		options = {'path': f'{cache_dir}/{chapter_no}.pdf'}
		if book_format == 'PDF':
			width, height = await page.evaluate("() => { return [document.documentElement.offsetWidth + 1, document.documentElement.offsetHeight + 1]}")
			options['width'] = width
			options['height'] =  height
		elif book_format == 'EPUB':
			options['margin'] = {'top': '20', 'bottom': '20', 'left': '40', 'right': '40'}

		# build pdf
		await page.pdf(options)
		await page.close()

sem = asyncio.Semaphore(PUPPETEER_THREADS)
await asyncio.gather(*[render_page(chapter_no, sem) for chapter_no in contents if not os.path.exists(f'{cache_dir}/{chapter_no}.pdf')])

asyncio.run(html2pdf())
merge pdfs
rel = requests.get(f"https://api.perlego.com/metadata/v2/metadata/books/{BOOK_ID}")
book_title = json.loads(rel.text)['data']['results'][0]['title']
print('merging pdf pages...')
merger = PdfMerger()
contents_value = contents  # Copy the value of 'contents'
contents_type = type(contents)  # Get the type of 'contents'
keys_list = list(contents.keys())
for chapter_no in sorted(contents):
chapter_pdf_path = os.path.join(cache_dir, f'{chapter_no}.pdf')
# merger.append(f'{cache_dir}/{chapter_no}.pdf')

pdf_document = fitz.open(chapter_pdf_path)
first_page = pdf_document[0]
lines = first_page.get_text("text").strip().split('\n')
first_line = lines[0].strip()
second_line = lines[1].strip() if len(lines) > 1 else ""
isFirstLine = True
is_empty_page = True
for page_num in range(pdf_document.page_count):
	page = pdf_document[page_num]
	text = page.get_text("text")
	if text.strip():
		is_empty_page = False
		break
if not is_empty_page:
	if first_line:
		isFirstLine = False
	if isFirstLine:
		chapter_title = first_line
	elif second_line:
		chapter_title = second_line
	else:
		chapter_title = f"Chapter {chapter_no}"
	merger.append(chapter_pdf_path)

pdf_document.close()

merger.write(f"output.pdf")
merger.close()
delete cache dir
shutil.rmtree(f'{book_format}_{BOOK_ID}')
`

As of september 2023 I got it to work. Clone this fork: https://github.com/Legenden84/perlego-downloader_fork

As of september 2023 I got it to work. Clone this fork: https://github.com/Legenden84/perlego-downloader_fork

i tried your version today multiple times but i always ended up with an error, im sure that i followed the instructions correctly:
Traceback (most recent call last):
File "C:\new\perlego-downloader_fork\downloader.py", line 251, in
asyncio.run(html2pdf())
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line 649, in run_until_complete
return future.result()
File "C:\new\perlego-downloader_fork\downloader.py", line 176, in html2pdf
browser = await launch(options={
File "C:\Users\RSSal\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pyppeteer\launcher.py", line 307, in launch
return await Launcher(options, **kwargs).launch()
File "C:\Users\RSSal\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pyppeteer\launcher.py", line 148, in launch
self.proc = subprocess.Popen(  # type: ignore
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\subprocess.py", line 971, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\subprocess.py", line 1440, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
OSError: [WinError 14001] The application has failed to start because its side-by-side configuration is incorrect. Please see the application event log or use the command-line sxstrace.exe tool for more detail

I made the changes in the original code as below and it worked.

Added the code lines in original code file for dealing with "Data"  values only from the code changes suggested by @usama455. I didn't used/adopt the method of copying, creating and merging PDF using mentioned by @usama455 since it's was throwing some errors.

Simply it's was needed to add code for just dealing with passing "Data" values.

As of september 2023 I got it to work. Clone this fork: https://github.com/Legenden84/perlego-downloader_fork

I had success with this fork on Ubuntu 20.04. Although I received download error: The read operation timed out when trying to input more than one loadPage token at a time. I was able to successfully run the downloader for each token (chapter) individually and merged the pdfs after. Thanks for the fix!!

Forgot to mention that im using wsl2. So most Linux distributions should do

As of september 2023 I got it to work. Clone this fork: https://github.com/Legenden84/perlego-downloader_fork

I had success with this fork on Ubuntu 20.04. Although I received download error: The read operation timed out when trying to input more than one loadPage token at a time. I was able to successfully run the downloader for each token (chapter) individually and merged the pdfs after. Thanks for the fix!!

how to use in windoes please help.

@MajidAmin112, consider installing python 3.10 as that's what pillow version natively supports within the requirements file.
I'd also recommend installing python 3.10 from the microsoft store (although you'll have to adjust some commands to specifically include the version, such as pip3.10 install -r requirements and python3.10 downloader.py.
Just installed on a somewhat recent build of windows 11

looks like perlego altered the way it fetch data from the server. now for each chapter in the event
{
    "action": "loadPage",
    "data": "longString"
}

it sends a long string, after some work, i found that this string/token does not expire and often same string/token works for several book
you can filter out this event loadPage and copy data values for each chapter.
 s.
place these strings/tokens in TOKEN_LIST list , modified in this PR : #76
this is sometimes a tiresome process, specially for books with lots of chapters but it works. i was unable to find the source of this string/token so this is the best I've got so far. Feel free to improve this .
Happy reading and Happy Coding :)

Tried the solution you mentioned, but it's still not working.

microsoft store

C:\Users\AHMS\Desktop\perlego-downloader_fork-main>python downloader.py
chapters 1-2 downloaded
Traceback (most recent call last):
File "C:\Users\AHMS\Desktop\perlego-downloader_fork-main\downloader.py", line 249, in
asyncio.run(html2pdf())
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line 649, in run_until_complete
return future.result()
File "C:\Users\AHMS\Desktop\perlego-downloader_fork-main\downloader.py", line 174, in html2pdf
browser = await launch(options={
File "C:\Users\AHMS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pyppeteer\launcher.py", line 307, in launch
return await Launcher(options, **kwargs).launch()
File "C:\Users\AHMS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pyppeteer\launcher.py", line 168, in launch
self.browserWSEndpoint = get_ws_endpoint(self.url)
File "C:\Users\AHMS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pyppeteer\launcher.py", line 227, in get_ws_endpoint
raise BrowserError('Browser closed unexpectedly:\n')
pyppeteer.errors.BrowserError: Browser closed unexpectedly:

I have solved this problem.
I use chrom browser to gather a lot of token.

@omoroiyan can you please explain me how you get all tokens with short way ?

I'm interested too for the short way to take chapter tokens. Thx
Il mar 19 set 2023, 04:27 omoroiyan @.***> ha scritto:

I have solved this problem.
I use chrom browser to gather a lot of token.
—
Reply to this email directly, view it on GitHub
https://github.com/evmer/perlego-downloader/issues/67#issuecomment-1724738556,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOH5X2EJRKSMOE5425AFENLX3D7I3ANCNFSM6AAAAAAXP6VOTU
.
You are receiving this because you commented.Message ID:
@.***>

downloader.zip
You should laod  all page, because each page have different token
We need get token but if your book have many page it is very hard work.

You need copy har after you paste txt. than you use detaoutput that can get all token but you need find feature of all token (for example: all token start and end word are same )
You need paste your token in token list.
there method is not smrat but i think it will work.

@omoroiyan first we have to browse and open all chapters ?

yes all page, because each page have token. This program means downloud each page by useing token of each page.
Therefor if you have 100 pages , you need 100 tokens

@omoroiyan  so i downloaded your file and now i just added auth token in downloader.py and recapcha and book id and then i have to run downloader.py ?

@omoroiyan  AI can help us to find short way to get all chapter tokens.

@jaan143 yes this is same to perlego-downloader, but it don't have token list. therefor i add token list and detaoutput. What it does is basically the same as the original program.

Why I am receiving this error.

How do Iff

looks like perlego altered the way it fetch data from the server. now for each chapter in the event
{
    "action": "loadPage",
    "data": "longString"
}

it sends a long string, after some work, i found that this string/token does not expire and often same string/token works for several book
you can filter out this event loadPage and copy data values for each chapter.
 s.
place these strings/tokens in TOKEN_LIST list , modified in this PR : #76
this is sometimes a tiresome process, specially for books with lots of chapters but it works. i was unable to find the source of this string/token so this is the best I've got so far. Feel free to improve this .
Happy reading and Happy Coding :)

Have you tried this? the values of load page data changes every time when new page/chapter is opened. Tried to run the code but error remained same. First one needs to understand the changes implemented by the API. Seems approach to be changed as well then code it to do the task.

Yes, the code works, i am using it.you need to copy data values for all pages/chapters and copy them in the TOKEN_LIST array . on refresh or reloading the same page, the token changes but the previous one also works

Hi can you guide me where to put that code. I am bit confuse?

sure @dhakalanamol . you need to pull code from this PR #67 (comment)
and put the tokens in this list TOKEN_LIST

you only need to add tokens for different chapters and not pages, code will do the rest

How do i get the tokens for chapters only? When I start scrolling these values get loaded for every page. And, if I have to copy the token for every page, is there a way to do this automatically? I have a 1500 page book which I really want to download.

How do Iff

looks like perlego altered the way it fetch data from the server. now for each chapter in the event
{
    "action": "loadPage",
    "data": "longString"
}

it sends a long string, after some work, i found that this string/token does not expire and often same string/token works for several book
you can filter out this event loadPage and copy data values for each chapter.
 s.
place these strings/tokens in TOKEN_LIST list , modified in this PR : #76
this is sometimes a tiresome process, specially for books with lots of chapters but it works. i was unable to find the source of this string/token so this is the best I've got so far. Feel free to improve this .
Happy reading and Happy Coding :)

Have you tried this? the values of load page data changes every time when new page/chapter is opened. Tried to run the code but error remained same. First one needs to understand the changes implemented by the API. Seems approach to be changed as well then code it to do the task.

Yes, the code works, i am using it.you need to copy data values for all pages/chapters and copy them in the TOKEN_LIST array . on refresh or reloading the same page, the token changes but the previous one also works

Hi can you guide me where to put that code. I am bit confuse?

sure @dhakalanamol . you need to pull code from this PR #67 (comment)
and put the tokens in this list TOKEN_LIST

you only need to add tokens for different chapters and not pages, code will do the rest

How do i get the tokens for chapters only? When I start scrolling these values get loaded for every page. And, if I have to copy the token for every page, is there a way to do this automatically? I have a 1500 page book which I really want to download.

Each token should normally correspond to one chapter only. I thought the same as you at first, but turned out I had only a dozen tokens in the end. You should not have 1500 tokens is what I want to say.

Weird, I tried scrolling through the page and it was generating a lot of tokens, one for every page, at the end I had like 400 of them cause I didn’t finish scrolling

@omoroiyan and @Legenden84: I have tried both of your versions, double checked the three variables and only used one token for a start, but I still get the "code 18" error each time I try.
Is there any way to get a more detailed reason for what might have gone wrong here (I'm on Mac using python 3.11)?
In case it's relevant, I'm trying it with this ePub book:
https://ereader.perlego.com/1/book/2194279/12
Thanks!

I would assume Its because you are using Mac. The requirements file i have is for a Linux system.
To solvey it.

Create a new ven.
Run the script.
2a) When you get Module Not Found. Just install that Module. Keep doing this until that error does not persist anymore
2a*) Some packages will not work because this script only works with older release of the. To sort this try and install some ov the versions i have pinned in my requirements.txt

Thank you for your prompt reply so early on a Sunday morning :)!
I have no troubles installing all the modules (pip3 install -r requirements.txt completes without errors after installing all the necessary requirements), it's just that I get the same error as mentioned above after the script is running for maybe 10 seconds or so ({'event': 'error', 'data': {'message': 'An unexpected error occurred.', 'code': 18}}).
Since it seems to have nothing to do with the installed modules, I'm not sure whether a new env would change anything here?
As I'm a develope myself, I know that one has a thousand other things to do, but if you could just try and see if your script works on the link mentioned above for just the first loadPage-Token, I would know that I have to search for the problem in my setup. But it would save me a lot of time if it was clear that this approach is not working (at least with this book/link).
Thank you!

I would assume Its because you are using Mac. The requirements file i have is for a Linux system.
To solve it.

Create a new ven.
Run the script.
2a) When you get Module Not Found. Just install that Module. Keep doing this until that error does not persist anymore
2a*) Some packages will not work because this script only works with older release of the. To sort this try and install some ov the versions i have pinned in my requirements.txt

@Legenden84 hello, sorry, do you know why it's not working in Termux? I have errors with Pillow and pymupdf installation. Thanks

I was having this same error, so I decided to make a Chrome extension for automation. If anyone is interested, here it is:
https://github.com/GladistonXD/perlego-download/

@GladistonXD: Thanks, works fine! The only thing that did not work were the links in the downloaded ePub, but I can easily live with that, so thanks again!

@GladistonXD: Thanks, works fine! The only thing that did not work were the links in the downloaded ePub, but I can easily live with that, so thanks again!

Hello, For me no:(
Some pages are missing in the html file.
Can you try with ebook I am trying to save please? telegram @ProseccoDoc

I was having this same error, so I decided to make a Chrome extension for automation. If anyone is interested, here it is: https://github.com/GladistonXD/perlego-download/

I translated this fully to English (US) if anyone else is interested. https://github.com/UberGuidoZ/perlego-download-extension

@UberGuidoZ Thanks, I'll gladly check it out :)!


test this please

chore(filters): change develop version API

Build FAILED. See http://<IP_ADDRESS>:8080/job/pull-request/3/ for more details.

Build FAILED. See http://<IP_ADDRESS>:8080/job/pull-request/4/ for more details.

The following instances are being terminated: https://elasticbox.com/services/instances/i-l3bxxm

Build FAILED. See http://<IP_ADDRESS>:8080/job/pull-request/5/ for more details.

The following instances are being terminated: https://elasticbox.com/services/instances/i-6fl2zf

Build FAILED. See http://<IP_ADDRESS>:8080/job/pull-request/6/ for more details.

Build FAILED. See http://<IP_ADDRESS>:8080/job/pull-request/7/ for more details.

The following instances are being terminated: https://elasticbox.com/services/instances/i-w1ylru

Build FAILED. See http://<IP_ADDRESS>:8080/job/pull-request/8/ for more details.

Build FAILED. See http://<IP_ADDRESS>:8080/job/pull-request/9/ for more details.

The following instances are being terminated: https://elasticbox.com/services/instances/i-ggt6hq

Build FAILED. See http://<IP_ADDRESS>:8080/job/pull-request/10/ for more details.

Build FAILED. See http://<IP_ADDRESS>:8080/job/pull-request/11/ for more details.

Build FAILED. See http://<IP_ADDRESS>:8080/job/pull-request/12/ for more details.

The following instances are being terminated: https://elasticbox.com/services/instances/i-qyiph2

Build FAILED. See http://<IP_ADDRESS>:8080/job/pull-request/13/ for more details.

The following instances are being terminated: https://elasticbox.com/services/instances/i-x6l7jj


CDN enablement

See the commit for the reasons.
Also, I wanted to migrate the paper links to the CDN, but mobisocial.stanford.edu is down...

Pull Request Test Coverage Report for Build 155

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.01%) to 54.349%

Totals

Change from base Build 153:
-0.01%

Covered Lines:
4164

Relevant Lines:
7098

💛  - Coveralls


branch to test a different operator

Signed-off-by: Itxaka<EMAIL_ADDRESS>

cc @juadk closing this :D


fix: fix wrong test button color on custom domain

Summary

Closes: #4560
How to test

Go to a custom domain not using a custom skin
The text button color should be correct for the primary button

Note
I am not able to test it with a true custom domain, and just test it by chaning the return value from function

Can you re-create a PR like this one https://github.com/snapshot-labs/snapshot/pull/4557
and I will base this PR on your new PR

Here you go https://github.com/snapshot-labs/snapshot/pull/4562

Base branch updated


Updated .travis.yml

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@9eecc27). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #12   +/-   ##
=========================================
  Coverage          ?   72.86%
=========================================
  Files             ?        5
  Lines             ?      199
  Branches          ?        0
=========================================
  Hits              ?      145
  Misses            ?       54
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9eecc27...cdc400f. Read the comment docs.


Allow installing CA from a url in install_ca

Allow downloading a certificate file directly into
/etc/pki/ca-trust/source/anchors/ in the zuul reproducer, whitout it
being part of an rpm. This change also introduces a small refactor of
the reproducer to separate the rhos_release setup and the CA
installation.
As a pull request owner and reviewers, I checked that:

[x] Appropriate testing is done and actually running
[x] Appropriate documentation exists and/or is up-to-date:

[x] README in the role
[x] Content of the docs/source is reflecting the changes

@cjeanner @viroel I've updated the PR. The change to install_ca to pull from ca was simple enough, but I also introduced a new role to do the same job that was done by the downstream hook. I'm not sure I covered all cases there, please take a look when free

recheck

/approve

/lgtm

recheck

recheck

recheck


Implement or

DO NOT MERGE
Using a separate branch to incrementally bring along changes that make sense - ORs bacame too messy to get in one go :(
(round 3)

https://github.com/dvargas92495/roamjs-query-builder/compare/main...mdroidian:roamjs-query-builder:implement-or-things-b


Add all the hosting found here.

http://en.wikipedia.org/wiki/Endurance_International_Group

People can submit them overtime, but I'm not gonna try and do all of that. :)


Clarify "identity expired" error messages

Peer and Orderer have several "identity expired" error messages.
Clarify error messages to indicate which identity has expired.
Signed-off-by: David Enyeart<EMAIL_ADDRESS>

@Mergifyio backport release-2.3

@Mergifyio backport release-2.2


Weird Android notification

I tried out Panphy on my Pixel via chrome and it invited me to turn it into an "app" and put it on one of my screens. Which I did, and it works just fine. This is not exotic, I have a few other "Apps" that are similar Chrome things.
Anyhow, once a day or so I get the attached notification. No harm in it, but also not something I see from any of the other "Chrome" apps.  Low-priority but I thought you might be interested.
(If you tap the notification it goes away.)

Did a little googling:

Why "Tap to copy the URL for this app" notification appears for my PWA every time I open it
Issue 1066658: "Tap to copy the URL for this app" notification

Seems rather odd… (the above issue became wontfix 😅). Currently, Phanpy doesn't have permission to show notifications yet.

Maybe I missed this, I never saw that on my Pixel.... OTOH I get one notification item per notification (not grouped) and they don't go way then I read them (that could be related to #176 or #270, although I feel the later should apply to the home feed only not notifications), so I have to dismiss them manually.