Commit Graph
32 Commits
Author SHA1 Message Date
Jordan Petridis 208f0c248d Update rest of the dependancies 2018-09-13 15:47:40 +03:00
Jordan Petridis b40c12efbd Pipeline: Avoid spamming stderr when not needed
This commit add a new DataError Variant for feeds that return 304.
Its expected behaviror and the current API of Source::into_feed
is kinda limiting the return type to make it easier to handle.

Up till now 304 was returning an Error to early return. Ideally
Source::into_feed will return a Multi variant Result Enum.

example:

enum FeedResult {
    Ok(Success(feed)),
    Ok(NotModified),
    Err(err),
}

Hopefully in a refactor in the near Future™

Till then we will just have to match and ignore
DataError::FeedNotModified.
2018-09-08 20:17:47 +03:00
Jordan Petridis 3c5ddad133 DataError: Improve the FeedRedirect variant 2018-09-08 20:17:47 +03:00
Jordan Petridis 1ca4cb40dd Pipeline: clear the imports of fututres 2018-09-08 20:17:47 +03:00
Jordan Petridis e633fa41ac Pipeline: Minor formatting improvment 2018-09-08 20:17:47 +03:00
Jordan Petridis dac303e33b Pipeline: Use a custom tokio threadpool
This reverts commit e64883eecb
and 40dd2d6923

Seems like core.run() returns once its done even if there
are still tasks in the Runtime underneath. A way to solve that
would be to call the shutdown_on_idle method.

We need ownership of the threadpool in order to invoke
`shutdown_on_idle` method but core.runtime only returns a
referrence so we need to create our own threadpool.
2018-09-08 20:17:46 +03:00
Jordan Petridis a991d9f512 DataError: Remove unused error variant
This was added due to Threadpool::spawn returning errors, but its
no longer used.
2018-09-08 20:17:46 +03:00
Jordan Petridis 674b3b54dc Pipeline: reuse the prexisting runtime executor
Instead of creating our own threadpool, we should reuse the executor
of the tokio::runtime::Runtime that backs the tokio::reactor::Core.
2018-09-08 20:17:46 +03:00
Jordan Petridis 09948845ca Pipeline: Remove use of clone! macro 2018-09-08 20:17:46 +03:00
Jordan Petridis 0b8a0695f7 Pipeline: Use tokio threadpool to index feeds 2018-09-08 20:17:46 +03:00
Jordan Petridis a7c95d5718 Pipeline: Remove dependancy on rayon_futures
This requires a RUSTFLAG to be set before hand for rayon to build.
This brakes a lot of tools like rls and clippy by default and
require special configs for itnegration.

Additionally, rayon_futures is still 0.1 and not much work seem
to have gone into it. Ideally it should be replased with the tokio
runtime/threadpool.
2018-09-08 20:17:46 +03:00
Jordan Petridis 8a05597e52 Feed: Remove another unnecessary wrapper 2018-09-08 20:17:45 +03:00
Jordan Petridis 145d45f800 Feed: Remove unnecessary function wrapper 2018-09-08 20:17:45 +03:00
Jordan Petridis 3f509f44a1 Update dependancies. 2018-08-30 20:45:53 +03:00
Jordan Petridis aa1d0161d3 cargo fmt 2018-08-27 21:08:29 +03:00
Jordan Petridis 79ac3b9700 Update dependancies 2018-08-18 18:33:26 +03:00
Jordan Petridis 7a3178896b Remove criterion
Haven't been able to use it effectivly with futures. Maybe will
revisit it again at a later time. For now it just adds extra
build time.
2018-08-18 18:02:28 +03:00
Jordan Petridis 89b99614a0 Refactor the tests to use ? operator
As of ructc 1.27 #[test] can return errors. This improves a great
deal the ergonomics.
2018-08-18 17:02:31 +03:00
Jordan Petridis 3abe0803d6 Source: Remove reduntant save() call
Source::clear_etags does a save already, so it's not needed to call
it twice.
2018-08-17 09:57:49 +03:00
Jordan Petridis 489e8aa4b3 Source: Improve http redirections handling
We should follow all redirects, and update the Source uri for
301, 302 and 308 codes.

Closes #85
2018-08-17 07:09:12 +03:00
Jordan Petridis 49aff9f22e Source: Only save Etag headers upon succesful requests
Additionally clear the Etags if the returned code is not 200 or 304.
Just to be extra safe. This is not as clean as it should, as this is
a temporary workaround until the API is reworked.

Related to #64
2018-08-17 05:36:34 +03:00
Jordan Petridis cc1a5783fd App: Do not update the db if its empty
If the source table is empty skipp the database refresh.
2018-08-14 15:19:31 +03:00
Jordan Petridis 03754c56c6 Fix the test-suite 2018-08-14 13:40:37 +03:00
Jordan Petridis 471f6ff93b Source: Remove ignore_etags option
This is never used anywhere else apart from the testsuite. Instead
of ignoring etags we should instead not save them if the feed does
not return 200 or 304. See #64.
2018-08-14 13:40:33 +03:00
Jordan Petridis c53ad56a6d Remove TODOs and FIXMEs
They are either no longer relevant or just forgotten
2018-08-14 13:40:28 +03:00
Jordan Petridis f6c7731377 Fix Rust 2018 edition warnings 2018-08-05 11:24:20 +03:00
Jordan Petridis f695ba4605 cargo fmt 2018-08-02 07:10:22 +03:00
Jordan Petridis 132c9bbdff ShowModel: Remove dead unused methods. 2018-07-30 23:23:58 +03:00
Jordan Petridis cc0caff8d0 EpisodeModels: Remove unused methods. 2018-07-30 23:20:10 +03:00
Jordan Petridis 83c44aa12c Replace links again 2018-07-26 07:41:43 +03:00
Jordan Petridis 53be091a31 Replace links 2018-07-25 03:26:41 +03:00
Jordan Petridis 04c68ba013 Initial massive renaming. 2018-07-25 03:26:35 +03:00